Articles by Brian Campbell

Using Apache Airflow's Docker Operator with Amazon's Container Repository thumbnail

Last year, Lucid Software’s data science and analytics teams moved to Apache Airflow for scheduling tasks. Airflow was a major improvement over our previous solution—running Windows Task Manager on analyst’s laptop and hoping it worked—but we’ve had to work through a few hurdles to get everything working. One interesting hurdle…

How to Debug Disk Full Errors in Redshift thumbnail

When working with Amazon’s Redshift for the first time, it doesn’t take long to realize it’s different from other relational databases. You have new options like COPY and UNLOAD, and you lose familiar helpers like key constraints. You can work faster with larger sets of data than you ever could…

Disaster-proof A/B Testing, Pt. 1: Managing metrics thumbnail

Almost every product change made here at Lucid is first validated by an A/B test. Over the years, we’ve made a lot of mistakes while running A/B tests and writing the code for managing them. In this series of posts, we’ll be sharing what we learned from these mistakes and…