Articles by Brian Campbell

Using Apache Airflow’s Docker Operator with Amazon’s Container Repository

March 22, 2019 Brian Campbell 5 Comments

Last year, Lucid Software’s data science and analytics teams moved to Apache Airflow for scheduling tasks. Airflow was a major improvement over our previous solution—running Windows Task Manager on analyst’s laptop and hoping it worked—but we’ve had to work through a few hurdles to get everything working. One interesting hurdle…

How to Debug Disk Full Errors in Redshift

January 31, 2018 Brian Campbell 2 Comments

When working with Amazon’s Redshift for the first time, it doesn’t take long to realize it’s different from other relational databases. You have new options like COPY and UNLOAD, and you lose familiar helpers like key constraints. You can work faster with larger sets of data than you ever could…

Disaster-proof A/B Testing, Pt. 1: Managing metrics

June 23, 2016 Brian Campbell 0 Comments

Almost every product change made here at Lucid is first validated by an A/B test. Over the years, we’ve made a lot of mistakes while running A/B tests and writing the code for managing them. In this series of posts, we’ll be sharing what we learned from these mistakes and…

Disaster-proof A/B Testing, Pt. 2: Ensuring a consistent experience (on 750+ arms)

June 22, 2016 Brian Campbell 0 Comments

When setting up an A/B testing framework, pay special attention to how users are assigned to different arms of a test (and which metrics you track). If you aren’t careful, you’ll give the same user a different experience each time they go through a tested flow. This creates a problem…