Data Science

Using Apache Airflow's Docker Operator with Amazon's Container Repository thumbnail

Last year, Lucid Software’s data science and analytics teams moved to Apache Airflow for scheduling tasks. Airflow was a major improvement over our previous solution—running Windows Task Manager on analyst’s laptop and hoping it worked—but we’ve had to work through a few hurdles to get everything working. One interesting hurdle…