Open Source

For an internal project, we decided to develop a machine learning model to perform an image recognition task on a large dataset of images, based on a defined set of categories to which an image can belong. Initially, we didn’t have a labeled set of images to train the model….

If you’ve ever scripted in Ruby before, you’ve more than likely encountered deeply nested arrays and hashes. These nested structures often come from Ruby’s JSON parser, but Ruby itself doesn’t have effective methods for dealing with them. Specifically, sorting these structures: the standard routine only shallow sorts the top layer…

Last summer, our team added support for importing Adobe InDesign files into Lucidpress, a cloud-based publishing app. Specifically, the app now supports IDML files, which consist of XML files packaged up in a .zip file. The process of conversion involves: Parsing the XML from the IDML file into a model…

How Apt transports work, how we wrote one for S3, and how you can write your own. Debs in S3 Lucid Software uses Debian packages (debs) for packaging and installation. Custom scripts download the debs from a private AWS S3 bucket. As much as we loved the quirks of homegrown…

We’ve all been there. After hours or even days of tirelessly trying to narrow in on the cause of some small bug, you finally close in on a particular section. You know the problem is in there somewhere, but the code is spaghetti—an impossible-to-read, jumbled mess of logic that does…

My thanks go out to Grant Klopper from The Guardian. Last week, he validated a change I made to the Play! Framework back in December. After upgrading from Play! 2.2 to 2.3, Grant noticed dramatic changes in response times and memory – both for the better. The changes were so awesome that Grant…

At Lucid, we deal with data—a LOT of it. Ever since we migrated our code to the Play! Framework, we’ve been leveraging the Anorm database access library provided by the framework. Being able to write our own SQL queries gave us a lot more control than using an ORM, and…

Piezo is a system to operate and manage a Quartz Scheduler cluster. It is made up of the Worker and the Admin. One of the main features needed before an official public release of Piezo was the ability to add, edit, and delete jobs from the admin interface. The bulk…

If I should call you up, invest a dime And you say you belong to me and ease my mind Imagine how the world could be, so very fine So happy together -G. Bonner & A. Gordon, “Happy Together”   What’s Missing from Quartz If you need scheduled tasks on…

[Editor’s note: This post was originally published on the Architected Availability blog and is republished with permission from the author] I am currently employed as the chief architect at Lucidchart. In my spare time (literally) I am also the ops guy. All of our servers are running on Amazon’s EC2 cloud. Using the…