What the Latest Updates to Amazon Rekognition Mean for You

Lather. Rinse. Repeat.

A day in the life of an AWS developer often goes a little something like this: Wake up, brush your teeth, read the day’s AWS news, decide how the news affects your code, and start refactoring.

Every. Single. Day.

I’ve brought up this pattern before, when AWS introduced event triggers for SQS. The cycle doesn’t change, but the AWS news does. Today, it’s about Amazon Rekognition.

What is Rekognition?

Amazon Rekognition is a fully managed service that allows application developers to add image and video analysis to their applications. Developers can use Rekognition to identify objects, extract text, and filter unsafe content application images and video.

Rekognition is powered by advanced deep-learning algorithms that are trained, served, and updated by Amazon. It provides machine learning capabilities to applications and services via a highly available and secure RESTful API.

A key feature of Rekognition is object labeling. Given an image, Rekognition uses deep learning algorithms to detect and label common objects, like people, airplanes, animals, and many other object types.

Common use cases for object labeling include security and access control, quality monitoring, and metadata tagging.

What’s new with Rekognition?

Last week’s announcement introduced several new object labeling capabilities to Rekognition: bounding box information, hierarchical metadata, and improved labeling accuracy.

Bounding boxes

Rekognition now returns bounding box information for objects, allowing applications to determine precise locations of objects in an image, count the number of objects detected, and calculate relative size of objects within an image. These new capabilities allow application developers to include richer, user-tailored features.

Here’s an example of the bounding box information returned:           

        "BoundingBox": {
                       "Height": 0.47562453150749207,
                       "Left": 0.03361765667796135,
                       "Top": 0.2657702565193176,
                       "Width": 0.9567702412605286
                   },

Before bounding boxes were included with labeled objects, applications didn’t know where objects were located in an image or if there were more than one of the same object in the image.

Hierarchical metadata

Rekognition now returns hierarchical metadata for objects, allowing applications to organize object labels more efficiently and accurately. For example, the label “Airplane” has ancestors that include “Vehicle” and “Transportation”:

           "Name": "Airplane",
           "Parents": [
               {
                   "Name": "Aircraft"
               },
               {
                   "Name": "Vehicle"
               },
               {
                   "Name": "Transportation"
               }

With this new metadata, you can now group and filter objects with ease.

Improved accuracy

While bounding boxes and hierarchical metadata provide additional context and enhanced capabilities, improved accuracy is the hidden jewel of this update. It demonstrates one of the greatest benefits associated with AWS managed services like Rekognition: We don’t have to lift a finger to benefit from it.

Behind the scenes, Amazon is continuously improving the algorithms and models associated with their managed machine learning services and transparently updating the underlying infrastructure.

Amazon hires data scientists and AI/ML experts so I don’t have to.

A practical application

@AircraftML is a Twitter bot built using AWS AI and ML services, designed to identify and classify aircraft in an image. Tweet a picture to it, and it will let you know if the picture contains a Boeing 767, an Airbus A380, or any number of other modern commercial aircraft. Try it out.

Behind the scenes, @AircraftML uses a self-managed (meaning I do all the work) object detection system based on a Single Shot Detector (SSD). SSDs are useful in that they return bounding box information, which allows @AircraftML to effectively detect and classify multiple aircraft in an image.

Periodically, I have to update the machine learning model used by @AircraftML’s object detection system and redeploy the system. Not particularly difficult, but it’s an extra step in a process that takes additional time and creates an opportunity for mistakes.

The introduction of bounding box information to Rekognition’s object labels means that I can replace my self-managed SSD with Rekognition. Using Rekognition, I’m able to reduce the overall overhead and burden of managing the @AircraftML system.

The bottom line

Sometimes it feels like keeping up with the pace of innovation at AWS is a full-time job. And the reality is, for large, complex systems, it can become a very time-consuming, albeit extremely important, part of managing your AWS infrastructure.

Ultimately, replacing self-managed systems with AWS managed equivalents allows you to spend more time focusing on your applications and less on your infrastructure.

Now if you’ll excuse me, I have code to deprecate.