Building an Aircraft Classifier with Google Cloud AutoML Vision: Part 3

Have you been thinking about checking out Google’s new Cloud AutoML Vision service? Are you interested in what AutoML can do for you and where you should use it? Then read on!

This article is the third in a series about the Google Cloud AutoML Vision service. This series explores AutoML Vision features and capabilities and how they can be used to build a working image classification system.

In the first part of this series, I introduced the Google Cloud artificial intelligence (AI) family of services and then discussed the Cloud AutoML services, focusing on AutoML Vision, the star of this series.

In the second part of this series, I described how to build and train an accurate machine learning model with AutoML Vision. The key to success was collecting and curating high-quality training images for the aircraft classification system.

In the final installation of this series, I’ll walk through the process of incorporating the classification model you built in the previous post into a working system that allows users to submit images for classification, all using just a few off-the-shelf Google Cloud services, including Google Cloud Storage and Google Cloud Functions.

Let’s go! Just follow the simple steps outlined below.

1. Set up the command line utilities

For this example, you’ll be building a framework for your object classification system using the Google Cloud command line interface (CLI), gcloud, and the utility application, gsutil.

gcloud is a command line tool for managing your Google Cloud account and infrastructure. If you haven’t done so already, install and initialize the Google Cloud SDK on a Google VM, your own computer, or some other system.

gsutil is a Python tool that lets you manage Cloud Storage buckets and objects from the command line. Follow the instructions in the gsutil Quickstart guide before proceeding.

Note: The AutoML Vision service is currently in beta release and subject to change, as are the following descriptions, screenshots, and instructions associated with AutoML.

2. Review the architecture

Before we get too far, let’s take a quick look at what we’ll be building. It’s really quite simple:

Cloud Storage

Google Cloud Storage provides highly available, durable, and secure object storage for your applications and can be configured to host static websites for domains that you own.

You’ll use a static website created using Cloud Storage to host the web assets for your object classification system. These assets include your website’s HTML, CSS, and JavaScript files.

Cloud Functions

Google Cloud Functions provide a scalable, available, and resilient way to run code in the cloud without having to manage servers. It can be easily configured to provide controlled interfaces to other Google Cloud services, like Cloud AutoML Vision.

You’ll create a serverless function with Cloud Functions to route browser requests to your model hosted by the Cloud AutoML Vision service.

Cloud AutoML Vision

Google Cloud AutoML Vision provides the ability to train high-quality machine learning models with minimum effort or expertise simply by collecting and curating training data.

You’re going to use the AutoML Vision model you created in part two of this blog post to classify objects in images submitted through the website.

3. Verify your AutoML Model

Before you use your AutoML model, you’ll need to be prepared with the following information:

<region>: the Google Cloud region your AutoML moded is hosted in
<project>: the Google project ID associated with your AutoML model
<model>: the AutoML model ID

You can find this information by navigating to the AutoML Vision Datasets page and selecting your dataset.

You can verify that your model is functioning correctly by testing it from the command line using the REST API provided by Google. Replace the highlighted parameters <project>, <region>, and <model> below with the information you collected above. Replace <image> with the image you wish to test with.

(echo -n '{"payload": {"image":"'; base64 <image>; echo '"}') > request.json
 
curl -X POST -H "Content-Type: application/json" \
 -H "Authorization: Bearer $(gcloud auth application-default \
 print-access-token)" \
 -d @request.json \ https://automl.googleapis.com/v1beta1/projects/<project>/locations/<region>/models/<model>:predict 

The JSON response will contain the best classification for the sample image you provided.

{ "payload": [
   {
     "classification": { "score": 0.9989391 },
     "displayName": "boeing_747"
   }
 ]
}

4. Create your Cloud Function

Note: Creating and using Google Cloud resources will incur cost. A simple object classification system like the one we’re going to build can operate in the GCP free tier, but your experience may differ.

After creating your AutoML model, the next step is to create a Google Cloud function. This function will be used to proxy image classification requests to the AutoML Vision service. The function takes a Base64 encoded JPG image as input, provides it to the AutoML service for classification, and then formats and returns the classification results.

To prepare and deploy the function, execute the following commands:

$ git clone https://github.com/gcpgeek/aircraftml
$ cd function
$ npm install
$ gcloud beta functions deploy aircraftml --trigger-http \
  --timeout 15 --memory 256 \
  --set-env-vars REGION=<region>,PROJECT=<project>,MODEL=<model>

Provide the <region>, <project>, and <model> information for your AutoML Vision model. This will download the necessary dependencies and then create and deploy the function to the Google Cloud. If you haven’t already done so, enable Cloud Functions APIs if prompted.

When your function is successfully deployed, gcloud will return the HTTP trigger URL for the function. Keep this URL handy as you’ll use it next to test the function/model interaction and then later when you build the website.

httpsTrigger:
 url: https://<region>-<project>.cloudfunctions.net/<function>

Testing your cloud function

Now that your function has been deployed, you can test it using a simple curl command and an image. The following commands convert the input file to a JSON request:

cd ..
(echo -n '{"image": "'; base64 <image>; echo '"}') > request.json
 
curl -H "Content-Type: application/json" \
  -d @request.json \
  https://<region>-<project>.cloudfunctions.net/<function>

This command includes the following information:

<image>: the image you’ll use for testing
<region>: the Google Cloud region your Cloud function is located
<project>: the Google project ID associated with Cloud function
<function>: the name of your Google Cloud function

The JSON response will contain an entry for each of the different object classifications in descending order, with the best match at the top.

[{  
  "aircraft":"boeing_747",
  "score":0.9982553124427795
},
{  
  "aircraft":"dornier_328",
  "score":0.00021892409131396562
} ...

5. Build your website

The website will host the static assets associated with your system, including the site’s HTML, CSS, and JavaScript files. This static website will allow you to easily upload images for classification using your AutoML Vision model.

Note: You must have a domain name and have the ability to assign a CNAME with your DNS provider before proceeding.

First, choose a domain name for your website, like aircraftml.xyz.com, assuming you own the xyz.com domain. Once you have a domain name chosen, create a CNAME record with your DNS provider that points your chosen domain name for your website to c.storage.googleapis.com. For example:

aircraftml.xyz.com -> c.storage.googleapis.com

If you need instructions or assistance creating CNAME record, please contact your DNS provider for help.

Next, create a Cloud Storage bucket with the same name as your domain name. When Google receives traffic destined for your domain name at c.storage.googleapis.com, it will direct that traffic to your bucket with the same name.  

 

gsutil mb gs://<your domain name>

If Google is not your DNS provider, you will be prompted to verify ownership of your domain. Follow the instructions provided to do so.

After creating your bucket, update the aircraftml.js file to use your <http function endpoint> when sending requests to your Cloud Function.

cd ../website
sed -i 's|FUNCTION_ENDPOINT|<your function endpoint>|g' aircraftml.js

Note the use of the “|” characters in the sed expression. These characters are required as the endpoint has forward slashes in it.

Then, upload your files to your bucket, make them publicly readable, and set the landing page for your website.

 

gsutil cp index.html gs://<your domain name>
gsutil cp styles.css gs://<your domain name>
gsutil cp aircraftml.js gs://<your domain name>
gsutil web set -m index.html gs://<your domain name>

Finally, make those files publicly accessible.

 

gsutil iam ch allUsers:objectViewer gs://<your domain name>

6. Test your image classification system

With the Cloud Function in place and the website built using Cloud Storage, you can now browse to your website’s domain name and test your system directly. Upload an image to the web page to verify end-to-end connectivity and to verify operation of the AutoML Vision service and your model. Once that’s done, upload a couple of aircraft images and see how accurate your image classifier is.

Note: This architecture is for demonstrative purposes only and should not be considered production ready.

Wrap-up

That’s it. I hope this has been helpful to you and that you’re able to use Google Cloud AutoML Vision (or Natural Language or Translation) in your next project. I found the Google Cloud AutoML Vision service simple to use (by design) and easy to integrate into applications and the benefits measurable.

The repository for the examples referenced in this document is located at https://github.com/gcpgeek/aircraftml.

If you have any questions or comments, feel free to reach out to me at @awsgeek on Twitter.