Review: Google Cloud AutoML is truly automated machine learning

Google’s AutoML lets you create customized deep learning models without any knowledge of data science or programming

Review: Google Cloud AutoML is truly automated machine learning
At a Glance

When you’re trying to train the best machine learning model for your data automatically, there’s AutoML, or automated machine learning, and then there’s Google Cloud AutoML. Google Cloud AutoML is a cut above. 

editors choice award logo plum InfoWorld

In the past I’ve reviewed H2O Driverless AI, Amazon SageMaker, and Azure Machine Learning AutoML. Driverless AI automatically performs feature engineering and hyperparameter tuning, and claims to perform as well as Kaggle masters. Amazon SageMaker supports hyperparameter optimization. Azure Machine Learning AutoML automatically sweeps through features, algorithms, and hyperparameters for basic machine learning algorithms; a separate Azure Machine Learning hyperparameter tuning facility allows you to sweep specific hyperparameters for an existing experiment.

These are good, but Google Cloud AutoML goes to a whole different level and customizes Google’s battle-tested, high-accuracy deep neural networks for your tagged data. Rather than starting from scratch when training models from your data, Google Cloud AutoML implements automatic deep transfer learning (meaning that it starts from an existing deep neural network trained on other data) and neural architecture search (meaning that it finds the right combination of extra network layers) for language pair translation, natural language classification, and image classification.

In each area, Google already has one or more pre-trained services based on deep neural networks and huge sets of labeled data. These may well work for your data unmodified, and you should test that to save yourself time and money. If these services don’t do what you need, Google Cloud AutoML helps you to create a model that does, without requiring that you know how to perform transfer learning or even how to create neural networks.

Transfer learning offers two big advantages over training a neural network from scratch. First, it requires a lot less data for training, since most of the layers of the network are already well trained. Second, it runs a lot faster, since it’s only optimizing the final layers.

Google Cloud AutoML Translation

So, for example, you can train against 1,000 two-language sentence pairs in an hour or two with Google Cloud AutoML Translation transfer learning. The base neural net being customized, NMT, took hundreds to thousands of hours to train from scratch for each language pair, on a large number of CPUs and GPUs. Note that the hourly charge for training a custom translation model is currently $76.

The AutoML Translation Beginner’s Guide explains the basics of what Google Cloud AutoML Translation can do, and why you would use it. Essentially, it refines an existing general translation model for a niche purpose. You don’t need to do any training for general translation of the hundred or so languages that Google already supports, but you would need to run the transfer learning if you want to create a translation network for specialized vocabulary or usage. One example Google mentions is translating time-sensitive financial documents in real time. General-purpose translation won’t always use the correct terms of art for finance.

 Setting up the training for Google Cloud AutoML Translation is a five-step process, as shown in the screenshots below, once you have prepared a file with sentence pairs. I used the 8,720 English-Spanish pairs for app prompts provided by Google in the AutoML Translation Quickstart, formatted as a tab-separated-values file. Google Cloud AutoML Translation also supports the XML-based Translation Memory eXchange (TMX) format for sentence pairs.

You’ll note that there is no option to control the hardware (CPUs, GPUs, TPUs, and memory) used to perform the training. That’s deliberate: The training will use what it needs. There are also no options to control the neural network layers being added to the model, the number of epochs to run, or the stopping criteria.

google cloud automl translation 1 IDG

To load a data set to use for translation training, you only need to specify the language pair, the file to use, and the split among the training, validation, and testing sets. Here I’ve allowed automatic splitting.

google cloud automl translation 2 IDG

Once you’ve uploaded a data set of sentence pairs, you can view a sample. This allows you to validate that the file loaded correctly.

google cloud automl translation 3 IDG

Before you start training, you see a summary of the languages being translated, and of the numbers of sentence pairs in each set.

google cloud automl translation 4 IDG

After you press “Start training” on the previous page, you get to select the base model for the transfer learning and see an estimate of the training time. Press “Start training” here and it will really start.

google cloud automl translation 5 IDG

After the training starts you’ll see this page. You can close it if you don’t want to cancel the training job.

Once the model training is complete, you can view the improvement (if all goes well) in the BLEU score over the base model, and try making predictions with the model. This training took 0.9 hour (less than predicted) and cost $68.34.

google cloud automl translation 6 IDG

After training completes, clicking on the link in the notification email will bring you to this page. The BLEU (Bilingual Evaluation Understudy) score represents the overall quality of the model. The increase of 12 points over the base model is excellent, and the score of 54 indicates a very high quality translation corpus.

google cloud automl translation 7 IDG

This page allows you to test your model by giving it sentences to translate. It displays the results from the base model as well as the transfer learning model. Note that the transfer model in this case gives a result that is direct and uses the informal form “tu” while the base model uses the formal form “su” and includes the polite “por favor” (please).

Google Cloud AutoML Natural Language

The Google Natural Language API takes text and predicts entities, sentiment, syntax, and categories (from a predefined list). If your text classification problem doesn’t fit any of those, you can supply a labeled set of statements and use Google Cloud AutoML Natural Language to create a custom classifier.

To set up AutoML Natural Language for training, you need to source your data, label it, prepare it as a CSV file, and run the training. You can also use the AutoML Natural Language UI to upload and label the data if you prefer.

google cloud automl natural language 8 IDG

To create a data set for natural language classification, provide a comma-separated variables file with the text and labels. If you don’t have such a file, you can defer the creation of the text items and labels to a later time.

google cloud automl natural language 9 IDG

Once you have created a data set you can view it in the workspace.

google cloud automl natural language 11 IDG

On the Train tab you’ll see a display of counts for each class. If the numbers are inadequate in any classes, you can add tagged text items, or you can combine sparsely populated classes into an “other” class.

Once the model training is complete, you can view the precision, recall, and confusion matrix of the model. You can also adjust the score threshold for the desired precision/recall tradeoff. To minimize false negatives, optimize for recall. To minimize false positives, optimize for precision.

This training took 3.63 hours (about as predicted) and cost $10.88.

google cloud automl natural language 14 IDG

This classification is far from perfect, with both precision and recall under 90 percent, and significant misclassifications showing up in the confusion matrix. However, reading some of the text items makes it obvious that not all items unambiguously fit into a single class.

google cloud automl natural language 15 IDG

You can test the model by entering your own text and getting predictions of the class. For this particular sentence the predictions are reasonable.

Google Cloud AutoML Vision

The Google Cloud Vision API classifies images into thousands of predefined categories, detects individual objects and faces within images, and finds and reads printed words contained within images. Google Cloud AutoML Vision allows you to define and train your own list of categories. Some real-life applications include detecting damage on wind turbines from drone photos, and classifying recyclables for waste management.

To set up a Google Cloud AutoML Vision data set you must source at least 100 images for each category, and label them in a CSV file. All of the images and the CSV file need to reside in a Google Cloud Storage bucket.

google cloud automl vision 18 IDG

Once the images have been imported you can view them and edit the classes if necessary. The warning about duplicate files reflects a known issue with the data set. It didn’t seem to matter to the training.

google cloud automl vision 19 IDG

On the Train tab you see a summary of the image counts in each class and the automatic splits among training, validation, and test images.

google cloud automl vision 21 IDG

Once the training is complete, you can see the precision and recall of the model. If they are not adequate you can resume the training.

I set this training up to run for a maximum of an hour, which is free for up to 10 models a month. I was pleasantly surprised to see good results from the free training, and didn’t bother continuing the training to improve the precision and recall.

google cloud automl vision 22 IDG

The Evaluate tab allows you to see the images of true positives, false negatives, and false positives for each class. As the note in the middle of the page suggests, you can improve the results by adding correctly labeled training images that are similar to the false negatives and false positives and training a new model.

google cloud automl vision 23 IDG

Here we can view false negatives and false positives for a different class, roses. Again, you can improve the model by adding similar images to the training set, tagged correctly.

google cloud automl vision 24 IDG

You can test your model on your own images. This image of roses, which I took in a neighbor’s yard, has been correctly predicted.

google cloud automl vision 25 IDG

Just to prove that the model doesn’t predict that all my images are roses, I tested with a photo of tulips that I took at a nearby art museum.

Google Cloud AutoML provides convenient options for performing targeted translations, customized text classification, and customized image classification. Each of these APIs works well if you give it enough accurately labeled data, and takes a lot less time and skill than building your own neural network model or even your own transfer learning model. With Google Cloud AutoML you’re actually creating TensorFlow models, without necessarily knowing anything about TensorFlow, Python, neural network architectures, or training hardware.

There are lots of ways to get the data preparation wrong, but fortunately the three APIs all check for the most common errors, such as having too few or too many exemplars for any category. The diagnostics shown after training give you a good idea of how well your model works, and you can easily tweak the models by adding more labeled training data and re-running the training.

Cost: Google Cloud AutoML Translation: Training costs $76.00 per hour, translation $80 per million characters after the first 500K. Google Cloud AutoML Natural Language: Training costs $3.00 per hour, classification $5 per thousand text records after the first 30K. Google Cloud AutoML Vision: Training costs $20 per hour after the first hour, classification $3 per thousand images after the first thousand. 

Platform: Google Cloud Platform

At a Glance
  • The three Google Cloud AutoML services provide customized language pair translation, text classification, and image classification. They require tagged data for training, but don’t require any knowledge of deep learning or programming.


    • Allows you to create customized deep learning models without knowing any data science
    • Creates language translation models for niche areas
    • Creates custom text classifiers
    • Creates custom image classifiers
    • Runs in the Google cloud


    • Doesn’t run on-premises

Copyright © 2019 IDG Communications, Inc.