Get started with TensorFlow

Begin harnessing the power of Google’s open source machine learning library with InfoWorld's hands-on tutorial

1 2 Page 2
Page 2 of 2

Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.

Extracting /tmp/data/train-images-idx3-ubyte.gz

Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.

Extracting /tmp/data/train-labels-idx1-ubyte.gz

Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.

Extracting /tmp/data/t10k-images-idx3-ubyte.gz

Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.

Extracting /tmp/data/t10k-labels-idx1-ubyte.gz

Now that we have the data loaded, we create a model using TensorFlow variables, placeholders, and functions. It is really nothing more than the matrix equation y = Wx + b, plus some setup to hold the tensors (784 is 28 by 28, to hold the pixels of the images; 10 is the number of categories, for the digits 0-9).

Creating a model using TensorFlow variables

Then we define a loss function—the way we define “goodness of fit,” or rather “badness of fit” for the model—and choose the optimizer to use for training.

Defining loss function and optimizer in TensorFlow

As you can read in the code and comments, the loss function is an averaged cross-entropy based on the Softmax (normalized exponential) function, and the training method is Gradient Descent set to minimize the cross-entropy loss function.

Finally, we’re ready to actually run a TensorFlow session. You’ll notice that the training happens batch by batch inside a loop.

Training session in loop in TensorFlow

Now that the training is complete (it should only take a second), we need to test the model and calculate the accuracy:

Testing a TensorFlow model and checking for accuracy

The accuracy output that I got was 0.9109, or 91 percent, which is not really very good as MNIST classifiers go. The point here was to understand the steps, however.

Now you try it. In a terminal, navigate to tensorflow/examples/tutorials/mnist/ in your TensorFlow repository, and run

$ python mnist_softmax.py

Now go read the full tutorial, which goes into more detail. Note that it uses a slightly simpler cross-entropy loss function than the Python code in the repository and explains why the code above is needed in a parenthetical note. I’ll be here when you come back.

Visualizing models in TensorBoard

TensorBoard is a suite of visualization tools for viewing TensorFlow graphs and plotting metrics, along with a few other useful tasks. Before you can use TensorBoard, you need to generate data files from a TensorFlow run.

In the same folder we just used, tensorflow/examples/tutorials/mnist/ in your TensorFlow repository, you’ll find another MNIST classification program, mnist_with_summaries.py. If you read through the code, you’ll find familiar-looking code, as well as code that might be new to you—for example, the use of tf.name_scope to clarify what we’ll see in TensorBoard, and the variable_summaries function:

Attaching summaries to a Tensor in TensorFlow

This, as the comment says, attaches a lot of summaries to a Tensor. If you read through more of mnist_with_summaries.py, you’ll see lots of with tf.name_scope(name) clauses that include calls to variable_summaries(var, name) and other specific tf.<x>_summary functions such as the ones shown above.

Pay attention to the model and optimizer. If you read the code closely, you’ll see some ReLu neurons and an Adaptive Moment Estimation (Adam, a variation on gradient descent) optimizer. You’ll also see the same cross-entropy definition we found in the mnist_softmax tutorial.

Go ahead and run the model:

$ python mnist_with_summaries.py

It’ll take less than a minute. When I ran it, the last few lines of output were:

Accuracy at step 950: 0.9664

Accuracy at step 960: 0.9669

Accuracy at step 970: 0.9671

Accuracy at step 980: 0.9671

Accuracy at step 990: 0.9663

Adding run metadata for 999

Now we can try TensorBoard, specifying the folder where the model saved the logs:

$ tensorboard —logdir=/tmp/mnist_logs/

Starting TensorBoard 23 on port 6006

(You can navigate to http://0.0.0.0:6006)

Once you open that URL in your browser, you’ll see a lot of web server logging going on in your terminal window, and with a few clicks you’ll be able to see the convergence graphs in the events pane:

Convergence graph from a TensorFlow session

With a few more clicks, you can view the graph and zoom in on parts of interest:

Summary on TensorBoard

For a more thorough discussion of how to create summaries for TensorBoard, see the Visualizing Learning tutorial. For some instruction on how to use TensorBoard, see the Graph Visualization tutorial.

Learning TensorFlow: Additional resources 

That should be enough to get you going, but there is a lot more material worth learning. For example, the MNIST data is discussed as part of the MNIST for Beginners tutorial along with the Softmax model. The follow-on tutorial “for experts” covers some of the same material again, faster, but then shows you how to create a multilayer convolutional neural network to improve the results to about 99.2 percent accuracy. The “Deep CNN tutorial”  goes into even more depth, still using MNIST data. If you want to go to the source for MNIST, it lives on a page maintained by Yann LeCun of the Courant Institute. If you want to explore techniques used for classifying the MNIST data, Chris Olah has a visualization page.

The TensorFlow team has produced more learning materials and improved the existing getting started tutorials, including a quickstart for tf.estimator. In addition, a number of third parties have produced their own TensorFlow tutorials. There are now multiple TensorFlow books in print and several online TensorFlow courses. You can even follow the TensorFlow for Deep Learning Research (CS 20) course at Stanford, which provides all the slides and lecture notes online.

Several new sections of the TensorFlow library offer interfaces that require less programming to create and train models. These include tf.keras, which provides a TensorFlow-only version of the otherwise engine-neutral Keras package, and tf.estimator, which provides a number of high-level facilities for working with models—both regressors and classifiers for linear, deep neural networks (DNN), and combined linear and DNN, plus a base class from which you can build your own estimators.

In addition, the Dataset API allows you to build complex input pipelines from simple, reusable pieces. You don’t have to choose just one. As this TensorFlow-Keras tutorial shows, you can usefully make tf.kerastf.data.dataset, and tf.estimator work together.

MNIST is one of the simpler benchmark data sets around for visual classification research, with 70,000 black-and-white 28-by-28 images of the handwritten digits 0 through 9. Another set of images, CIFAR-10, is used to benchmark image-processing techniques by classifying 60,000 RGB 32-by-32-pixel images across 10 categories. The Convolutional Neural Networks tutorial shows you how to build a small CNN for classifying CIFAR-10 images. You’ll want at least one GPU if you’re going to try this model—that will bring the training time down to a few hours. If you have multiple GPUs, you can use them with a variation of the model, cifar10_multi_gpu_train.py.

Beyond image processing, you may want to learn about natural language processing using word embeddings, recurrent neural networks (RNNs), and sequence-to-sequence models for machine translation. Finally, the TensorFlow Model Zoo and the TensorFlow articles in the Google Research blog should be worth your while as starting points for your own models.

Related articles

Copyright © 2018 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2
How to choose a low-code development platform