How to use MQTT for IoT messaging

Ordinary Web applications may not need reliable messaging, but the multiplicity of endpoints represented by the Internet of things demands a messaging protocol that measures up

How to use MQTT for IoT messaging

Architectural patterns come and go, but if there's one constant, it's the need for the reliable delivery of messages.

Message queuing may have had its heyday in the client/server world, handling communications with mainframes and gluing together our financial networks, but it never really went away -- despite the fact that direct access to APIs is the hallmark of much modern development.

Direct access to APIs works well for much of what we do on the Web, where a limited number of servers and processes communicate with a back end. Things change, however, when we move to microservices architectures and the cloud -- and particularly when we factor in the Internet of things.

Suddenly, with the Internet of things, we're dealing with systems made up of an unknown number of components, deployed by algorithms, and handling messages sent by thousands, if not millions, of endpoints. The familiar MVC design patterns break down at that sort of scale, so we need to shift to microservice-safe patterns, bring back computer science concepts like actors, and use actor/message patterns to handle how our services communicate with each other and any number of endpoints.

Publish/subscribe to the rescue

That's where message queuing tools come into play, providing reliable ways to deliver data to services using publish/subscribe methods. Microservice actors can subscribe to published connections and then receive messages that trigger appropriate actions. It's a model that works well, providing a way to connect devices to services, ensuring that a service scales and that as new devices are added, their messages get through.

Developed by IBM, MQTT (Message Queuing Telemetry Transport) is a lightweight publish/subscribe messaging protocol, now standardized by OASIS. Initially designed to link large-scale WebSphere MQ systems to SCADA devices, MQTT has evolved to become a general purpose message queuing system that allows many thousands of endpoints to be managed by a single server, with a very simple messaging protocol that's not restricted to TCP/IP and can be used by devices with extremely limited processing power.

Publish/subscribe protocols aren't new. While they may be at the back of many developers' toolboxes, they're an easy way of building a messaging bus for a service architecture. MQTT is a relatively simple implementation of publish/subscribe, with messages published to topics that can be subscribed to by client applications. Topics are hierarchical, so you can subscribe to a single source or an element of a hierarchy.

Hierarchical topics

For example, a set of sensors in a refrigeration unit on a truck might set up a hierarchy of topics formatted as refrigerationsensors/TRUCK/temperature/ZONE where TRUCK and ZONE are identifiers that can be used to filter messages to a single vehicle or a single area on multiple vehicles.

An application can subscribe to the available topics on an MQTT server using an explicit subscription to get a specific message feed or a wild card to aggregate messages in different topic hierarchies using wildcards. You can use a wildcard to replace a single level in a hierarchy or all the remaining elements.

In our example, instrumented trucks refrigerationsensors+temperature+ would allow us to see the temperatures in all zones on all trucks, while refrigerationsensors/TRUCK1# would get all refrigeration sensor readings for a truck with the name TRUCK1. This approach means you can have multiple microservices handling different topics -- one for dealing with day-to-day monitoring of specific sensors or data, another looking for error conditions, and so on.

MQTT gives you the option of retaining the last message sent, so when a new client connects to a server there's the option of downloading an initial message as soon as it makes a subscription, without having to wait for a fresh message to be delivered from a source. Sources can also have a "will" -- a preset message stored on the server that can be delivered if there's a disconnection. Clients are able to disconnect at will, too, and if they don't set a clean session flag, they can reconnect and see existing subscriptions. The combination of session flags and wills allows you to handle issues that arise from unreliable networks and from automated service scaling.

Setting quality of service

One of MQTT's more useful features is the ability to set QoS rules for your messages. Three QoS levels are available:

Level 0 is a "best effort" option, with no option to retry if there's a failure. This approach works well for continuous monitoring, where trends matter rather than the actual message. You'd connect a QoS level 0 message queue to a stream processor or a machine learning system, where you're handling bulk messages as part of a control system.

Level 1 is an "at least once" option, where a message is guaranteed to be delivered, though multiple deliveries can occur. If you're handling a relatively low volume of messages and are able to programmatically deal with duplicates, then this may be your best option. It's fast, low impact, and works with a wide range of different application models.

Exactly once delivery is the most computationally intensive level, but it ensures that each message is received once and only once.

At heart, MQTT is a very simple protocol, which makes it ideal for building IoT applications or for handling scaling in a hyperscale cloud. As new devices are added to a network, they can quickly become new topic sources, while new clients can subscribe to topics as soon as they are created. A URI scheme makes it easy to connect to a server and deliver and receive messages. Plenty of tools exist to help test MQTT connections, including desktop and mobile applications -- and even apps that use websockets.

It's easy to set up message-driven services for IoT applications using MQTT, with a range of servers including the popular Mosquitto, as well as sample code for Arduino and other maker boards. On the client side, MQTT is also supported by familiar tooling, including the Node.js-based Seneca microservices framework and the visual Node-Red programming environment, with a growing community to help you get started building your apps.

Worried about how the MQTT protocol might perform in the real world? Here's proof that it scales well: It powers one of the Internet's largest and busiest apps, Facebook's Messenger.

Copyright © 2015 IDG Communications, Inc.