MQTT Protocol: Conceptual Immersion

The Message Queuing Telemetry Transport (MQTT) protocol has been used for many years, but now it is especially relevant due to the explosive growth of IoT: both consumer and industrial devices implement distributed networks and edge computing, and devices with continuous data transmission are becoming part of everyday of life.

This means that lightweight, open, and affordable protocols will become even more important over time. This article provides a conceptual immersion in MQTT: how it works, how it is used now and how it will be used in the future.

Small introduction

MQTT is a publisher-subscriber-based messaging protocol (pub / sub). The initial version in 1999 was published by Andy Stanford-Clark of IBM and Arlene Nipper of Cirrus Link. They viewed MQTT as a way to maintain communication between machines in networks with limited bandwidth or unpredictable communications. One of the first options for its use was to ensure that fragments of the pipeline were in contact with each other and with the central links via satellites.

Given the harsh operating conditions, the protocol is made small and light. It is ideal for low power devices and with limited battery life. These now include the ubiquitous smartphones, and the ever-growing number of sensors and connected devices.

Thus, MQTT has become a protocol for streaming data between devices with limited CPU power and / or battery life, as well as for networks with expensive or low bandwidth, unpredictable stability or high latency. That is why MQTT is known as the ideal vehicle for IoT. It is built on the TCP / IP protocol, but there is an MQTT-SN branch for working via Bluetooth, UDP, ZigBee and other IoT networks other than TCP / IP.

MQTT is not the only real-time pub / sub messaging protocol of its kind, but it has already gained widespread acceptance in various environments that depend on machine-to-machine communications. His peers include Web Application Messaging Protocol , Streaming Text-Oriented Messaging Protocol, and Alternative Message Queueing Protocol .

MQTT is the logical choice for developers who want to create applications with reliable functionality and broad compatibility with devices and applications connected to the Internet, including browsers, smartphones and IoT devices.

How MQTT Works: The Basics

The communication system built on MQTT consists of a publisher server, a broker server, and one or more clients. The publisher does not require any settings for the number or location of subscribers who receive messages. In addition, subscribers do not need to tune to a specific publisher. There may be several message brokers in the system.

MQTT provides a way to create a hierarchy of communication channels - a kind of branch with leaves. Whenever the publisher has new data for distribution to customers, the message is accompanied by a note on delivery control. Higher-level clients can receive each message, while lower-level clients can receive messages related to only one or two basic channels, “branches” at the bottom of the hierarchy. This facilitates the exchange of information ranging in size from two bytes to 256 megabytes .

An example of how you can configure the client to connect through the MQTT broker :

var options = { keepalive: 60, username: 'FIRST_HALF_OF_API_KEY', password: 'SECOND_HALF_OF_API_KEY', port: 8883 }; var client = mqtt.connect('', options); 

Any data published or received by the MQTT broker will be encoded in binary format, since MQTT is a binary protocol. This means that the message must be interpreted in order to receive the original content. Here's what it looks like with Ably and JavaScript:

 var ably = new Ably.Realtime('REPLACE_WITH_YOUR_API_KEY'); var decoder = new TextDecoder(); var channel = ably.channels.get('input'); channel.subscribe(function(message) { var command = decoder.decode(; }); 

MQTT brokers can sometimes accumulate messages related to channels that do not have current subscribers. In this case, the messages will either be discarded or saved, depending on the instructions in the control message. This is useful in cases where new subscribers may need the most recently recorded data point, rather than waiting for the next send.

It is noteworthy that MQTT transmits security credentials in clear text, otherwise authentication or security functions are not supported. This is where the SSL framework comes into play , helping to protect transmitted information from being intercepted or otherwise tampered with.

In addition, in MQTT, you can use Ably authentication on tokens if you do not want to disclose your API key to the actual MQTT client at all (in the case of MQTT without SSL, tokens are required to prevent the transfer of API keys in clear text). Token authentication example:

 var options = { keepalive: 60, username: INSERT_TOKEN_HERE, password: '', port: 8883 }; var client = mqtt.connect('', options); client.subscribe("[mqtt]tokenevents", { /* Create a new token called 'NEW_TOKEN' */ client.end(); options.username = NEW_TOKEN; client = mqtt.connect('', options); }); 

MQTT functionality: deeper immersion

According to IBM , MQTT has the following properties:

A member of the MQTT system can take on the role of publisher, consumer, or both roles at once.

One of the distinguishing characteristics of MQTT is its unique understanding of channels: each of them is treated as a file path, for example:

 channel = "user/path/channel" 

Channels ensure that every customer receives messages destined for him. By treating channels as file paths, MQTT performs all kinds of useful communication functions, including filtering messages based on where - at what level or in which branch - clients subscribe to the file path.

MQTT message format

Look at the two components that make up each MQTT message:

The QoS flag in byte 1 deserves special attention, as it underlies the variable functionality that MQTT supports. QoS flags contain the following values ​​based on the intent and urgency of the message:

Let's look at how to use different QoS levels in IoT devices and other applications.

Where can I use MQTT?

As IoT applications are now being implemented on a massive scale, MQTT has come to the fore as an open, simple, and scalable way to deploy distributed computing and IoT functionality for a wider user base - both in the consumer and industrial markets.

As stated above, MQTT is a lightweight messaging protocol built for untrusted networks and devices with restrictions on power supply and CPU. However, this does not mean that the connection with potential packet loss is its only application. MQTT provides various levels of service for various types of IoT infrastructure, from repeating data sampling to managing industrial machines:

When not to use MQTT?

Developers have a wide selection of protocols for designing and deploying bidirectional IoT communication channels, including MQTT, HTTP, CoAP , WebSockets (if the CPU / battery allows) and others. Whether MQTT is the best choice depends on the hardware and the application task.

Designed for environments with extremely low bandwidth, the MQTT protocol can be quite inflexible in its quest to save every byte. For example, the specification defines only five error messages with which the server can reject the connection (for example, an invalid username / password or an unacceptable version of the protocol). If the server wants to point to some other error, it is out of luck. Even worse, if an error occurs after starting the connection, there is no mechanism for reporting an error at all. The server can only shrug its shoulders and abruptly interrupt the TCP connection, leaving the client without a clue why they dropped it (and without any way to distinguish intentional disconnection from a temporary network problem). For people accustomed to the more flexible and easy to debug (though less economical in terms of bandwidth) pub / sub protocols, such a Spartan approach may seem a little primitive.

MQTT is often referred to along with HTTP, so Google conducted a study comparing them in response time, traffic volume and other attributes important to developers. MQTT took first place in Google tests, but only in conditions when the connection can be reused to send several payloads.

HTTP and MQTT are a good choice for IoT applications due to the relatively small amount of traffic, low battery and memory requirements.

CoAP is another protocol that is often compared to MQTT for developing IoT systems. They are similar, but there are noticeable differences. MQTT is a many-to-many protocol, while CoAP is basically a one-to-one protocol for communication between a server and a client. At the same time, CoAP provides metadata, discovery, and content negotiation features that MQTT does not have .

In cases where clients should only receive data, Server-Sent Events is also a suitable option.

How to quickly set up MQTT

The MQTT repository on GitHub has an extensive list of open source MQTT libraries in various languages. The following are two examples of customization using the open source MQTT broker, JavaScript library, and .NET library.

Eclipse Mosquitto - open source MQTT broker

Eclipse Mosquitto is an open source message broker (EPL / EDL licensed) that implements the MQTT protocols versions 5.0, 3.1.1 and 3.1. Mosquitto is lightweight and suitable for use on all devices: from low-power single-board computers to full-fledged servers.


MQTT.js is a client library for the MQTT protocol, written in JavaScript for Node.js and a browser. Here is an example of sending a message using MQTT.js:

 var mqtt = require('mqtt') var client = mqtt.connect('mqtt://') client.on('connect', function () { client.subscribe('presence', function (err) { if (!err) { client.publish('presence', 'Hello mqtt') } }) }) client.on('message', function (topic, message) { // message is Buffer console.log(message.toString()) client.end() }) 


MQTTnet is a high-performance .NET library that provides both the client and the MQTT server (broker).

MQTT Client Installation:

 // Create a new MQTT client. var factory = new MqttFactory(); var mqttClient = factory.CreateMqttClient(); 

After you configure the MQTT client settings, you can establish a connection. The following code shows how to connect to the server:

 // Use WebSocket connection. var options = new MqttClientOptionsBuilder() .WithWebSocketServer("") .Build(); await client.ConnectAsync(options); 

Receive incoming messages:

 client.UseApplicationMessageReceivedHandler(e => { Console.WriteLine("### RECEIVED APPLICATION MESSAGE ###"); Console.WriteLine($"+ Topic = {e.ApplicationMessage.Topic}"); Console.WriteLine($"+ Payload = {Encoding.UTF8.GetString(e.ApplicationMessage.Payload)}"); Console.WriteLine($"+ QoS = {e.ApplicationMessage.QualityOfServiceLevel}"); Console.WriteLine($"+ Retain = {e.ApplicationMessage.Retain}"); Console.WriteLine(); Task.Run(() => client.PublishAsync("hello/world")); }); 

Post Publication:

 var message = new MqttApplicationMessageBuilder() .WithTopic("MyTopic") .WithPayload("Hello World") .WithExactlyOnceQoS() .WithRetainFlag() .Build(); await client.PublishAsync(message); 

See the MQTTnet documentation and wiki for more examples .

For enterprise-class providers, there are ready-made MQTT servers for scalable messaging between mobile applications, industrial machines, and a wide range of other IoT uses. This guide tells you how to use MQTT through an enterprise level broker.

What about scaling?

When it comes to scaling MQTT, there are two considerations to consider: 1) is this the correct protocol; 2) regardless of the protocol choice, what infrastructure and network capabilities are needed to handle the increased traffic between devices using MQTT.

Lightweight Machine-to-Machine (LWM2M) is another protocol that can be used with MQTT at the enterprise level. Compared to MQTT, it is sometimes better suited for long-term IoT systems . MQTT is ideally suited for a trial run of IoT effortlessly, while the LWM2M provides features for a long-term, versatile infrastructure. LWM2M also provides superior device management tools such as connection monitoring, firmware updates, and remote device actions. For enterprises with a large number of unmanaged devices that send large amounts of data to a central platform, LWM2M is the best choice. However, we are talking about large-scale deployments of IoT, so usually MQTT is more than an adequate option. In addition, MQTT is more common and has wider support.

Now about the possibilities of infrastructure. When it comes to server loading, the number of simultaneous connections is rarely the bottleneck. Most good MQTT servers / brokers support thousands of concurrent connections, but what is the workload required to process and reply to messages after the MQTT server has received the actual data? As a rule, there are all kinds of potential problems, such as reading and writing to and from the database, integration with the server, distribution and management of resources for each client, etc. As soon as one machine ceases to cope with the load, additional servers must be added. that is, think about load balancing, message synchronization between clients connected to different servers, generalized access to the client state regardless of the connection time or the specific server to which the client is connected - the list of products zhaetsya and continues.

Such problems deserve a separate article, and a lot of information can be found in the Engineering section of our blog . In particular, see the article on some of the complexities of servicing a large-scale real-time messaging infrastructure .

What is the current situation with MQTT?

In April 2019, OASIS released MQTT v5.0 as an official standard. OASIS is a non-profit consortium of 600 member organizations and 5,000 individual members .

Version 5.0 introduces a number of new features that should be of interest to developers of real-time systems. These new features are backward compatible with current versions of MQTT. Among them:

For a complete list of the new MQTT 5.0 features, see Appendix C to the official standard.

In addition to the many consumer devices and services on the market, MQTT has found use in corporate infrastructure of all shapes and sizes . These are smartphones and tablets, energy monitoring systems, medical devices, oil rigs and rigs, the automotive and aerospace industries, as well as sensors and machine vision systems used in material handling, construction, supply chain, retail, and much more.

MQTT and Ably

MQTT is a popular, widely supported, and relatively mature protocol. It is great for many real-time applications , and not just for deploying IoT. However, as real-time data production and consumption continue to grow exponentially , MQTT may not always be the right protocol to meet your streaming needs. Follow our Realtime Concepts section for information on other protocols and how they suit your situation.

Ably provides a broker and an MQTT protocol adapter with translation to Ably's own protocol in both directions, which allows you to integrate with any existing systems and connections. Supported WebSockets, HTTP, SSE, gRPC (in development), STOMP, AMQP and other protocols for organizing a distributed messaging infrastructure in real time. There are over 40 SDK client libraries and support for proprietary real-time protocols.


All Articles