Previously I have been writing about how topics in messages can have a significant impact on the amount of data for publishing via MQTT. Since then the MQTT version 5 standard has been released and first implementations are available. Naturally, Qt MQTT follows along and this post will reflect how new features in the standard are available in our Qt for Automation module.
In this post, we will not go into every detail of the updated specification, but rather showcase some highlights and benefits for upgrading to the latest versions.
Also, before we forget, make sure to sign up for my upcoming webinar on the same topic. Tune in and discuss!
Topic Alias
As a reminder, the topic of a message is always sent uncompressed in clear text as part of the byte stream. For example, a sensor network reporting temperature values could have a topic hierarchy designed as
sensors/Europe/2af89b42-d2a6-11e8-a8d5-f2801f1b9fd1/Temperature
Consequently, the byte layout of a message has this shape:
This sums up to 71 bytes for each message, with 4 bytes containing the requested information.
In the previous post, we talked about methods like
- Reducing the topic length
- Combining multiple sensor values to enhance the overhead-information ratio.
MQTT5 introduces Topic Aliases which allow assigning IDs to a specific topic.
During the connection handshake, the client and the server agree on whether topic aliases are available and what the maximum number of topic aliases is. While ids are available, they can be assigned to topics. This assignment is done during a publish message. The new layout to send a message and concurrently set a topic alias looks like this:
For the first message, the data size is 74 bytes, three more than for a conventional publish. However, the second publication already benefits from this concept
The message size has been reduced to 11 bytes, simply by exchanging the topic with an ID and keeping the topic string empty. Starting from the second message this is a major improvement to reduce the required bandwidth.
It is important to note that topic alias assignments are only set for one connection. Other subscribers on this topic, especially ones that do not use topic aliases, will receive messages including the full topic. Alternatively, those connections can create their own topic alias matching. It is the responsibility of the server to forward topics with either the full topic, or a different alias.
Furthermore, topic aliases are not static. An alias can always be re-assigned to another topic.
Let’s move on by showcasing how to use this feature with Qt MQTT. Support for MQTT v5 has been added for Qt MQTT with Qt for Automation 5.12.
- Connect with topic aliases
To set topic alias support for a connecting client, the QMqttClient needs to be updated before connecting to the server
const int userMaxTopicAlias = 9;
QMqttConnectionProperties userProperties;
userProperties.setMaximumTopicAlias(userMaxTopicAlias);
client.setConnectionProperties(userProperties);
[…]
client.connectToHost();
- Verify that the server supports aliases
When a server connected, query that the server also supports topic aliases
const auto serverProperties = client.serverConnectionProperties();
const quint16 serverMaxAlias = serverProperties.maximumTopicAlias(); // Returns the number of available aliases
A connection can use up to the minimum of client and server topic aliases.
- Publish a message while setting topic alias
const QMqttTopicName topic(QLatin1String("sensors/Europe/2af89b42-d2a6-11e8-a8d5-f2801f1b9fd1/Temperature"));
QMqttPublishProperties aliasProperties;
aliasProperties.setTopicAlias(2);
client.publish(topic, aliasProperties, msgContent, 1);
Internally, Qt MQTT stores the assignment to simplify follow up publications.
- Publish follow up messages
client.publish(topic, msgContent, 1);There is no need to add the properties for subsequent calls unless the purpose is to reassign an ID.
Qt MQTT tries to use topic aliases automatically unless a user specifies them. After a connection has been established and Qt MQTT recognizes support for using topic alias, it “converts” classic publish messages to use topic aliases until all available aliases have been assigned. It uses an internal cache for this purpose, which can be controlled by using above QMqttPublishProperties.
There is no heuristics included in that cache, thus we still recommend using topic aliases manually. It depends on the project to identify which topics are frequently used and which topics can create the biggest benefit by using aliases.
As a closing note, Qt MQTT also takes care of the subscribing end about topic aliases. The process is the same, with the server specifying the topic and alias match. Qt MQTT then tracks this relationship and passes the full topic string inside a QMqttMessage to not require user specific code for this.
If a user wants to know, whether a topic has an assigned alias, the publish properties of a QMqttMessage contain this information.
Properties everywhere
You might have recognized the usage of a publish property class above. This reveals a new concept introduced with MQTT 5. While staying a lean and simple to use protocol, efforts have been made to allow for more dynamic setups.
Properties can be embedded into the existing commands, but it is not a must to include them. Default values cover the most used scenarios or ensure a compatible behavior with MQTT 3.1.1. Providing a zero byte indicates no properties to be set.
All available properties and their matching commands are listed below:
For each command, a new property class has been introduced with version 5.12. Please check the documentation on how to precisely use them.
Security
Authentication has been updated in a major fashion within MQTT 5.0, to the extent that a new command AUTH has been introduced. After sending a connection request from a client to a server, the server might ask for further authentication information. This happens via sending an AUTH command including a Continue reason code. The client then needs to provide authentication data, which either leads to successful connection, the server asking for even more data or the connection being closed due to a failed authentication. There is no limit on the number of authentication steps.
The actual method used for authentication is decided on the server side. This can be for instance OAuth, or any other functionality integrated on the server side.
In Qt MQTT the QMqttClient has an authenticationRequested and authenticationFinished signal to reflect a notification from the server. Furthermore, the authenticate function has been introduced to send authentication data to a server.
What’s more?
Descriptive arguments can be used to provide more details on the content of messages. And even more, it is now possible to allow servers to provide error messages to a client. This can be of big help when implementing a communication protocol between a client and a server. The standard used to demand a disconnect in case of invalid behavior. It still does, but now a server can include either error codes or even error strings as guidance for the client or the developer.
Naturally, all these things are reflected in Qt MQTT.
We would like to understand more about your usage of this module and which features would be required most to make your project succeed faster. If you have any comments, feel free to get in touch with us either via the comment section, directly via email or catch me live during the upcoming webinar.
