Over-the-Air Updates, Part 2: Device Integration, API and Creating Updates

With Qt 5.7 for Device Creation we introduced a new piece of technology - an OSTree-based solution for Over-the-Air software updates for the whole software stack. For a more detailed introduction about this new component of the Boot to Qt software stack, read part one of the blog post series. This blog post contains a step-by-step guide on how to add OTA update capability to your Boot to Qt device, discusses Qt OTA API and finally demonstrates how to generate OTA updates for shipped devices.

Device Selection.


For this tutorial we are going to use the Intel NUC board - a new reference device in Qt 5.7 for Device Creation. Intel NUC is a low-cost, pint-sized powerhouse solution for a wide variety of fields, including Internet of Things where OTA update capability is especially desired. I chose this board for the tutorial because it is not your traditional ARM-based embedded Linux board that uses the U-Boot bootloader. Instead, Intel NUC is a x86-64 target with the GRUB 2 bootloader. The OTA solution in Technology Preview supports U-Boot and GRUB 2 bootloaders, and adding support for additional bootloaders is a straightforward task (as long as a bootloader has the means to read from an external configuration file).

Device Integration Steps.

I won't go into too much detail at each step, as that is already covered in the OTA documentation. The goal of this tutorial is to show the necessary steps to add OTA capability to a device and to demonstrate that it doesn't require months of effort to add such a capability. Rather, it takes just a few hours when using the OTA solution from Qt for Device Creation.

1. Generate OSTree boot compatible initramfs image.


This step requires booting the device with the sysroot to be released, so that the tool can generate initramfs that match the kernel version of the release. The device has to be connected to the machine from which you will run the generate-initramfs tool:

SDK_INSTALL_DIR/Tools/ota/dracut/generate-initramfs

2. Bootloader integration.


This is the only step that requires manual work. The bootscript used by your device has to be changed to use the configurations that are managed by OSTree. This will ensure that, after OTA updates or rollbacks, the correct kernel version (and corresponding boot files) will be selected at boot time. On U-Boot systems this requires sourcing uEnv.txt and then integrating the imported environment with the bootscript. On GRUB 2 systems, whenever the bootloader configuration files need to be updated, OSTree executes the ostree-grub-generator shell script to convert bootloader-independent configuration files into native grub.cfg format. A default ostree-grub-generator script can be found in the following path:

SDK_INSTALL_DIR/Tools/ota/qt-ostree/ostree-grub-generator

This script should be sufficient for most use cases, but feel free to modify it. The ostree-grub-generator file contains additional details. The script itself is about 40 lines long.

3. Convert your sysroot into an OTA enabled sysroot.


The conversion is done using the qt-ostree tool.

sudo ./qt-ostree \
--sysroot-image-path ${PATH_TO_SYSROOT} \
--create-ota-sysroot \
--ota-json ${OTA_METADATA} \
--initramfs ../dracut/initramfs-${device}-${release} \
--grub2-cfg-generator ${CUSTOM_GENERATOR}

This script will do all the necessary work to convert your sysroot into an OTA enabled sysroot. The ${OTA_METADATA} is a JSON file containing the system's metadata. The following top-level fields have convenience methods in the Qt/QML OTA API: version and description. The API provides the means of manually fetching and parsing the file (which consequently can contain arbitrary metadata describing the sysroot).

4. Deploy the generated OTA image to an SD card.


sudo dd bs=4M if=<image> of=/dev/<device_name> && sync

5. Test that everything went according to plan.


Boot from the SD card and run the following command from the device:

ostree admin status

The output should be something similar to:

* qt-os 36524faa47e33da9dbded2ff99d1df47b3734427b94c8a11e062314ed31442a7.0
origin refspec: qt-os:linux/qt

Congratulations! Now the device can perform full system updates via a wireless network. Updates and rollbacks are atomic and the update process can safely be interrupted without leaving the system in an inconsistent state. If an update did not fully complete, for example due to a power failure, the device will boot into an unmodified system. Read about the other features of the update system in the OTA documentation.

User Space Integration.


With the device being OTA capable, we need to take advantage of that. We provide C++ / QML APIs to make OTA update functionality integration with Qt-based applications a breeze. Offline operations include querying the booted and rollback system version details and atomically performing the rollbacks. Online operations include fetching a new system version from a remote server and atomically performing system updates. A basic example that demonstrates the API:


Label { text: "CLIENT:"; }
Label { text: "Version: " + OTAClient.clientVersion }
Label { text: "Description: " + OTAClient.clientDescription }
Label { text: "Revision: " + OTAClient.clientRevision }

Label { text: "SERVER:"; } Label { text: "Version: " + OTAClient.serverVersion } Label { text: "Description: " + OTAClient.serverDescription } Label { text: "Revision: " + OTAClient.serverRevision }

Label { text: "ROLLBACK:"; } Label { text: "Version: " + OTAClient.rollbackVersion } Label { text: "Description: " + OTAClient.rollbackDescription } Label { text: "Revision: " + OTAClient.rollbackRevision }

RowLayout { Button { text: "Fetch OTA info" onClicked: OTAClient.fetchServerInfo() } Button { visible: OTAClient.rollbackAvailable text: "Rollback" onClicked: OTAClient.rollback() } Button { visible: OTAClient.updateAvailable text: "Update" onClicked: OTAClient.update() } Button { visible: OTAClient.restartRequired text: "Restart" onClicked: log("Restarting...") } }

The above sample presents version information for the booted and rollback system, as well as what system version is available on a remote server. The sample program also contains buttons to initiate OTA tasks. The code below is used for logging OTA events. The API is still in Technology Preview, so the final version might have slight changes.


Connections {
    target: OTAClient
    onErrorChanged: log(error)
    onStatusChanged: log(status)
    onInitializationFinished: log("Initialization " + (OTAClient.initialized ? "finished" : "failed"))
    onFetchServerInfoFinished: {
        log("FetchServerInfo " + (success ? "finished" : "failed"))
        if (success)
            log("Update available: " + OTAClient.updateAvailable)
    }
    onRollbackFinished: log("Rollback " + (success ? "finished" : "failed"))
    onUpdateFinished: log("Update " + (success ? "finished" : "failed"))
}

This API could easily be used to write a daemon that communicates its version details to the server and the daemon could send a notification to the user when an update becomes available. The server could send out updates in batches, first updating a small subset of devices for field testing, fetching update statuses from daemons and if there are no issues, update the remaining devices. Some tools for this type of tasks are in the roadmap of OTA solution for the Boot to Qt stack.

Ship it! Some time later ... a critical bug emerges.


As we took a precaution and built an embedded device with OTA capability as well as creating a Qt application for handling updates, there are only few simple steps to follow to resolve the issue.

1. Fix the bug.


I will leave the details up to you ;) We will use the updated sysroot in the next step.

2. Generate an update.


This is done by using the qt-ostree tool. Generating an OTA update is a completely automated task.

sudo ./qt-ostree \
--sysroot-image-path ${PATH_TO_SYSROOT_WITH_THE_FIX} \
--ota-json ${OTA_METADATA_DESCRIBING_NEW_SYSROOT} \
--initramfs ../dracut/initramfs-${device}-${release}

The above command will create a new commit in the OSTree repository at WORKDIR/ostree-repo/, or create a new repository if one does not exist. This repository is the OTA update and can be exported to a production server at any time. OSTree repositories can be served via a static HTTP server (more on this in the next blog post).

3. Use Qt OTA API to update devices.


It is up to system builders to choose an update strategy.

Availability


The Boot to Qt project leverages The Yocto Project to built its own distribution layer called meta-boot2qt. The Boot to Qt distribution combines the vendor specific BSP layers and meta-qt5 into one package. This distribution is optimized to make the integration with the OTA tooling as simple as possible. All source code is available under commercial and GPL licenses.

Conclusion


Enabling the OTA update feature on Boot to Qt devices is a quick and worthwhile task. With OTA enabled devices you can ship your products early and provide more features later on. In the next blog post I will write about OSTree (OTA update) repository handling, remote configuration and security.


Blog Topics:

Comments