Qt Weekly #18: Static linking with Qt

Qt has supported static builds for a long time, but iOS made this way of shipping Qt more widespread. To make the discussion about static linking more practical, we will not discuss things abstractly, but we will look at how to compile the Weather Info example application from the Qt Positioning module statically for iOS, and address questions as they come up.

 

Weather Info example app Weather Info example app

First of all, to build a static application, we need a statically compiled Qt. This is not difficult -- adding the "-static" option to configure mostly does it. However, ensuring that the dependent libraries are also available and linked statically can take some effort. In the case of iOS, Qt is statically built by default, so we can simply use the default Qt packages.

To build the Weather Info application statically, let's just try building as usual, running qmake and make (or even better, let Qt Creator call them for us).

cd weatherinfo
qmake -r weatherinfo.pro CONFIG+=debug
make

...and... it runs!

So, it seems that we don't have to do anything special once we have a static Qt.
Great, so is this blog post already finished?

Well, not quite. If we take a look at the linking step, we see a huge command line (EDIR=/Users/fawzi/Qt5.3.1/Examples/Qt-5.3/positioning/weatherinfo):

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -arch armv7 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk -L$EDIR/Debug-iphoneos -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/platforms -L/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk/System/Library/Frameworks -L/Users/fawzi/Qt5.3.1/5.3/ios/lib -L/Users/fawzi/Qt5.3.1/5.3/ios/qml/QtQuick.2 -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/accessible -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/qmltooling -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/position -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/bearer -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/imageformats -F$EDIR/Debug-iphoneos -filelist $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo.LinkFileList -dead_strip -headerpad_max_install_names -stdlib=libc++ -u _qt_registerPlatformPlugin -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/platforms -framework UIKit -L/Users/fawzi/Qt5.3.1/5.3/ios/lib -framework OpenGLES -L/Users/fawzi/Qt5.3.1/5.3/ios/qml/QtQuick.2 -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/accessible -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/qmltooling -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/position -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/bearer -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/imageformats -lz -lm -miphoneos-version-min=5.0 -lqios_debug -framework Foundation -framework QuartzCore -framework CoreFoundation -framework CoreText -framework CoreGraphics -lQt5PlatformSupport_debug -framework Security -framework SystemConfiguration -lqtquick2plugin_debug -lqtaccessiblequick_debug -lqmldbg_qtquick2_debug -lQt5Quick_debug -lqmldbg_tcp_debug -lQt5Qml_debug -lqtposition_cl_debug -framework CoreLocation -lqtposition_positionpoll_debug -lQt5Positioning_debug -lqgenericbearer_debug -lQt5Network_debug -lqdds_debug -lqicns_debug -lqico_debug -lqjp2_debug -lqmng_debug -lqtga_debug -lqtiff_debug -lqwbmp_debug -lqwebp_debug -lQt5Gui_debug -lqtharfbuzzng_debug -lQt5Core_debug -Xlinker -dependency_info -Xlinker $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_dependency_info.dat -o $EDIR/Debug-iphoneos/weatherinfo.app/weatherinfo

It looks mostly OK, but contains a lot of libraries. System and Qt libraries are expected, but we also have:

  • the platform plugin:-lqios_debug that loads plugins/platforms/libqios_debug.a
  • the QtQuick2 plugin:-lqtquick2plugin_debug that loads qml/QtQuick.2/libqtquick2plugin_debug.a
  • the accessibility plugin for qtquick:-lqtaccessiblequick_debug that loads accessible/libqtaccessiblequick_debug.a
  • the plugin to improve debugging of qtquick2:-lqmldbg_qtquick2_debug that loads plugins/qmltooling/libqmldbg_qtquick2_debug.a
  • the plugin to talk to the QML debugger:-lqmldbg_tcp_debug that loads plugins/qmltooling/libqmldbg_tcp_debug.a
  • the plugin giving the position using the core location framework: -lqtposition_cl_debug that loads plugins/position/libqtposition_cl_debug.a
  • the generic polling plugin for position:-lqtposition_positionpoll_debug that loads plugins/position/libqtposition_positionpoll_debug.a
  • the plugin giving the generic bearer implementation: -lqgenericbearer_debug that loads plugins/bearer/libqgenericbearer_debug.a
  • support for the DDS image format:-lqdds_debug that loads plugins/imageformats/libqdds_debug.a
  • support for the ICNS image format:-lqicns_debug that loads plugins/imageformats/libicns_debug.a
  • support for the ICO image format:-lqico_debug that loads plugins/imageformats/libqico_debug.a
  • support for the JPEG 2 image format:-lqjp2_debug that loads plugins/imageformats/libqjp2_debug.a
  • support for the MNG image format:-lqmng_debug that loads plugins/imageformats/libqmng_debug.a
  • support for the TGA image format:-lqtga_debug that loads plugins/imageformats/libqtga_debug.a
  • support for the TIFF image format:-lqtiff_debug that loads plugins/imageformats/libqtiff_debug.a
  • support for the Wireless Application Protocol Bitmap image format:-lqwbmp_debug that loads plugins/imageformats/libqwbmp_debug.a
  • support for the WebP image format:-lqwebp_debug that loads plugins/imageformats/libqwebp_debug.a

The platform plugin and a couple of others should come as no surprise, but Wireless Application Protocol Bitmap? A format by WAP for black and white images? When WAP has basically disappeared? Does our application really need it?

While, the non-debug version libqwbmp.a is only around 15 KB in size, things like libqjp2.a can be around 600 KB, so this is definitely worth some investigation.

One could hope that unneeded code gets stripped away. After all, that is one of the advantages of static linking which leads to a smaller executable than shipping all dynamically loaded libraries. If a function in a library is not used, that function is not part of the static executable.

Unfortunately, plugins make the situation more complex.

Qt Plugins

Qt internally uses dynamic linking in several places, mainly with plugins. Plugins are used for platform dependent support, image formats, audio and video codecs, sensors, and so on.

If you take a look in the plugin directory (qmake -query QT_INSTALL_PLUGINS) you will see the various plugins that Qt might use.

For example, linking QtPositioning, and QtMultimedia is not enough to have a position fix and being able to display a movie, respectively. The required plugins also need to be loaded.

Calling the Q_IMPORT_PLUGIN marcro from user code will load that plugin (which will thus need to be linked).

When using dynamic linking, the plugins are loaded automatically at runtime when needed.
With static linking, one has to decide which plugins are required already at build time.

Starting with Qt 5.3, thanks to Ossi, a meaningful set of plugins (depending on the Qt modules you added) is automatically loaded.
That is the reason why without any extra effort things just worked in the above example, but that is also the reason we have so many plugins linked to our executable.

qmake does not just add the -l<plugin> to the link flags (which has no effect if not used), but really uses the plugin.
Indeed, if we look at the linked files from $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo.LinkFileList:

$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/main.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/appmodel.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_qml_plugin_import.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_plugin_import.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/qrc_weatherinfo.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/moc_appmodel.o

we see weatherinfo_plugin_import.o which comes from the compilation of the autogenerated
$EDIR/weatherinfo_plugin_import.cpp which contains the Q_IMPORT_PLUGIN that takes care of loading the Qt plugins.

This is nice, but we want a lean application, without unnecessary cruft, so how can we get rid of the plugins we do not need?

As described in How to Create Qt Plugins, it is possible to switch off the automatic loading of plugins with CONFIG -= import_plugins, but this disables the generation of the weatherinfo_plugin_import.cpp file, and then we would have to do everything manually: add Q_IMPORT_PLUGIN somewhere in our code, define the QT_STATICPLUGIN preprocessor variable and link the plugin (as one has to do without qmake).

We just want to specify which plugin to load, but still leave the ugly details up to qmake.
How to Create Qt Plugins also states that one can set the list of plugins to load for each class with QTPLUGIN.<className>. The position plugins we will actually need, but the image conversions? We should be able to get rid of all of them.

So let's add

QTPLUGIN.imageformats=-

to our project file, weatherapp.pro, and indeed no more image formats plugins are linked, the release executable becomes around 1MB smaller (13'492'384 -> 12'539'088 bytes), avoids unneeded plugins initializations, and... still works.

If some imageformat plugins are needed, it is possible to add them to QTPLUGIN.imageformats, or directly to QTPLUGIN.

It is worth noting that for this to work, the plugin creator has to define both PLUGIN_TYPE and PLUGIN_CLASS_NAME to make the automatic loading work. There were some issues with that, but hopefully all have been fixed.

QML Plugins

Our example basically uses just QtQuick 2, but in general one has to scan recursively the imports of all QML files used to find all plugins to load, and to load them. Morten wrote a tool that scans the QML files (qmlimportscanner) and qmake uses it to generate $EDIR/weatherinfo_qml_plugin_import.cpp which gets compiled to the weatherinfo_qml_plugin_import.o file we have previously seen in the linked files.

For this to work, the QML files have to be found by the tool (which might for example require installing Qt in some cases) and the qmldir files have to specify the classname of the plugin.

Conclusions

Static linking avoids the dependency on external libraries and leads to smaller packages for a single self-contained application (all the unused code of a library is stripped).
But for the linker, "used code" means code that might be reached, and to make plugins work, one has to pull in the plugin using Q_IMPORT_PLUGIN and link it.

qmake simplifies this operation by generating special files and linking the modules, but its default choices about which modules your application wants to use are not always correct. Furthermore, reachable code does not necessarily mean code that will be executed, or that does something useful for your application. For example, the initialization of a plugin might be a bit expensive, so doing it unconditionally at startup is not always a good idea.

Thus removing plugins that are not needed is a good way to make your application leaner.
There are also other places where Qt might waste CPU time (to initialize things) or space (both in application size and memory), as for example the MIME DB or Unicode information, but that is another story...

It is important to note that static linking has consequences with respect to the licenses of Qt you might want to use. While you are probably safe if your application is distributed according to the terms of a recognized free and open-source license, you should definitely check the licensing issue for proprietary or closed-source applications. Qt Enterprise license is the recommended one for closed-source applications.

I hope this has made the issue and advantages of static linking with Qt clearer. Happy coding and let me know if you have comments or tips.


Blog Topics:

Comments