Miguel Costa

Qt in Visual Studio: A New Approach Based on MSBuild

Published Wednesday January 24th, 2018
23 Comments on Qt in Visual Studio: A New Approach Based on MSBuild
Posted in Dev Loop, integration, Labs, Qt, Random, Visual Studio, Windows

As mentioned in a previous post, we have been working on a new official release of the Qt Visual Studio Tools. Ahead of this release, we would like to present some key changes and what impact they will have in usability and performance. In this post, we describe a new approach to the integration of the Qt tools moc, rcc and uic, both in the build process and at design time. In a follow-up post we will then discuss how these and other modifications contribute to an improvement in performance.

Custom Build Tools

Integrating external/third-party tools in Visual Studio has traditionally been accomplished by means of “Custom Build” steps associated to files in the project. The files could be, for example, headers containing Qt macros which must be processed by moc to generate the corresponding meta-object code. In this case, each header file is labeled as a custom build item and will have a custom build property page:

custom_build_step_property_page

This approach has some disadvantages:

“In older versions of Visual C++, you can specify, for a specific file, a custom command that gets executed before the C++ compilation step. This functionality, called Custom Build Steps, has some limitations when working with large projects and complex external tools. You cannot set up a custom build command to be run on all files of a particular type. Also, just for using an external application as a custom build step, you need to have extended knowledge of the configuration flags available on the command line.”
https://msdn.microsoft.com/en-us/library/aa730877(v=vs.80).aspx

Unlike the tools shipped with Visual Studio, such as the compiler or the linker, which have their own property pages, custom build steps are limited to a single text field where the path and arguments to the external tool must be specified. Also, as noted, a custom build step cannot be associated to a group of files – such as: “all headers containing Qt macros” – even though it would be possible to have only a single, global set of properties at project level. Each file must have its own custom build step information, which will be stored independently in the project file, regardless of repetition. This can lead to performance issues with large projects that make extensive use of custom build tools, as is the case with Qt projects.

MSBuild and Visual C++

In more recent versions of Visual Studio, MSBuild has become the underlying build engine for Visual C++. The content of C++ project files (*.vcxproj) is formatted according to the MSBuild XML schema and the build process is organized into MSBuild targets and tasks. Built-in tools, like the compiler and the linker, are parameterized through definitions – called “Rules” in Visual Studio-speak – which include the set of properties associated to a particular item type (e.g. C++ source files). The syntax of rule definitions also follows the MSBuild XML schema.

As an alternative to custom build steps, an external tool can be integrated into the build process by providing specialized MSBuild rules and targets. Instead of having to rely on the pre-defined behavior of the custom build mechanism, with this alternative approach it is possible to control the way the external tool is invoked using the full expressiveness of MSBuild. Also, by defining a rule for the external tool, project files can then be labeled as inputs to that tool rather than as generic custom build items. Instead of providing just one text field for the command line, Visual Studio will be able to provide those files with a property page specific to the external tool.

Qt and MSBuild

The new release of the Qt Visual Studio Tools will include MSBuild rules and targets specific to moc, rcc and uic (Qt/MSBuild for short). It will be possible, for example, to label header files as moc inputs. The property page that Visual Studio will display for these files will be specific to moc, and will contain fields covering all the options of that tool (the same will also apply for rcc and uic). As with internal Visual Studio tools like the C++ compiler, properties can be defined either for specific files or globally at project level:

moc_property_page_at_project_level

Projects created with the new version of the Visual Studio Tools will use Qt/MSBuild instead of custom build tools, and it will be possible to convert custom build tools in older projects. This conversion is not mandatory; the new version of the VS Tools will still work with custom build tools as before. When importing .pro files, the conversion of custom build tools to Qt/MSBuild will automatically take place.

The XML files with the Qt-specific rules and targets will be included in the VS Tools installation package and will be available to anyone that installs it. There is, however, no dependency to the VS Tools. Sharing a project that uses Qt/MSBuild (for example, with someone that does not have the VS Tools installed) will just require that the corresponding XML files be copied together with the project files.

The full expressiveness of MSBuild provides access to new possibilities which were not available with custom build tools. For instance, it is now possible to run moc, rcc and uic in parallel processes instead of having to go through every file in sequence, which was the only option with custom build tools. This feature is implemented in Qt/MSBuild using two targets. The first target goes through all input files, generates the command line to run for each file and adds it to a list of work to do. Another target then reads from this list and spawns the corresponding processes. The result, as expected, is a visible decrease in processing time:

parallel_execution_of_qt_tools

Another important benefit of using MSBuild is the possibility to directly influence the build process, including what project items are processed and with what tools. Inside an MSBuild target, it’s possible to add new items to the build and choose with which tools to process them. The Qt/MSBuild targets make use of this feature, for instance, to handle C++ source files containing Qt class definitions. These files are manually added to the project labeled as moc inputs, and are then dynamically reintroduced into the build process as inputs to the C++ compiler.

Furthermore, by using the dynamic source feature of MSBuild, source files generated by the Qt tools can be added “on-the-fly” to the C++ compilation during the build process. As such, there will be no need to include generated files in the project. This has a positive effect on the performance of the VS Tools, which will be part of the discussion in the next blog post.

Future Improvements

The new approach to the integration of moc, rcc and uic may potentially break some use cases where a file needs to be processed by a Qt tool and by some other third-party tool. This type of use case is currently possible given that the custom build is a generic mechanism that can be used to invoke any number of commands. Taking the example of a header file containing Qt macros, there is no guarantee that only moc will be called in that file’s custom build step. The conversion of custom build to Qt/MSBuild will only consider calls to moc, rcc and uic; all other commands in the custom build steps will be ignored.

One such use case concerns the integration of the tool IncrediBuild. If this tool is installed, when importing a .pro file using the current version of the VS Tools the custom build steps will contain additional instructions for IncrediBuild to run moc, rcc and uic in parallel/distributed processes. For the upcoming release, this integration will no longer be supported. The parallel execution of Qt tools will still work when using IncrediBuild but only on the local machine (built-in tools like the C++ compiler will still be able to run on distributed processes). A specific fix for the integration with IncrediBuild will be available in a later version of the VS Tools. As a generic solution to this issue of additional commands to run together with the Qt tools, we could consider allowing those commands to be given as pre- or post-events in the moc, rcc and uic property pages.

Wrap-Up

We have presented the outline of a new approach to the way moc, rcc and uic are integrated in Visual Studio. This new approach provides a design time experience and build time integration identical to that of built-in tools. In a follow-up post, we will discuss what impact these and other changes to the VS Tools will have in terms of performance. Hopefully, users will find that, with these changes, developing with Qt on Visual Studio will become even more productive.

Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in Dev Loop, integration, Labs, Qt, Random, Visual Studio, Windows

23 comments

Michael says:

I saw a blog post from someone on the c++ committee and I believe he had patched qt such that all the moc macros could be done with std c++.

That seems like a better approach that way everything will just work.

Markus says:

Are you talking about metaclasses, as presented by Herb Sutter on CppCon? What the standard commitee discusses typically is about future versions of the standard (C++20 and later), and it will take quite some time until Qt can make use of this – currently the minimum requirement for Qt is still C++11, not even C++14 . So – given the downwards compatibility restrictions of Qt – this could possibly be a part of Qt 7 (!), not earlier – and only if the feature really added to the C++ standards …

Other alternatives have been discussed here: https://woboq.com/blog/verdigris-qt-without-moc.html , but I don’t really like the alternatives to what we have now: You either get more ugly macros with all their ugly side effects (see https://stackoverflow.com/questions/14041453 for example), or you get massive bloats in code size, compilation time and/or runtime efficiency.

Moc actually does the job quite well, and Qt is not the only framework with tools like this – that is quite nicely demonstrated by Herb Sutter in the CppCon presentation about metaclasses in C++ (on Youtube), so probably the Qt company is not doing so bad here 😉

Miguel Costa Miguel Costa says:

Odds are Qt developers will still be using moc for the foreseeable future, not to mention rcc and uic. So, if you are developing in Visual Studio with Qt, the VS Tools will help you become more productive, even more so with this new release.

Michael says:

Thanks for the above info. (Qt7!)
And these tools updates will be great as well.

Bruno says:

That’s great news! What about qml support? Does this extension have support for qml code complete?

Miguel Costa Miguel Costa says:

Support for QML is something we very much would like to add to the VS Tools. But right now we needed to address the performance issues; that’s mainly what this upcoming release will be about. Once we’re confident that we have a stable and productive platform, QML support will be one of the first things we will work on.

Jakub Narolewski says:

+1

Markus says:

Great to see how the tool is being improved! I can’t wait to see a version of the tool which has all the issues marked for version 2.2 in the issue tracker, there are many points there which would greatly improve productivity with the tool!

Just to get a vague idea: When are you planning to release the next version?

Miguel Costa Miguel Costa says:

Work on most of those issues is already complete or in an advance stage so a release should happen soon.

Robert says:

Will the new Visual Studio Tools work with legacy Qt (<5)? That's always a topic when using a single installation to work both on old and new projects.

Miguel Costa Miguel Costa says:

The Qt VS Tools are designed to work with Qt5. Previous versions of Qt are not supported.

Danny says:

Really good news. The ability to moc files in parallel is very welcome!

m][sko says:

I can’t import my project with Qt Vs Tool. It always block whole VS.
Are there any way how to enable some debug information?

Miguel Costa Miguel Costa says:

I recommend you open an issue here: https://bugreports.qt.io/browse/QTVSADDINBUG and include any information that might be relevant to the problem that you’re experiencing.

Alexander Golde says:

Even for not that large projects, the convert can take some time.
In my case ~4minutes.
I have approx. 20 projects in the solution.

Alexander Golde says:

Will the new approach also speed up the conversion from Qt projects into VS Solutions?

Miguel Costa Miguel Costa says:

The import of Qt projects should be faster with the new release of the VS Tools, owing not only to the new approach to the integration of moc in the VS build process but also to the optimization of the import procedure. So you should be able to see an improvement in the conversion time from those 4 minutes.

Alexander Golde says:

Thank you! That would be nice.
Will the import procedure take advantage of a bunch of cores?

Miguel Costa Miguel Costa says:

We’ve been focused on optimizing the import of individual projects, which does not present much opportunities for parallel processing. So, for now, the import will remain single-threaded. For a future version, we’ll perhaps consider optimizing the import of solutions with many projects by spawning the processing of those projects in separate threads.

Alex says:

Will qmake generate vcxproj projects that use these MSBuild features instead of the good old custom steps? I’m especially interested in parallel MOC invokation.

Miguel Costa Miguel Costa says:

It would be possible to include the Qt/MSBuild XML files with the Qt installation and have qmake generate references to them when creating Visual Studio projects. But there are currently no plans for this. At least for now, the Qt/MSBuild files will only be released with the Qt VS Tools.
If this feature is something you would really be interested in, I recommend adding a suggestion for it here: https://bugreports.qt.io

Will Stokes says:

For clarification, when using qmake + jom (and the VC compiler) from the command line, will we will an increase in performance when moc is invoked or is this speed up only when compiling within the Visual Studio GUI?

Miguel Costa Miguel Costa says:

What is discussed in this post concerns only the integration with Visual Studio and MSBuild. There will be an improvement in overall performance when compiling in Visual Studio (or with MSBuild on the command line) because moc, rcc and uic steps will now be run in parallel. In fact, since jom is able to spawn parallel processes for all makefile steps (including moc, rcc, uic), this improvement brings the compilation in Visual Studio in line with qmake + jom in terms of performance.

Commenting closed.

Get started today with Qt Download now