We have made visible and invisible changes to the Qt Help module in the 5.9 and 5.12 branches, which influence the behavior of both Qt Creator’s Help mode and Qt Assistant.
What’s new in Qt Help in 5.9?
The most important changes to the Qt Help module in Qt 5.9 were replacing the CLucene search engine with SQLite with FTS5 mode turned on and fixing the documentation indexing.
Replacing the search engine
Internally, Qt Help used the CLucene third party code for full-text search functionality. However, the CLucene lib has not been actively maintained since a couple of years, and therefore possible bugs for CLucene have not been resolved. Also, the CLucene code base is getting older every year, and modern compilers raise more and more warnings when trying to compile it.
Switching to the SQLite database did not mean adding extra third party code to Qt, because SQLite is already included in Qt and the Qt Help module is already using it. Removing CLucene removed about 60.000 LOC from Qt sources for CLucene itself and replaced about 3.000 lines of glue code between Qt Help and CLucene. In comparison, only about 650 lines of glue code between Qt Help and SQLite’s FTS5 had to be added. Comparing these numbers to the current size of the Qt Help library itself, which is about 10.000 LOC in total, the decrease of LOC is really huge. The smaller the code base, the better. From maintenance point of view, this looks like a good move.
Additional benefits include faster and easier compilation of the Qt Help module, which affects all developers building Qt themselves. Further, the binary files that are generated are smaller in size, which affects all users of Qt and Qt Creator.
The replacement of the full-text search engine improved not only the maintenance efforts, invisible at the first glance. There are also more visible consequences, like elimination of random crashes when using text search functionality or getting more reliable search results after adding or removing documentation files.
Simpler user interface
In addition, we have made searching easier. We have replaced the Advanced search GUI interface, with many fields, with just one simple line edit, where the user may specify queries using the AND, OR, or NOT operators or use quotations to search for multi-word expressions. For example, to search for documents containing at least one occurrence of the word six or seven and at least one occurrence of eleven or twelve, we could write the following query: (six OR seven) AND (eleven OR twelve). Before, it was not possible to mix OR and AND operators in one query.
Search results snippets
Another useful feature of the new search engine is the ability to display the search results within short text snippets, showing the context of the actual results. Before, the search results displayed a document address of the result that usually was not very useful to the user. Currently, we display an excerpt from the documentation file containing the queried word or phrase.
Fixing the documentation indexing
The other visible change in Qt Help was fixing the documentation indexing. Even though the indexing was slightly faster before than after the fix, it did not produce reliable content for the search results. This is noticeable when trying to search for the terms “charset” or “UTF-8”. Because the indexing was inaccurate, all documents would match the queries above. However, when viewing the results, no document actually contained the queried word.
Apart from the visible changes in the GUI or behavior, many changes have been done internally, such as replacing the old string based connections with …, with … [Not fun getting old, my friend. How was that new typed connection style called again? I’ll search for string based connection. Let’s ask uncle G. The first result is c# – Use different connection string based on the application url…. No, that’s not what I was looking for. OK, let’s try Qt Help 5.8. The result is QSqlDatabase Class. Hmm, no, wrong hit. So, let’s try again using Qt Help 5.9. The result is Differences between String-Based and Functor-Based Connections. Wow! That’s what I was looking for! No, wait… Where is this result on uncle’s and Qt Help 5.8 lists? The uncle has it listed as the fourth hit, while in the Qt Help results it’s listed as the… 12th hit! Looks like the new search engine is doing its job! I need to remember to mention it in the blog. Cool, I may now finish my sentence:] … with functor-based connections, replacing Java-style iterators with faster STL-style iterators, and other code modernization.
What’s new in Qt Help in 5.12?
The most important changes in the Qt Help module in version 5.12 are that QHelpEngine doesn’t keep every registered .qch file open anymore and that it not detects the version numbers of documentation files.
Closing registered help files
Let’s imagine that a user of Qt Creator has installed 10 different Qt versions. Each Qt version comes with a documentation set consisting of about 60 .qch files. This means that the Qt Help integrated in Qt Creator has been keeping track of about 600 open-file descriptors at a time! On some platforms, this number may exceed the maximal allowed number for the open-file descriptors per process. Certainly, it was a real issue.
Starting from Qt 5.12, Qt Help keeps a small cache inside the help collection file for all registered documentation files. The help engine updates the cache upon every registration or de-registration of the documentation files. Qt Help opens .qch files only on demand and closes them immediately after getting the data. This way, when Qt Help needs to return the index or content results, it doesn’t need to keep all the .qch files open. This also speeds up returning these results.
Detecting documentation version numbers
If documentation files cross-reference each other and the user installed many documentation versions, clicking on such a cross-reference link usually resulted in showing the correct page, but its version didn’t match the version of the source page. We have solved this issue in the Qt Help 5.12 branch.
Another annoying bug that is fixed in 5.12 is the order of content items inside the Contents view. Before, the order was rather arbitrary, depending in some way on the order of registration of documentation files. Currently, content items are sorted alphabetically.
In addition, we have done some cleanups in version 5.12. We got rid of an ancient porting tool called qhelpconverter. We have merged the qcollectiongenerator tool into the qhelpgenerator, which now accepts both .qhp and .qhcp files as input and generates .qch and .qhc files, respectively.
Not fun getting old, my friend. I’ve forgotten to mention that there is quite a big chance that you’ll get better results for your search queries starting from Qt Help 5.9, compared with the previous versions. Try searching for e.g. “QHelpEngine”. 🙂