Rainer Keller

Qt Virtual Keyboard Updated with Handwriting Recognition

Published Wednesday March 2nd, 2016
3 Comments on Qt Virtual Keyboard Updated with Handwriting Recognition
Posted in Dev Loop, Embedded

We are proud to present the new release of Qt Virtual Keyboard with Hand Writing Recognition (HWR), performance improvements, Nuance T9 Write integration, and support for Traditional Chinese language!

Qt Virtual Keyboard is now updated with new features and versioning scheme. As part of the recent licensing change announcement, the formerly commercial-only feature, the Qt Virtual Keyboard is now available also with GPLv3 license for open-source users, in addition to commercial Qt licensees. We released a technology preview of Qt Virtual Keyboard 2.0 a while back, and have now been improving it based on the feedback received. We have also adapted a new version numbering scheme: the Qt Virtual Keyboard now follows Qt versions. With the nupcoming Qt 5.6, Virtual Keyboard is still packaged only to the commercial installers, open source users need to get it from the repository. From Qt 5.7 onwards, Qt Virtual Keyboard is also included in the open source installers.

Qt Virtual Keyboard is a fully comprehensive out-of-the-box input solution. The most important new features of the Qt Virtual Keyboard include:

  • A reference implementation of handwrite input method with Lipi toolkit alphabetic + numeric recognizer integration (English)
  • Performance optimizations for Lipi toolkit
  • Accelerated processing of HWR recognition results
  • Full screen HWR mode (on-top writing)
  • Nuance T9 Write HWR integration
  • Word reselect for Hunspell word prediction
  • Support for runtime language switching (from the application)
  • Traditional Chinese keyboard layout

Check out the following video to see the latest version of the Qt Virtual Keyboard in action:

Full screen HWR mode (on-top writing)

In the technology preview, the HWR integration used only the regular keyboard layout as HWR input area. In the new fullscreen HWR mode, which can be used in addition, the whole screen is used as an input area instead. Fullscreen HWR mode can be activated from the keyboard by double tapping the full screen HWR button. When the fullscreen HWR mode is active the keyboard is hidden. Trace input is activated and inactivated from a floating button on screen. Fullscreen HWR mode is then deactivated by double tapping the floating button. The switching from writing to selection mode and back is done by single tapping the floating button.

Handwriting in fullscreen mode

Performance optimizations for Lipi toolkit

We are using Lipi toolkit as an open source alternative handwriting recognition engine. Based on the technology preview, we found out that it did not perform well enough on low-end hardware. One obvious reason for this is that the Lipi toolkit was not optimized to run on embedded devices. We conducted some analysis and the performance is now improved by code-level optimization. With that, we have improved the performance by a neat 10-40% in recognition and data model loading!

Accelerated processing of recognition results

The Qt Virtual Keyboard runs the HWR tasks in a separate background thread. This allows the UI thread to continue its operation while the HWR tasks are running. But the recognition results could be produced even faster by starting the recognition already while waiting for the timeout of the user input.

Nuance T9 Write HWR integration

Nuance T9 Write is a commercial HWR engine, which can be enabled at compile time, if the user has a suitable license from Nuance. It is much faster than the Lipi toolkit on embedded hardware. Nuance T9 Write engine is integrated to Qt Virtual Keyboard as an alternative recognition engine for HWR mode. The initial Nuance T9 Write integration supports Latin languages, and is implemented in such a way that it is easy to support also the non-Latin languages in future releases. The integration can also be used with the on-top writing mode. Current support contains x86 and ARM targets.

Word reselect for Hunspell

Added word reselect feature for Hunspell input method. The word reselect allows the user to activate predictions / spell corrections for an existing word by touching the word in the input field. Before it was not possible to reselect a word.

Support for runtime language switching from the application

Earlier versions only supported to switch the language by pressing the language button on the virtual keyboard but there was no way to change the language from the program side. Because Qt API does not provide a uniform mechanism to change the input language, the QInputMethod API provides the current input locale as read-only property. Now the virtual keyboard settings API is extended to include options for controlling input language.
  • New properties locale, availableLocales and activeLocales are added to settings
  • The locale(if defined) overrides the system default locale in startup
  • The locale property can also change the runtime language
  • The availableLocales property is read-only and provides the list of “installed” locales
  • The activeLocales is application defined subset of availableLocales specifying the locales which can be activated runtime. If the list is empty, then all the available locales are active

Traditional Chinese

Added support for Traditional Chinese / Cangjie input method. The input method implementation is ported from Apache 2.0 licensed 3rd party library.
The Qt Virtual Keyboard supports 3 different Chinese input methods:
  • Pinyin(Chinese simplified)
  • Cangjie
  • Zhuyin

The type of available input methods is configured at compile time.

Get Qt Virtual Keyboard
Qt Virtual Keyboard is included in the commercial Qt 5.6 packages, Qt 5.6.0 final being released later in March. It has now also been contributed to open source under GPLv3 license by The Qt Company, and will be part of the Qt 5.7 release packages. If you are an open source user, please check the code from the repository. If you already have a commercial license, you can choose to install the new Qt Virtual Keyboard with handwriting support in conjunction with Qt 5.6.
Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in Dev Loop, Embedded


gyll says:

Since 5.6 is a long term support release, i think you should include the virtual keyboard in the open source 5.6 package also, it’s be a pity to force LTS users to build it themselves. And btw, what is the reasoning for not including it in the open source 5.6 release?

@gyll: It is related to the timeline of freezing release content, it was not feasible to make all the needed licensing and packaging changes in time for Qt 5.6. The wait should not be that long, as Qt 5.7 is coming in a couple of months time.

Olivier says:

i guess more and more components won’t be usable on LGPL Licence.
I understad the fact that you can’t work for free. But maybe it could be an option to put it in LGPL in later version. For example it is in GPL on 5.6 but will be in LGPL in 5.8.
Like this people that want it quick will use the commercial licence.

Commenting closed.

Get started today with Qt Download now