Etienne Dauphin

Handwriting, the essential input method for a successful User Experience

Published Wednesday November 28th, 2018
4 Comments on Handwriting, the essential input method for a successful User Experience
Posted in Biz Circuit & Dev Loop, Guest Blog | Tags: , , , ,

What is the aim of human-machine interfaces (HMI), if not to facilitate users in their primary tasks? Since 2011, MyScript has been conducting User Experience (UX) studies to define what users need to interact with their devices seamlessly.  In the case of cars, HMIs have greatly enhanced the driving experience, and many options are now available for drivers to enjoy their journeys behind the wheel. But no matter what car manufacturers’ focus is on, safety and reducing the driver’s distraction is always a top priority. MyScript’s studies for in-car UX revealed that drivers are not fully satisfied with a single input method and that on the road, handwriting is often the only solution that can clear out communication obstacles between users and their cars.

Along these lines, the Qt Virtual Keyboard offers to activate a handwriting recognition mode as an additional out-of-the-box input method solution.

qt-avenueMyScript focuses on user satisfaction to provide the best handwriting solution for car HMI. We leverage our artificial intelligence technology to adapt to the way people handwrite while driving: letters and/or fragments of words written in cursive or handprint, on top of each other or even without any spaces in between – our technology recognizes all these writing styles.

Critical situations where MyScript’s handwriting recognition (HWR) will be beneficial include:

When voice falls short

Voice control is a natural input method when conditions are good, but noisy environments, an unusual voice, speakers or microphones already taken by another action are some common situations where voice won’t do.

Plus, only handwriting has been authorized by U.S. regulatory bodies as an alternative to voice to interact with a car interface while driving. In the absence of it, drivers must stop the vehicle to use their keyboards.

For these reasons, manufacturers like Honda (Acura) or Audi have chosen our technology to equip their latest car models.

When touch surfaces are small­

Typing on small surfaces can be complicated and take time, while handwriting removes all frustrations. As a result, we power many connected devices with tiny surfaces, the smallest of all being the Samsung Galaxy Watch, with its one-inch display.

When mixing input types is inevitable

Use cases depend on the device, the form factor and the target field. As the Qt Virtual Keyboard offers different input modes (text, digits, emails with a variety keyboard layouts), MyScript can constrain its recognition engine, and therefore boost its accuracy by taking into account the context and the expected type of result.

Good news for developers, they can configure MyScript’s HWR as needed

MyScript’s handwriting recognition is totally flexible. Our engine relies on three main processes taking place simultaneously: symbol classification, segmentation and linguistic analysis. Information is exchanged between the three processes to find the most probable recognition result.qt-myscriptengine2

The symbol classification relies on the knowledge of the existing characters related to a text input. Developers select the language with an alphabet knowledge resource that defines the proper set of characters. They can also limit the number of characters with a subset knowledge resource.

The segmentation cuts or groups strokes together. Each group hypothesis is meant to match an existing symbol. For instance, it can group strokes to form a Chinese character or it can cut a single stroke in multiple characters in English cursive writing. In case of superimposed text, the segmentation unstacks strokes to form characters. Developers can configure the desired handwriting style with the appropriate alphabet knowledge resource.

The linguistic analysis relies on a set of words and grammars that rule the way the existing symbols can be combined to make up meaningful content. MyScript provides tools for developers to create specific linguistic knowledge resources that will match the expected use case e.g.: navigation, media, etc.

The combination of QT’s virtual keyboard with MyScript’s handwriting recognition technology offers the most efficient and adaptative IME. It is available for a total of 71 languages including all 40 languages offered in the Qt VK.

Have a project in mind? Start your own evaluation:

  • Download our contribution code and guidelines
  • Send us an e-mail at qt@myscript.com to get a MyScript Text SDK certificate with free access to our tools and support
Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in Biz Circuit & Dev Loop, Guest Blog | Tags: , , , ,

4 comments

d3fault says:

Do I understand correctly that you’re depending on proprietary software for handwriting recognition? Do you really expect your open source “users of Qt” to follow suite!?!? I tried to find the source code for MyScript Handwriting Recognition, but all I found was a Pricing page: https://developer.myscript.com/pricing

Am I wrong?

@d3fault: Qt 5.12 LTS supports three engines to choose from for handwriting recognition: Lipi Toolkit, T9 Write and MyScript. Out of these the Lipi Toolkit is available under open-source license, but only supports English language. It also does not necessarily perform as well as the commercially licensed alternatives, especially on embedded targets. T9 Write and MyScript offer more languages to choose from, but are only available under commercial terms. More info about the languages and handwrite in general can be found in the Qt documentation: http://doc-snapshots.qt.io/qt5-5.12/handwriting.html.

Kervala says:

Do you plan to support Windows native HWR under Microsoft platforms ? https://docs.microsoft.com/en-us/windows/desktop/api/msinkaut/nn-msinkaut-iinkrecognizer

We are using them in our software and it works very well 🙂

MyScript is working well too but at the time we needed a HWR solution (on 2010), they were too expensive for our small company 🙁

Etienne Dauphin Etienne Dauphin says:

Hello Kervala,
2010 is a long time ago! Our technology and our commercial offer have evolved a lot since then. What we provide presents two huge benefits:
– A cutting-edge, pre-integrated technology that comes right out of the box with Qt.
– An agnostic solution: it runs on Windows, Android, Linus, iOS, QNX, Tizen, and so forth.

For more information, contact us at qt@myscript.com.

Etienne.

Leave a Reply

Your email address will not be published.

Get started today with Qt Download now