Laszlo Agocs

Qt Weekly #23: Qt 5.5 enhancements for Linux graphics and input stacks

Published Tuesday January 20th, 2015
6 Comments on Qt Weekly #23: Qt 5.5 enhancements for Linux graphics and input stacks
Posted in Embedded, Graphics, OpenGL, QPA

The upcoming Qt 5.5 has received a number of improvements when it comes to running without a windowing system on Linux. While these target mainly Embedded Linux devices, they are also interesting for those wishing to run Qt applications on their desktop machines directly on the Linux console without X11 or Wayland.

We will now take a closer look at the new approach to supporting kernel mode setting and the direct rendering manager, as well as the recently introduced libinput support.

eglfs improvements

In previous versions there used be a kms platform plugin. This is still in place in Qt 5.5 but is not built by default anymore. As features accumulate, getting multiple platform plugins to function identically well gets more complicated. From Qt and the application’s point of view the kms and eglfs platforms are pretty much the same: they are both based on EGL and OpenGL ES 2.0. Supporting KMS/DRM is conceptually no different than providing any other device or vendor-specific eglfs backend (the so-called device hooks providing the glue between EGL and fbdev).

In order to achieve this in a maintainable way, the traditional static, compiled-in hooks approach had to be enhanced a bit. Those familiar with bringing Qt 5 up on embedded boards know this well: in the board-specific makespecs under qtbase/mkspecs/devices one comes across lines like the following:

  EGLFS_PLATFORM_HOOKS_SOURCES = $$PWD/qeglfshooks_imx6.cpp

This compiles the given file in to the eglfs platform plugin. This is good enough when building for a specific board, but is not going to cut it in environments where multiple backends are available and hardcoding any given one is not acceptable. Therefore an alternative, plugin-based approach has been introduced. When looking at the folder qtbase/plugins/egldeviceintegrations after building Qt 5.5, we find the following (assuming the necessary headers and libraries files were present while configuring and building):

  libqeglfs-kms-integration.so
  libqeglfs-x11-integration.so

These, as the names suggest are the eglfs backends for KMS/DRM and X11. The latter is positioned mainly as an internal, development-only solution, although it may also become useful on embedded boards like the Jetson TK1 where the EGL and OpenGL drivers are tied to X11. The former is more interesting for us now: it is the new KMS/DRM backend. And it will be selected and used automatically when no static hooks are specified in the makespecs and the application is not running under X. Alternatively, the plugin to be used can be explicitly specified by setting the QT_QPA_EGLFS_INTEGRATION environment variable to, for instance, eglfs_kms or eglfs_x11. Note that for the time being the board-specific hooks are kept in the old, compiled-in format and therefore there is not much need to worry about the new plugin-based system, unless KMS/DRM is desired. In the future however it is expected to gain more attention since newly introduced board adaptations are recommended to be provided as plugins.

libinput support

libinput is a library to handle input devices, providing device detection, pointer, keyboard and touch events, and additional functionality like pointer acceleration and proper touchpad handling. It is used by Weston, the reference Wayland compositor, and in the future potentially also in X.org.

Using libinput in place of the traditional evdevmouse|keyboard|touch input handlers of Qt 5 has a number of advantages. By using it Qt applications get the same behavior, configuration and calibration that other clients, for example Weston use. It also simplifies bringup scenarios since there will be no need to fight Qt’s input stack separately in case libinput is already proven to work.

On the downside, the number of dependencies are increased. libudev, libevdev, optionally libmtdev are all necessary in addition to libinput. Furthermore keyboard mapping is performed via xkbcommon. This is not a problem for desktop and many embedded distros, but can be an issue on handcrafted systems. Or on an Android baselayer. Therefore libinput support is optional and the evdev* handlers continue to be the default choice.

Let’s see it in action

How can all this be tested on an ordinary Linux PC? Easily, assuming KMS/DRM is usable (e.g. because it is using Mesa with working KMS and DRM support). Below is our application (a standard Qt example from qtbase/examples/opengl/qopenglwidget) running as an ordinary X11 client, using the xcb platform plugin, on a laptop with Intel integrated graphics:

Qt app with widgets and OpenGL on X11

Now, let’s switch to another virtual console and set the following before running the application:

  export QT_QPA_PLATFORM=eglfs
  export QT_QPA_GENERIC_PLUGINS=libinput
  export QT_QPA_EGLFS_DISABLE_INPUT=1

This means we will use the eglfs platform plugin, disabling its built-in keyboard, mouse and touchscreen support (that reads directly from the input devices instead of relying on an external library like libinput), and rely on libinput to get mouse, keyboard and touch events.

If everything goes well, the result is something like this:

Qt app with widgets and OpenGL on KMS/DRM

The application is running just fine, even though there is no windowing system here. Both OpenGL and the traditional QWidgets are functional. As an added bonus, even multiple top-level widgets are functional. This was not supported with the old kms platform plugin, whereas eglfs has basic composition capabilities to make this work. Keyboard and mouse input (in this particular case coming from a touchpad) work fine too.

Troubleshooting guide

This is all nice when it works. When it doesn’t, it’s time for some debugging. Below are some useful tips.

(1)
Before everything else, check if configure picked up all the necessary things. Look at qtbase/config.summary and verify that the following are present:

  libinput................ yes

  OpenGL / OpenVG: 
    EGL .................. yes
    OpenGL ............... yes (OpenGL ES 2.0+)

  pkg-config ............. yes 

  QPA backends: 
    EGLFS ................ yes
    KMS .................. yes

  udev ................... yes

  xkbcommon-evdev......... yes

If this is not the case, trouble can be expected since some features will be disabled due to failing configuration tests. These are most often caused by missing headers and libraries in the sysroot. Many of the new features rely on pkg-config so it is essential to get it properly configured too.

(2)
No output on the screen? No input from the mouse or keyboard? Enable verbose logging. Categorized logging is being taken into use in more and more areas of Qt. This includes also most of the input subsystem and eglfs. Some of the interesting categories are listed below:

  • qt.qpa.input – Enables debug output both from the evdev and libinput input handlers. Very useful to check if a given input device was correctly recognized and opened.
  • qt.qpa.eglfs.kms – Enables logging from the KMS/DRM backend of eglfs.
  • qt.qpa.egldeviceintegration – Enables plugin-related logging in eglfs.

Additionally, the legacy environment variable QT_QPA_EGLFS_DEBUG can also be set to 1 to get additional information printed, for example about the EGLConfig that is in use.

(3)
Check file permissions. /dev/fb0 and /dev/input/event* must be accessible by the application. Additionally, make sure no other application has a grab (as in EVIOCGRAB) on the input devices.

(4)
Q: I launched my application on the console without working keyboard input, I cannot exit and CTRL+C does not work!
A: Next time do export QT_QPA_ENABLE_TERMINAL_KEYBOARD=1 before launching the app. This is very handy for development purposes, until the initial issues with input are solved. The downside is that keystrokes go to the terminal, so this setting should be avoided afterwards.

The future and more information

While the final release of Qt 5.5 is still some months away, all the new features mentioned above are there in the dev branch of qtbase, ready to be tested by those who like bleeding edge stuff. The work is not all done, naturally. There is room for improvements, for example when it comes to supporting screens connected or disconnected during the application’s lifetime, or using alternative keyboard layouts. These will come gradually later on.

Finally, it is worth noting that the Embedded Linux documentation page, which has received huge improvements in the few recent major Qt releases, has been (and is still being) updated with information about the new graphics and input capabilities. Do not hesitate to check it out.

Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in Embedded, Graphics, OpenGL, QPA

6 comments

Volki21 says:

Very interesting topic. I wonder how one could create two Qt applications that both use the kms plugin but render to different screens. As far as I know there can be only one DRM Master process, but the implementation within qkmsdevice.cpp seems to expect to be the only DRM master.

Is there any plan to allow a scenario, where two processes render via KMS (so no windowing system) but to two different screens (I guess this would only work with hacks or some additional server process that manages the KMS calls, doesn’t it).

Laszlo Agocs Laszlo Agocs says:

That falls into the “there’s room for improvement” statement for now. The eglfs backend does not currently bother with setting/dropping the DRM master privileges on VT switch, similarly to the old kms plugin.

m][sko says:

Can anybody look at Qt on lumia 1520
I don’t know if it is problem on all phones with not so accurate touch layer
but moving in listview on L1520 isn’t that fluid as silverlight apps

Another question
What if I have device with not so accurate touch layer
Can I expect that all QML elements handle touch inputs and interpolate touch movements ?
So all movements will be elastic and smooth ?

Laszlo Agocs Laszlo Agocs says:

This is completely off-topic here. It can be caused by the touchscreen, but the graphics performance and other factors are also very relevant. Remember that Qt does not use the same controls as Silverlight apps so the behavior can differ.

If the performance is seriously degraded, feel free to create a report at https://bugreports.qt.io

jiangcaiyang says:

Wow, This is a quite uesul feature! Is it possible that one can start Linux kernel and go directly to our application without any desktop environment present? If it comes true, I have sketch up a good idea, we can make our games based on Linux kernel and Qt application, at that time people who wants to play games does not need to wait for long time launching desktop environment because they always focus on our games, this is exclusive to those applications who need to explicitly launch after desktop environment. Furthermore, to create an entry on GRUB on device set up will be a good choice! Not only Linux embedded, but all devices that can launch Linux!

Laszlo Agocs Laszlo Agocs says:

Yes, keep in mind however that this will only work on systems where KMS is supported. Some vendors’ own binary drivers (say, NVIDIA) don’t. So if your goal is to reach a wide range of systems then betting only on the KMS backend is not going to be sufficient.

Commenting closed.

Get started today with Qt Download now