Qt Invaded By Aliens — The End of All Flicker

Published Thursday August 9th, 2007
51 Comments on Qt Invaded By Aliens — The End of All Flicker
Posted in KDE, News, Qt

No, this is not a wet dream — it’s all true; with Qt 4.4 you will be able to get rid of issues you have had with flickering. Just to make sure we are talking the same terminology, I’ve created a small movie demonstrating the problem:

Given the fact that we’ve passed the 90’s, this is not acceptable and up to Qt’s standard. As of today, every single widget in Qt has a native window that is managed by the window system — in this case the X server. The flicker you see on the movie is a result of multiple X windows trying to synchronize and play funny games with the X server.

But how can we improve? With Qt 4.1 we took a huge step towards resolving similar issues when introducing the backing store, but it can’t cover resize flicker as it’s fully caused by the X server. The only consequent and necessary next step is therefore to bypass the X server and let Qt do the job. In order to do that we must make our child widgets non-native — a.k.a. alien — such that X doesn’t know they exist. That sounds pretty easy, doesn’t it?

Moving away from native windows isn’t done over the night. Window handles have been in Qt from day one, and consequently our GUI kernel is heavily built around them. The main technical challenges in that respect have been core functionality such as mouse events, widget cursors, enter/leave events, drag’n’drop and so on. With native windows all widget events are related to a window handle and our job is simply to map system events into Qt events. Consequently, when we make our child widgets windowless, all events are related to the top-level window and we have to figure out ourselves which widget is the correct receiver for the given event. To make it even more interesting, all this depends on the platform your application is running on, so there’s no way it can be fixed properly in cross-platform code.

A somewhat funny scenario is drag’n’drop on Windows which is based on Object Linking and Embedding (OLE). Every drag’n’drop operation consists of at least two COM objects representing the source and the target. A widget acting as a drop target must register its window handle such that Windows can communicate between the source and the target. It’s not hard to imagine that some black magic is needed when the top-level window is the only native window we have, possibly acting as source and target at the same time.

And what happens if we suddenly create a widget that requires a window handle? Consider the scenario where a QGLWidget is the central widget of QMainWindow. Widgets with the Qt::WA_PaintOnScreen attribute set — in this case the QGLWidget — must have a native window to function properly. That’s OK, but the fun starts when we have to deal with clipping between native and non-native windows. Clipping between native windows is indeed handled by the window system, but it doesn’t help much when the only clipping we get for free is between the top-level and the GL widget.

Needless to say, but apart from reducing flickering, an application with windowless child widgets is much more light-weight and resource friendly as we don’t allocate native windows for every single widget.

I wish we could have taken this step a bit earlier, now that KDE 4 is around the corner, but there’s little point in adding hacks and workarounds now to avoid flicker with Qt 4.4 bringing the real solution soon.

I could probably tell you a lot more, but I’ll keep this blog short. Currently this work is going on in a separate research branch so it won’t show up in our snapshots, but within few weeks you should be able to try it out. I’ll blog with more details as soon as we have it integrated.

Last, but not least, here’s the obligatory movie showing how resizing with windowless child widgets looks like:

Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in KDE, News, Qt

51 comments

Max Howell says:

I love the results, such flickr really sucks, I assume it will also help where you move several widgets, and you can see the widgets move one after another (if you are moving many anyway).

I haven’t written any code that requires child widget WinIDs, but I imagine some people have, what will they do? And perhaps I’m glad if you will decide not to care, as the results in that movie are superb.

OSX I notice neatly avoids this kind of flickr. Is that just because it buffers everything everywhere?

Will Stokes says:

If I’m correct, this improvement is literally *huge*. Flicker is annoying yes, although as you said Qt 4.1 laregely fixed the most annoying flicker issue. What really peaked my interest was your comment about things being much lighter weight. Correct me if I’m wrong but widget construction should be cheaper with this change, right? If so you’ve really gotten me exicted. In the commercial app I’m writing we have some very complicated dialogs with a lot of widgets. Creating these dialogs is so painful we cache and reuse the dialogs after resetting them if we need to pull them up again. The pain is most particularly felt on OSX. I’m hoping the work you’re describing we’ll help reduce the widget construction penalty and speed up dialog and main window construction and make Qt based apps feel far more snappy.

I’m using QWidget::winId() for phonon-xine’s VideoWidget implementation. But in addition I’m calling setAttribute(Qt::WA_PaintOnScreen, true) so this widget should just behave the same as it did with previous Qt versions (I’d expect).

Anyway, great work!

Ossi says:

maxH: your world is slightly too black&white. 😉
there is nothing to prevent adding a WA_NativeWindow flag – in fact, the mentioned QGLWidget would auto-set it. not sure it is actually worth adding another flag though – i’ve yet to come up with a case where WA_NativeWindow but no WA_PaintOnScreen is needed.

bnilsen says:

Max: If you’re talking about moving of child widgets, the backing store optimizes that very well in most cases, but you might see improvements there as well.

Code that requires window ids will still magically work, as we’ll force creation of a native window when calling QWidget::winId.

Mac OSX has its own backing store which is a bit smarter than X is 🙂

bnilsen says:

Will: I don’t have any numbers yet, but widget creation will definitely be much cheaper without having to create native windows.

monkeykong says:

impressive!

I’m very happy with TT working out this problem, because i’ve been suffering for some time for not being able to put widget animation effects in my application, as flicker was making such effect actually look awful.

Anyway, how is this problem going to be solved under X? I mean, having just one toplevel window is probably not optimal because most child widget do not want all mouse movement events, but only enter/leave notifications, and getting all such events would put some load over the connection with the X server. Another idea that came up discussing about this problem was using InputOnly windows, this would solve this problem but there should still be one window per widget, and widgets would not be so ‘lightweight’.
I’m just a bit curious about which will be the final solution, maybe i’m simply overstaing the performace hit of receiving all mouse events through the connection with X.

Thanks a lot for fixing this and keep up the good work!

nick says:

this is all great. but why don’t you just improve xorg? because its code is too ugly? 🙂

bnilsen says:

Maurizio: We haven’t noticed any performance issues with the processing of events received from X. We could of course look into it and see if there’s something we can optimize, but I suspect that would be micro optimization 🙂

Rince says:

I wonder why nobody’s pointed out yet that you have the “flickerless” and “flickerful” videos swapped. 🙂

Andrew Medico says:

Am I mistaken, or will this change (no native widgets for child widgets) make Spy++ unable to usefully inspect Qt 4.4 applications?

bnilsen: good to hear, it is always nice when in real life there are fewer problems than expected! 🙂

Great work,
but it maks me sad, that It won’t make it into KDE4.0… because this one is a big one for a Desktop Environment….

bnilsen says:

Andrew: True, Winspector/Spy++ will only see the top-level window.

xbix says:

So there’s no other way than simply dropping native widgets? How do flickerless native GUIs work in general then? Or is it related to the fact, that Qt paints the widgets itself?

Ossi says:

re andrew’s comment: i guess that means another command line switch to enforce native widgets everywhere. 🙂

Andreas says:

Wow, exciting! Changing the guts of QWidget after so many versions really does not sound like a cakewalk…
Now you have us all curious, and I have a number of questions:
– Are there any approximate numbers for CPU and memory efficiency gains?
– Does it have an impact on input to display latency? That is something where I have never seen Windows beaten, and OS X was even worse than X11 when I tried it.
– How does it influence performance of remote X sessions? (OK, who cares? 😉 )
– When will the new code be released in an official version, i.e. 4.4?

Jengu says:

The second video looks like an awesome improvement and is something that has always bothered me. As a commercial product, this seems like an excellent feature for QT to have. On the other hand, as part of the general Linux software stack, might this be solving the problem at the wrong level? I’m not too familiar with Xlib, but this seems like something that should be fixed at the X level. Why is all the weirdness happening between the native child windows and the server in the first place? I’d prefer a fix that helped my GTK / Xlib / your_favorite_toolkit apps as well.

Also, wasn’t XCB supposed to help with this problem somehow?

A few questions:

1. Did you tryed to improve XOrg? Why did you deamed that possibility?

2. What about resizing the main window? I mean currently, when you resize a window, the border resizes, and the content keep its old size for a little moment. Windows would feel more “rock solid” if the window manager waited for the main window to be reflowed & repainted offscreen and then the window and the resized border be painted all at once. I think GNOME solved that with Compiz or something else (I don’t remember exactly). Of course, it’s a KWin problem (perhapse fixed in KDE 4) but this involves the same X11 protocol adaptation as your hack requieres to have flicker-free widgets, IMHO.

3. When will Qt use XBL (or was it XCB?), the replacement of XLib, but asynchronous, this time, so Qt does not wait X11 replies for every single operation? This would result in huge latency improvements! We heard about it a few years ago but it seams dead: non news lately.

Andreas says:

As long as X11 is asynchronous (with all its benefits, by all means), resize and move operations will also be asynchronous, and flicker will appear as X11 first clears the exposed area (or leaves a trail of junk behind), then sends an expose notification, allowing the client to redraw. You can probably avoid the flicker with clever use of XFlush, but that will mean performance goes down the drain. With this solution, we don’t only remove the flicker, and remove the need to wait for X11 roundtrips, but we also free up X11 resources as Qt no longer needs the subwidgets on the server side. Which means less memory consumption :->. And on Windows, it means people can create more widgets than they could before…

GN says:

Max Howell. I’m not a grammar nazi, I just thought it was fun that you spelled flicker as flickr 🙂 Strange how we are influenced by brand names. It’s simply del.ici.us

Jengu says:

@Andreas: The problem with X11 roundtrips I believe is what XCB is supposed to solve. It returns handles to results rather than results directly, so you don’t have to block until the result is actually used (the handle is “cashed in”). Again, I am not an X hacker though 🙂

For whom it may be interesting:

Here is another windowless toolkit: http://harmonia.terrainformatica.com/.
That is Harmonia toolkit written in D language ( http://www.digitalmars.com )
Live demos (windows) are here: http://harmonia.terrainformatica.com/HarmoniaDemo.zip (1.7 MB)

Matthias says:

Sebastien, Xlib is by far not as synchronous as you imply it was. Creating an incompatible X-server was never an option, no. It wouldn´t help existing customers (this server will never make it onto commercial UNIX systems) and it wouldn´t help our Windows users. Our solution now is cross platform, not restricted to X11.

Ossi says:

i’m confused … XCB replaces Xlib, and it is still X11-based. there is no reason why it could not be another lib in 3rdparty/ for those antiquated commercial UNIXes.
whether it would be actually such an improvement is another question …

Max Howell says:

At the end of the day it is impractical to expect Trolltech to spend the time fixing X.org. The patch may be simple, but the social requirements ridiculous. It is very hard for one project in the commercial or open source world to influence the direction of another.

Far simpler is it to implement what you need and hope the other project notices and decides to do it too.

Trolltech did the right thing. Don’t hold it against them that they didn’t try or wait to get X.org to do it, it’s bloody hard.

dejavu says:

Dejavu. Delphi does this on Windows, too – for more than one decade now. With the difference that it came with a tool to follow native window messages and its own ones as well, iirc (don’t remember the name, didn’t code in Delphi for a long while).

What about the downsides of this approach? E.g. protocols like NX base their compression on the X11 window changes, don’t they!?

skierpage says:

Xlib is implented as a layer on top of XCB in X.Org 7.2, see http://www.x.org/wiki/Releases/7.2. Using XCB directly can eliminate some synchronous requests and server round-trips, but I’m sure TT is already smart about minimizing these in Qt.

bnilsen, “tell you allot more”: “a lot” is two separate words, just like “a bit” and “a little”. (Your English is way better than most native speakers!)

Will Stokes, “really peaked my interest”: that’s “piqued” as in hooked, it’s from the French.

I’ll stop now, this fundamental re-engineering is damn hard and you don’t have time for grammar nazis.

Martin says:

How does this affect performance when running X over a slow network link? I was concerned that it might be slower in some cases, since the X server can no longer simply rearrange mostly unchanged sub-windows when the parent is resized, but rather has to ask the client to redraw the whole thing. I am not so familiar with the inner workings of X so this may be totally of the mark.

Nevertheless; did you profile this change for the communication-limited case? Thanks!

felix says:

it looks like the relationship between swing and awt of java.
but there are a z-order problem if we using awt’s heavy component and swing’s light component in the same top-level window.
the suggest from swing is that do not using awt and swing like this way.
how do you deal with this problem on QT?

NoTiG says:

can someone tell me if this knowledge applies to gnome? I seem to have jagged windows problems when i move them around and it annoys me so, despite having a powerful machine

http://www.youtube.com/watch?v=8gh7ABSs1NY

thomas muders says:

Not only Delphi does it on Windows, but I think a lot of bigger Windows applications like IE and MS Office also do it, I remember some blog entry by a MS developer that they had to do it because of performance reason and because there are in fact hard limits on the number of native widgets.

Moma says:

The XCB library is going to replace the old XLib.

Do you think that XCB will give better performance and drawing speed?

See:
http://xcb.freedesktop.org/

http://en.wikipedia.org/wiki/XCB

Moma says:

Sorry…
XCB was already discussed by others in the earlier postings.

gogoliu says:

I think widget system should base on canvas.

lu_zero says:

Hmm, what about providing links for those who don’t use flash? (yes there are some still around)

nick says:

lu_zerolu_zero: apt-get install youtube-dl

dpeterc says:

The gray old ugly bloated eighties Motif can do resize without flicker (*).
So the problem might not be in X Windows, but in the way Qt uses it.
You get rid of majority of flicker, if you do not clear window on
resize, but just paint into it. And if the Qt’s geometry management and
draw code would not be slow, even a full repaint would not be noticeable.
So why not just profile the Qt, see where time is lost, and fix that?

(*)
Check my application ArahWeave from
http://www.arahne.si/installAWeave.html
Run ArahWeave, go in Thread pattern -> Colors,
http://www.arahne.si/colorsPantone.gif
and resize this dialog.

Matt says:

dpeterc:
Isn’t that exactly what they’ve done here? Profiled Qt, saw that the problem was “a result of multiple X windows trying to synchronize and play funny games with the X server,” and then fixed the problem.

Uno says:

Well, nothing new to me. Ultimate++ uses non-native windows for child widgets since its beginning. And it really works. Like in the movie from qt 4.4

darkob says:

I think that removing native child windows is definitely right way.

I’m coding small user interface library targeted for people that don’t want flickering in their applications. Of course, it’s not so complex like Qt, but it uses non-native child widgets and it’s interesting project to demonstrate power of this idea. I noticed great speed improvements in the desktop and totally fast repainting (I’m implemented small set of reapint functions myself that’s CPU optimized, but Qt also has similar ones).

Another way to speedup Xlib based application is to use XShm extension. But I thing that Qt disables this by default (why…?).

See http://www.wdesktop.org for the library, but it’s not completed, so only usable as a real demonstration how non-native widget repainting is fast. If anybody is interested about implementation, send me mail to kobalicek dot petr at gmail [dot com].

Raj says:

Am I the only one to think that such a major change in the QT architecture requires a different major version number? I love QT and KDE, but I would appreciate if TrollTech brings out the not-native widgeting scheme in Qt 5.x series instead of confusing a lot of people by introducing it in 4.x. That would be make life easy for developers, administrators and package-maintainers to explain the PHBs (who are also a part of our IT industry).

motif says:

I guess what is old is new again. The XToolkit(Xt) had this in the eighties with the concept of Widgets and Gadgets. Widgets had physical window space on the XServer, Gadgets were less overhead versions that did not have any physical window.

Andreas says:

Raj, if we can do a change like this while keeping the library binary and source compatible, we’ll keep the major version number. If we bump the major number, everyone will think we broke compatibility somehow, apps and distributors would resist moving to this version, and this wonderful feature wouldn’t be in your hands for years. Unlike some other software vendors, we don’t use version numbers to create hype, we use them to let people know what’s changing.

Elias says:

@NoTiG

Turn on a composition manager (even an oldie will do, like xcompmgr).
You don’t need to use any of the super-duper-ultra effects

Graeme says:

I don’t use Qt or C++, but like the developer friendly API etc. I’m a Object Pascal guy. 🙂 I’ve implemented a x-platform gui framework called fpGUI (http://opensoft.homeip.net/fpgui/). My first design was using ‘windowless’ widgets. It ended up being a nightmare handling events (distributing events), manually handling clipping etc… My experience was the oposite to what you experienced. Repainting was very bad. Two months ago I rewrote the GUI code but this time using a ‘windowed’ widget design. After all, Xlib and GDI was designed to handle messages, clipping, etc. My code is now much cleaner and easier to maintain. All painting gets done via double buffering and even resizing a complex window with dynamicly resizing widgets I hardly see any flicker. Not nearly as bad as your video showed. Why are the button on the right of the screen disappearing like that!?!

I think Qt got something wrong somewhere and X11 isn’t to blame. Maybe the layout managers or something. Also why is it the my fpGUI widgets don’t flicker and neither does other toolkits like Motif which is also based on a ‘windowed’ widget design? Also I can show ‘windowless’ toolkits that flicker as well, so that isn’t the holy grail of designs.

I do agree that resource allocation will be improved and that is well worth the effort for mobile devices, but desktop applications don’t have that limitation anymore – well, like they did in the early GUI days at least. My new design under X11 uses delayed resource allocation. Which means when I create a widget, resources are not created yet. Only when it actually becomes visible does it allocate resources. This allows me to send more setup information in one go to the X Server instead of many smaller calls. This improved speed as well, especially over a remote X Server connection with a bigger lag time.

bnilsen says:

Graeme:

We have a fairly decent backing store which handles clipping and painting very well, and regardless of whether we use native windows or not, almost the same code path is executed. Having native windows on top of that is just overhead and would only give us flicker for free. When it comes to event handling it gets a bit more complicated, but it’s absolutely doable. In the end there are no problems, just solutions 🙂

Martin Schreiber says:

I did 3 approaches to develop a pascal based GUI framework, the first was based on VCL, the second on CLX both with one window handle per widget. Based on the experience I get with the development of them, I decided to start from scratch for the development of MSEide+MSEgui with only one window handle for the toplevel window/widget. I never regret it. For the handling of transparent or semi transparent widgets I used many weeks with the multi-window-handle approach and it never worked without glitches, in MSEgui the implementation of transparent widgets was done in 10 minutes, believe it or believe it not.
If you are interrested in MSEide+MSEgui:
http://homepage.bluewin.ch/msegui/

Martin

Dumb User says:

As dumb application user I never noticed it as a real problem.

matthias says:

so everyone is going the lightweight way…
It’s funny, I just got a bugfix mail today from Sun telling me they finally fixed the lightweight / heavyweight clipping problem in java! It only took them a few ages….

Commenting closed.

Get started today with Qt Download now