Richard Moe Gustavsen

Multi-touch on Mac

Published Tuesday June 23rd, 2009
12 Comments on Multi-touch on Mac
Posted in Labs, Qt

Not long after Brad blogged about multi-touch in May, I got my hands on a brand new MacBook Pro under the excuse of implementing multi-touch support for Qt on Mac. This machine has a big and sexy trackpad (excuse my sexual orientation) that goes beyond normal mouse control to also include multi-touch- and gesture support.

After jumping on to the implementation, my first thought was that this was going to be a fairly quick thing to do. The API from Apple is rather straight forward. But after a short while, we started to realize that a touchpad (or trackpad in the case of a Mac) is more different from a touch screen than originally anticipated. The main difference being that on a touch screen, you place your finger directly on the widget you want to interact with. Qt then identifies that widget as the receiver of any following touch events. With a touchpad, you cannot place your finger directly on any widget in a similar fashion. You need to move the mouse to the widget first using the touchpad. This means that your first touch (and all following touches) will go to the widget under the cursor when the move starts, and not to the widget you are navigating to. So you would then need to do an extra step of lifting all your fingers and put them back-on the pad once the cursor is over the correct widget. In practice, this feels wrong and unintuitive.

After some hallway discussions and try-outs, we found that which widget to send touch events to should be decided after the second finger has been pressed. This is almost the same as saying that you will only get touch events when you are not controlling the mouse. This will normally be enough for applications that uses multi-touch for implementing gestures. If you are implementing single touch gestures, or otherwise need to handle the first touch, we added a flag you can set to override this default behavior. On a touch screen, you will always get all touch events.

A second problem regarding single touch (on a touchpad only) is the process of mapping touchpad position to screen position. When the touch begins, it is easy: screen position should be where the mouse is. But then it gets more blurry. The mouse will now move on the screen with a user defined speed and acceleration. So when your finger moves two centimeters on the touchpad, the mouse might move ten centimeters on the screen. In that case, should the screen position of the touch move with two centimeters? Or should the touch follow the mouse? The user would normally expect the latter (since there exist a logical connection between the touchpad and the mouse), especially if there are no visual clues on screen to mark the touch. However, implementing a strategy like this will grow in complexity (and inconsistency) once you start moving more fingers on the touchpad. And besides, recognizing gestures with mouse acceleration applied is not for the faint of heart. Conclusion: a touch will move on screen 1:1 with the touchpad. If you need to know where the mouse is, use mouse events.

In some cases, it might be more useful to know where the fingers are on the touchpad, rather than on the screen. To accommodate for this, we added to the touch events what is called a normalized position. This position range from zero to one, with the upper left corner of the touchpad being (0, 0), and lower right being (1, 1). With this information you can implement your own screen mapping, if you like.

Finally, there are certain limitations for using multi-touch. The API available from Apple is only for Cocoa, and only for Mac OS 10.6 (Snow Leopard). As a result, multi-touch for Qt/Mac will only be available for the Cocoa port running on OS 10.6. On the upshot, gestures are supported starting with OS 10.5.2, both in Carbon and Cocoa. Qt will of course mirror the same support.

Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in Labs, Qt

12 comments

Will Stokes says:

Very exciting. I can’t wait to implement gesture support (specifically three finger swipes and “the pinch” for zooming) to our app. I suppose basic gesture support for OS X 10.5.2 won’t get into Qt until 4.6? It would be great if gestures could be squeezed into 4.5.x while full multi-touch (finger painting) could be delayed until 4.6.0.

Andreas says:

Will: Unfortunately we can’t add features to 4.5.x :-).

Kensai says:

Impressive. After seeing such demos on the desktop I’m asking myself when Apple will allow Qt to be targeted on the iPhone. Having Qt to create applications for both S60 and iPhone would be amazing.

Is there any hope? Any formal talks?

Anon says:

“I’m asking myself when Apple will allow Qt to be targeted on the iPhone.”

Are Apple doing anything to prevent Qt on the iPhone? (Honest question :))

Will Stokes says:

@Kensai: Ah, the dreaded question I never dared to ask. I was somewhat optimistic before Nokia bought Trolltech, but now that Apple is a direct competitor to Nokia I doubt this will happen any time soon. Such a shame. Case in point, I see potential useful extensions of the desktop application I’m developing w/ the iPhone. It would be neat to sync data to the iPhone which you could then take with you. Qt on the iPhone would be quite natural since our desktop app is Qt based. I can think of other situations where this would also make sense, even mundane ones like a recipe organizer with built in shopping list builder that could be synced to the phone and carried with you to the store. You obviously could accomplish this by writing the iPhone part using Cocoa Touch directly, but it would be easier to just use Qt throughout and by using Qt you could also support other smart phones as well.

boulabiar says:

Very nice,
Now having the multitouch support in Linux kernel, this will simplify the handling of it, specially with Gesture support.

Kensai says:

Ah, the dreaded question I never dared to ask. I was somewhat optimistic before Nokia bought Trolltech, but now that Apple is a direct competitor to Nokia I doubt this will happen any time soon. Such a shame.

Why can’t we all just get along?! 😀

Perhaps the Nokia administration is not interested in actively pursuing something like that, but I’m not sure if the open source nature of Qt prohibits some genius developers to try their luck in adding that functionality and promote it to be implemented (merged or even third-party plug-in) in future Qt toolkits…

Richard’s videos on special Mac functionality (for the desktop) makes me wonder. Imagine a future “Qt Mobile” doing for the mobile world what Qt has done for desktop. Imagine using Qt to write mobile applications that can be easily ported from S60 to Windows CE, iPhone, Maemo, Android, etc.

*sigh*

Kensai says:

“Ah, the dreaded question I never dared to ask. I was somewhat optimistic before Nokia bought Trolltech, but now that Apple is a direct competitor to Nokia I doubt this will happen any time soon. Such a shame.”

Why can’t we all just get along?! 😀

Perhaps the Nokia administration is not interested in actively pursuing something like that, but I’m not sure if the open source nature of Qt prohibits some genius developers to try their luck in adding that functionality and promote it to be implemented (merged or even third-party plug-in) in future Qt toolkits…

Richard’s videos on special Mac functionality (for the desktop) makes me wonder. Imagine a future “Qt Mobile” doing for the mobile world what Qt has done for desktop. Imagine using Qt to write mobile applications that can be easily ported from S60 to Windows CE, iPhone, Maemo, Android, etc.

*sigh*

Thiago Macieira says:

@Anon: yes, Apple is doing something. Apple doesn’t allow libraries to be installed system-wide on the iPod Touch and iPhone (jailbroken devices don’t count). Sure, you can add any library to your app’s bundle. But if you bundled Qt, it would make every app using Qt very big, and they wouldn’t share any data.

Now, I doubt that any amount of talking from our (Nokia) side will convince them. However, if enough app developers were to talk to Apple and request system-wide library installation, then maybe we could get somewhere. So it’s up to you to talk to Apple.

Anon says:

@Thiago:

I see – thanks for the information!

trenton says:

@Anon

Just to clarify what Thiago said. Apple currently doesn’t even allow you to have shared libraries in your “bundle,” they have to be static. While a big download is less of an issue on the desktop, it can be a problem when you are downloading over EDGE.

Just because I’ve been asked this questions a few times. Here’s some extra bits.

For people who really are tenacious. It’s not impossible to get things like QtCore building for the iPhone (emulator at least). You need to put some thought into what you are doing, but the biggest bear seems to be making it all play nicely in Xcode since Makefiles and iPhone development don’t mix. GUI is a much more difficult problem, and that’s where porting work would have to begin. Also, there’s no theming API on the iPhone, so a lot of work would be spent either re-architecting how Qt works or painstakingly drawing the iPhone widgets by hand. That emulating will also add to the size of the library.

This information could be found by looking at the Qt and iPhone headers+docs, so this is nothing shocking.

There, that’s about all I can say regarding the iPhone and Qt from a programmer’s point of view. However, the (needless) ballooning of apps is probably the biggest show stopper of why Qt doesn’t make the jump.

There will be Qt for all these nice S60 phones, so all is not lost 🙂 You will likely be having to do /some/ redesign of your desktop GUI for a phone anyway. So, if you feel you must have an iPhone version to go along with S60 as well, then Cocoa Touch is much easier to program in then, say, Xlib or Win32. Assuming you can get over the Objective-C syntax.

aclassifier says:

I have filed a similar question at the Apple dev forum. For those who are members there, see https://devforums.apple.com/message/93373#93373. However, what it really does is to point to this discussion, since the only reply I received there was “Ask Nokia”.

Commenting closed.

Get started today with Qt Download now