A new Symbian toolchain for Linux

Published Wednesday October 28th, 2009
21 Comments on A new Symbian toolchain for Linux
Posted in Build system, Git, Qt, S60, Symbian

One day I was sitting at my computer, waiting for my build of S60 to finish. I had run my usual round of build commands and custom scripts to speed up the build if but a little, and was expecting to wait for at least 10-12 min. At this point it occurred to me (well, it had occurred to me before): The Symbian build system is really overkill. What could it be doing that’s taking so long, for something that should be relatively simple: compile and link?

Symbian and Tux

So I thought to myself that this can definitely be improved, and that’s how I started the work on a Symbian make specification that is based entirely on qmake and make (no Symbian build chain stuff, although it uses the tools from there), and runs on Linux.

Then you might say “What about the Raptor build system that is supposed to improve things?”. It is true that Raptor improves a lot of the deficiencies of good ol’ abld, but I still felt that the huge Raptor build system is sooo overkill for something really simple. I like to obey the KISS principle. Call it a personal itch!

With this in mind I have two design goals:

  1. Ditch all unnecessary file types. This includes bld.inf and MMP files.
  2. Be fast!

Whether I’ll be able to fulfill them in the end, time will show, but that’s what I’m aiming for at least.

But let’s get into the gory details. Be reminded that the build system is still in the very early stages, and is not ready for for the end user yet, but if you like to tinker with the latest and greatest, read on. Here’s what works currently:

  1. Running configure.
  2. Building QtCore.dll with RVCT.
  3. Running QtCore.dll
  4. Building user applications with RVCT.
  5. Running user applications.

Here’s a few things that don’t work yet, but I plan to get them working:

  1. Building the host tools automatically (this needs to be done manually ATM).
  2. Building QtCore.dll with GCCE (this doesn’t work with the official port either, but as soon as it works there, it should work here as well).
  3. Other Qt dlls. QtGui.dll should probably be easy to get compiling, but it needs a few header fixes first.
  4. Building user applications with GCCE.
  5. Making the process of building a package automatic.

Here’s one thing that will probably never work:

  1. Building for the emulator. The emulator won’t run under Wine, so there’s no point in building for it. If the build system gets ported to Windows maybe it’ll be supported, but not before then.

The procedure for getting the system up and running is subject to change as the project goes forward, so instead of posting it here, I have put it in the README.s60-mkspec in the Git repository. To get it, go to http://qt.gitorious.org/+qt-developers/qt/s60-linux-mkspec, and clone the repository from there. Then check out the “working” branch. This branch is often rebased based on the latest work in the topic branches. “master” should not ever be rebased, but many things are missing from there at the time of writing.

Here’s a few short term goals for the project in the following weeks:

  1. Get the other modules compiling.
  2. Get the process to be more streamlined (e.g. more like the other platforms)
  3. Move from ABIv1 to ABIv2. The former is the old way of linking binaries on Symbian. Since this is being superseded by ABIv2, which is binary compatible at run time, but not compile time, it makes sense to only support that.
Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in Build system, Git, Qt, S60, Symbian

21 comments

Axel Jäger says:

This is so great news! Finally besides the lots of people complaining about the speed of the symbian toolchain someone started over. I am happy to see that this person is from Nokia Qt Development Frameworks. Hopefully more and more people inside at Nokia get infected by KISS.

detro says:

Well, kamlie, what can I say.
If I meet you I’m going to kiss you. So, be aware 😛

Seriously speaking, the Symbian Toolchain was designed by a set of crazy monkeys that pushed buttons on the keyboard until they managed to build. A great team!
I know I can upset some people here, and I’m theoretically sorry, but there is really no excuse for such ashamed piece of… being part of the “Most used Mobile Platform in the Solar System” (or whatever is the slogan that Symbian uses for itself).

As Axel says, I’m glad the person undertaking the responsibility for such a project is a Troll: smart, honest, productive and passionate guys.

AlexBravo says:

Kamlie, it looks like you want to make a new build system that’s better than the old one.
If it is to be better than the old system, why do people need two systems?
If you don’t need two systems, then the new system needs to work everywhere.

Then this statement:
> If the build system gets ported to Windows
needs to be rewritten to
“When the new build system gets ported to Windows”.
What do you think?

kamlie says:

Thank you for the kind words!

AlexBravo: I suppose you have a point, hehe. It is the long term plan to port the system back to Windows, but right now Linux has priority since Windows already works. And about one single build system: I think it will take a long time before the rest of Nokia and Symbian can adopt a new build system, so the two will probably have to be run in parallel for a while. But we’re taking one step at a time, so one may hope!

Axel Jäger says:

Has anyone thought about getting it running on a mac? For my last symbian project, I did a fake-emulator to be able to develop using creator on mac: Using a compile time define, I wrapped my main widget into 5800-skin and had a small controll panel next to it to fake orientation change events. I would be so happy If I would no longer have to use a vista-vm to do a device build.

kamlie says:

Axel Jäger: From a build system perspective it should be quite easy to get it to run on Mac. The question is whether the Symbian tools themselves will run. But I hope that a Wine-type solution can be used also here.

Tim G says:

Sounds great, although in fairness to the SBSv1 and SBSv2 (Raptor) the functionality is needed … if you are building the OS or an SDK. The problem is that using the full-fat build system creates a tax for application developers in terms of complexity and performance. Many of the Symbian OS APIs have the same problem, they were designed for OS developers not application developers.

Anyway, I like the KISS approach but now Symbian and tools are open source let’s try to collaborate, re-use where things work and replace if they don’t. I want a simple, fast tool chain for application development … but not 10 incompatible tool chains 🙂

I work for the Symbian Foundation. An entirely new tool chain based on GCC, GDB and QEMU is under way. Obviously it will support Mac OS and Linux. This should be complete in Q1 2010 and the Symbian^4 SDKs are almost certain to be officially provided for all 3 platforms.

This does not in any way address the complexities of the bld.inf/mmp system, so providing an alternative pure Makefile based system is a good idea for simple projects.

Nils says:

Android 2.0 is there.

It’s hard to believe Nokia is still working on ToolChains and animated Widgets.

Nils

Nils says:

Hi Allesandro, we talked about that after the Qt Creator Bootcamp talk on Developer Days, maybe you can remember me 😉
Please don’t get me wrong, I really hope this all will finally pay off. I’ll be one of the first to replace my Fruit Device with a Nokia Nwhatever if you’ve got all those things sorted out.
Maybe causing negative attitudes/comments is the downside of such an open development process. But please carry on nevertheless.

Good Luck!

Nils

espenr says:

@sebastian: I love how you put Qt in the “simple projects” category 🙂 You realize this toolchain, once done, will be able to build Qt _the library_, not just Qt applications?

Timothy Murphy says:

Hi,

I did what you are doing in 2006 but I couldn’t give it away because I was working for a company and they owned my work – it built a few target types and not properly as I found out later but near enough for me personally. Then I went to work for Symbian to produce Raptor. This is the world of open source so I’m glad that you are working on alternative although I’d really prefer to have your help on Raptor to be honest – if you are ever interested then let us know because it’s now under the EPL license.

Raptor has to support legacy formats otherwise you’d never see any change over from abld. It also has to build everything and that’s where the complexity is. Your problem will be what my problem was when I sat down to start on a little makefile – that it got bigger and harder the more types of program we tried to build.

But one thing is a little misleading. I don’t know what version of Raptor you have used (since new versions come out every two weeks) but it’s fast on the desktop and extremely simple to use. At the simplest you do this:

cd group
sbs

.. and it uses up 4 cores of your machine to build your code quickly. The big features are pretty simple too – if you have several linux machines set up you might have to introduce something major like:

sbs -e pvmgmake

To start a build that uses them all. Not too hard.

It’s incremental but there is still a lot to improve when you only expect to build one or two files. I wish you would help us with this – we have a lot of “pull” from the people trying to do huge builds and not so much from people like you who are actually prepared to do something.

Regards,

Tim Murphy

Marius says:

@Timothy: I’m sure we’d be able to make qmake build these “huge” projects just fine. Qt isn’t exactly small, and qmake is capable of building WebKit after all. 😉 BTW, I tried Raptor on the SDK v5 just last week. Didn’t work. Turns out that Cygwin doesn’t like my Windows 7 machine, and even after beating in a new Beta version of Cygwin, it still didn’t work :-/ I would have loved to use it over abld, but oh well, that’s Cygwin for you; works fine on one machine, barfs on the next. (Relocation issues of the DLLs)

tnmurphy says:

Hi,

We’re not fortunate enough to have any Windows 7 machines to test on. Would you mind letting me know what build number you have so that we can have a go at getting it and reproducing the problem? I read reports about cygwin success and failure on google which refer to Windows 7 build numbers so I want to test on the “right” one and know if it makes any difference.

You’re right, BTW, you can build anything with anything if you put some effort into it. It’s just that we have done that. Qt is small only when compared relative to the rest.

Cheers,

Tim

kamlie says:

@Timothy Murphy: It’s good that Raptor supports the legacy formats. That makes it a good replacement for abld. However, the whole idea with this build system is to leave all that legacy behind and start fresh. It certainly will not be able to handle many existing Symbian application builds with complex setups, but it will support new ones in its own way. Another incentive is that both abld and Raptor feel very alien to many non-Symbian developers, and can be time consuming to set up. If we can make it easier for them by making the setup more lightweight and familiar, we might attract more developers.

Marius says:

@Timothy: I’m using build 7600 (so, Microsoft Windows [Version 6.1.7600])
That being said. Another Windows 7, build 7600, x64 machine (which mine is), with mostly the same setup, only that I also have all the MSVC compilers installed since VC 6.0; is running Raptor just fine. So, it’s the fact that Cygwin on my machine has some colliding DLLs so Cygwin tries to relocate DLLs, which fails.. (as far as I can tell)

Timothy Murphy says:

@Markus – thanks for the information. Hopefully we’ll find out some way of detecting the problem you’re experiencing, perhaps the particular DLLs and that might give us some way to deal with it or at least find some workaround.

@Kamelie – Don’t be put off by me – my team has spent 3 years doing this and it has been an immense effort so I’m bound to come across as slightly annoyed at the idea of it being reimplemented quickly. Even so, I am sort of grudgingly glad that you are trying it out.

We are not fond of our legacy formats either and would love to have had the permission to get rid of them. But as we have dealt with them we have come across hundreds of unspecified, barely-documented features that turn out to be actually useful and needed by one type of program or another in S60. So what we’d like (as individuals, not officially) is to reinvent these formats in a way that was truly clear and simple and obvious but not losing the concepts that are needed.

Anyhow, keep at it. You might find that some of the FLMs (function-like-makefiles) in the Raptor lib/flm directory are useful when you are trying to find out how to do something. e32abiv32.flm is the biggest and most complicated but it does a lot. GCCE support is on it’s way too (a bit beta in 2.10.0) and that might be more useful to you.

lizardo says:

Nice work! I hope to see it fully building Qt for S60 soon :). I have some questions:

* Is it possible to get the symbian SDK location from EPOCROOT environment variable instead of having to edit qmake.conf ? Does qmake support reading environment variables from these mkspecs ?

* I see that you add gnupoc as a dependency to build Qt for S60 following this scheme. But, if I understand correctly, your approach is kind of different and seems not to be using any of the native tools provided by gnupoc (e.g. the codesourcery toolchain, the modified make, and the various native tools and wrapper scripts), just the headers and libraries from the SDK itself. Am I right?

Therefore isn’t simpler to provided your own script that:
– unpacks the Symbian SDK in some directory
– do the filename case fixes

* I wonder how you actually test the built Qt libraries on the device. So far I not managed to get working libraries built by myself. First I had various capabilities and UID/SID/VID issues, but after fixing all them (i.e. reducing capabilities to the ones allowed by self-signed certificates, and changing IDs using elftran) the resulting sis is installable but does not work at all (the application seems do crash or does not open). User applications, on the other hand, build and work just fine. I use RVCT 2.2.686.

Jussi says:

Try scons for symbian toolchain, it was able to reduce 9 minute abld build to less than 3 minutes alone and after adding ccache, the time was reduced to less than 2 minutes… And it works on Linux too.

http://code.google.com/p/scons-for-symbian/

Tihomir says:

Yup, here are some statistics of building my project with scons for symbian… 9min with ABLD…. 5:30 with scons -j 1… On my dual core laptop, scons -j 4 does job for 2:30…. And with distcc scons -j 14 it does whole build in 1:30… Linux machines running distccd are also building for me 🙂 Next step is to use pump distcc scons to get below 1 minute 😉 Good is that I found a way to build/clean from Carbide which also installs and start application (with dlls) in debug mode so I can step in my code 🙂

kamlie says:

lizardo:
> Is it possible to get the symbian SDK location from EPOCROOT environment variable instead of having to edit qmake.conf ? Does qmake support reading environment variables from these mkspecs ?

Most likely I’ll be using the EPOCROOT in some way, although I haven’t quite decided how yet. The reason is that EPOCROOT has a slightly different meaning in this build system. For example, generated files do not end up there when they are built. Instead, the epoc directory is treated as an install target. This means that libraries and includes may not always come from EPOCROOT, but from wherever you built, although the standard S60 ones always will, of course. The advantage is that the SDK does not get cluttered with build files all over, and you can reuse the same SDK for many different builds.

> I see that you add gnupoc as a dependency to build Qt for S60 following this scheme.

Yes, I didn’t want this dependency in the beginning, but there is one simple reason why I had to: The resource compiler won’t work under wine (it runs, but produces wrong output), so the native Linux port of that tool is necessary. Otherwise you’re right, I don’t use most of the other tools.

> – Therefore isn’t simpler to provided your own script that:
> – unpacks the Symbian SDK in some directory
> – do the filename case fixes

Oh yes! 🙂

I want to have that, but that’s more of a high level requirement at the moment. I need to get Qt on its legs first. 😉

> I wonder how you actually test the built Qt libraries on the device. So far I not managed to get working libraries built by myself.

Yes, I had to do quite a bit of manual labor to get the device successfully on the phone. If I had to guess, I would say it is due to the vtables not being imported correctly. Your “LIBS” statement probably specifies “-lQtCore”, however you need to add “-lQtCore.lib(VtblExports.o)” to that. This will of course be automatic later on, or go away entirely with the switch to ABIv2.

Commenting closed.

Get started today with Qt Download now