Qt/3D features in Qt 4.6

For the last year, we have been investigating API's that Qt needs to support 3D applications and clever 2.5D effects with OpenGL.  When we started all this a year ago, the problem was broken down into three main areas:

  • Enablers - Basic building blocks like matrices, shaders, vertex buffers, etc.
  • Portability API - API's that make it easier to write code that ports between desktop OpenGL and embedded OpenGL/ES.  Particularly OpenGL/ES 2.0 which does not have a fixed function pipeline.
  • Real 3D - API's that take Qt into new application spaces beyond animations and 2D effects.

Obviously that covers a lot of ground, so in this post we will just focus on a few of the Enablers - specifically the ones that made it into 4.6 as the first taste of Qt/3D.  In future posts, we'll publish Qt/3D repository details and show you more of our plans for later Qt/3D releases.

Math3d

Traditionally, Qt has relied upon the OpenGL library to provide mathematical primitives, using functions like glOrtho(), glRotate(), and so on to manipulate matrices and vectors.  However, with the advent of OpenGL/ES 2.0 it is no longer possible to rely upon the OpenGL library to do the heavy-lifting - the programmer has to do all the work. Also, the traditional OpenGL functions are really only useful when drawing objects - they aren't of much use when building object meshes in memory and transforming them prior to uploading to the GPU.

So we really needed a hardcore 3D math library, just like the other 3D toolkits (Coin3D, Ogre, OpenSceneGraph, etc).  But we didn't want to go overboard - it is very easy to re-invent all of linear algebra and lose sight of the core goal: make typical 3D mathematical operations fast and elegant.  We recognized that libraries like Eigen were very good at doing everything in mathematics, but our own goals were more focused.  So what did we do?

The central workhorse is of course QMatrix4x4, which is highly optimized for 3D operations.  Internally it keeps track of its "type" - whether it is a translation, scale, rotation, etc - so that it can more efficiently build up transformations than a naive "make matrices and multiply" implementation might.  QTransform does the same thing for 2D transformation matrices. The following is an excerpt from the hellogl_es2 example in Qt 4.6 which builds up a modelview matrix and sets it on a shader program:

QMatrix4x4 modelview;
modelview.rotate(m_fAngle, 0.0f, 1.0f, 0.0f);
modelview.rotate(m_fAngle, 1.0f, 0.0f, 0.0f);
modelview.rotate(m_fAngle, 0.0f, 0.0f, 1.0f);
modelview.scale(m_fScale);
modelview.translate(0.0f, -0.2f, 0.0f);
program1.setUniformValue(matrixUniform1, modelview);

As can be seen, it is very similar to the traditional OpenGL functions:

glRotatef(m_fAngle, 0.0f, 1.0f, 0.0f);
glRotatef(m_fAngle, 1.0f, 0.0f, 0.0f);
glRotatef(m_fAngle, 0.0f, 0.0f, 1.0f);
glScalef(m_fScale, m_fScale, m_fScale);
glTranslatef(0.0f, -0.2f, 0.0f);

The choice to make the functions similar was deliberate: code that uses the existing OpenGL functions can be quickly converted into more portable code that uses QMatrix4x4.

The QGenericMatrix template is used for creating "other" matrix sizes that commonly crop up in OpenGL work: 2x2, 2x3, 2x4, 3x2, 3x3, 3x4, 4x2, and 4x3.  It can do a lot more of course, being a template, although we did draw the line at supporting sparse matrices - the matrix sizes that occur in 3D code are rarely very large.  A common question is why didn't we make QMatrix4x4 an instance or subclass of QGenericMatrix.  The main reason is performance - the 4x4 class needs to be very fast and it is easier to performance-tune a concrete class that isn't at the mercy of the compiler's template expansion system.  The other reason is to reduce user confusion - the API's for all QGenericMatrix sizes is exactly the same, but QMatrix4x4 is extremely rich in the additional operations it provides.

QVector2D, QVector3D, QVector4D provide vector classes of various sizes to complement QMatrix4x4. An interesting feature for the purposes of OpenGL is that these classes are guaranteed to use the same floating-point type internally as GLfloat on the system. QPointF wasn't suitable for our 2D vector needs because it uses qreal, which can either be float or double depending upon the compilation flags passed to Qt's configure. The GLfloat guarantee is very important when building large 3D object meshes: you want to get the vertex data into the most efficient format as early as possible. If we had made the internal type qreal, then Qt/3D would have needed to do a lot of floating-point conversions when uploading vertex data to the GPU.

The QQuaternion class is the last in our current math3d set. It provides an efficient implementation of rotations in 3D space for use with camera positioning, rotation, and animation.

Shader Programs

The fixed function pipeline in OpenGL is getting very "old school".  These days, OpenGL is all about shaders, shaders, shaders.  But resolving the extensions and managing the compilation, linking, and use of shader programs can be quite daunting.  In Qt 4.5, we had no less than three different internal shader program wrappers for pixmap filters, the OpenGL2 paint engine, and the boxes demo.  So in Qt 4.6 we have merged all of these efforts and devised a new public API to wrap the extensions.  The result is the QGLShader and QGLShaderProgram classes, which:

  • Support the GLSL and GLSL/ES shader languages.
  • Handle vertex and fragment shaders (geometry shaders are coming in future versions of Qt).
  • Support writing portable shaders that work on both GLSL and GLSL/ES.

That last point is probably the most interesting for Qt.  GLSL has a lot of built-in variables like gl_Vertex, gl_Normal, gl_ModelViewProjectionMatrix, etc that don't exist in GLSL/ES.  In turn, GLSL/ES has additional type qualifiers like highp, mediump, and lowp that are used to specify the desired precision.  These issues can make it a pain to port existing shader code from desktop to embedded.  We didn't want to have to write two sets of shaders for the OpenGL2 paint engine, so a solution needed to be found.

The solution we chose was to use GLSL/ES as the primary language for writing shaders in Qt, and provide #define's for the extra keywords to make the code compile on desktop GLSL systems.  It is still possible to use the full GLSL language if you want to, but portability will suffer.

The following example demonstrates how to compile and link a simple shader program that can be used to draw triangles with a flat color:

program.addShaderFromSourceCode(QGLShader::Vertex,
"attribute highp vec4 vertex;"
"attribute mediump mat4 matrix;"
"void main(void)"
"{"
" gl_Position = matrix * vertex;"
"}");
program.addShaderFromSourceCode(QGLShader::Fragment,
"uniform mediump vec4 color;"
"void main(void)"
"{"
" gl_FragColor = color;"
"}");
program.link();
program.bind();

int vertexLocation = program.attributeLocation("vertex");
int matrixLocation = program.attributeLocation("matrix");
int colorLocation = program.uniformLocation("color");

The highp and mediump keywords are added to keep GLSL/ES happy - on desktop they #define to an empty string. Also, we have deliberately used user variables for the vertex position, matrix, and color rather than relying upon the desktop-specific gl_Vertex, gl_ModelViewProjectionMatrix, and gl_Color variables. We can then draw a green triangle as follows:

QVector3D triangleVertices[] = {
QVector3D(60.0f, 10.0f, 0.0f),
QVector3D(110.0f, 110.0f, 0.0f),
QVector3D(10.0f, 110.0f, 0.0f)
};

QMatrix4x4 pmvMatrix;
pmvMatrix.ortho(rect());

program.enableAttributeArray(vertexLocation);
program.setAttributeArray(vertexLocation, triangleVertices);
program.setUniformValue(matrixLocation, pmvMatrix);
program.setUniformValue(colorLocation, QColor(0, 255, 0, 255));

glDrawArrays(GL_TRIANGLES, 0, 3);

program.disableAttributeArray(vertexLocation);

Note the use of QMatrix4x4 above to create an orthographic projection matrix to pass to the vertex shader, and the use of QVector3D to build the vertex array.  And that's basically it!  Shaders 101.

What's Next?

Lots and lots of stuff.  Wrapper classes for vertex buffers and textures will probably go into Qt in the near future.  Geometry handling for building object models.  Special-purpose 3D viewing widgets. Integration with Declarative UI for quickly building 3D applications.  And the portability API.  More to come on these in the next post ...


Blog Topics:

Comments