Some time ago I published a couple of blog posts talking about Qt WebGL Streaming plugin. The time has come, and the plugin is finally merged into the Qt repository. In the meantime, I worked on stabilization, performance and reducing the calls sent over the network. It also changed a bit in the way the connections are handled.
New client approach
In the previous implementations, the client was accepting more than one concurrent connections. After the latest changes, the plugin is going to behave like a standard QPA plugin. Now, only one user per process is allowed. If another user tries to connect to the web server, it will see a fancy loading screen until the previous client disconnects.
The previous approach caused some problems with how the desktop applications and GUI frameworks are designed. Everyone can agree that desktop applications are not intended to work with concurrent physical users, even if the window instances were different for all users.
No more boilerplate code
Previously the application had to be modified to support this platform plugin. This code was needed to make the application work with the plugin:
class EventFilter : public QObject
{
public:
virtual bool eventFilter(QObject *watched, QEvent *event) override
{
Q_UNUSED(watched);
if (event->type() == QEvent::User + 100) {
createWindow(true);
return true;
} else if (event->type() == QEvent::User + 101) {
window->close();
window->deleteLater();
return true;
}
return false;
}
};
And install the event filter into the QGuiApplication.
No more modifications in applications are needed anymore.
How to try
So, if you want to give it a try before Qt 5.10 is released (~November 2017) do the following:
Prerequisites
Since WebGL was modelled using OpenGLES2 as reference, first thing you will need is to have an OpenGLES2 version of Qt built. To do that, you need to pass the parameter -opengl es2 to the configure before building.
Example:
./configure -opensource -confirm-license -opengl es2
Depending on your system it is possible you will need some aditional headers and libraries to be able to use es2.
Testing the plugin
After building everything, you can try to run a Qt Quick example.
To try the photoviewer example we need to build it and run with the -platform webgl parameters:
./photoviewer -platform webgl
If you want to try the Qt Quick Controls 2 Text Editor:
./texteditor -platform webgl
Supported options
Currently, the plugin only supports an option to configure the port used by the embedded HTTP Server. If you want to listen in the default HTTP port you should write -platform webgl:port=80.
The future
The plugin will be part of Qt 5.10 release as a technology preview (TP), as it needs to be improved. Currently, the plugin contains an HTTP Server and a Web Sockets server to handle the browser connections. I’m planning to remove the servers from the plugin and start using a Lightweight QtHttpServer we are working on right now. Once it’s ready, you will be able to create an application server to launch different process inheriting the web socket to communicate with the browsers. This will allow for supporting more than one concurrent user instead of sharing applications among users.
Note: The plugin only works using the Threaded Render Loop. If you use Windows or a platform using a different render loop ensure you set QSG_RENDER_LOOP environment variable to threaded




This is pretty cool, but what’s the use case for it?
Interact with remote applications. For example from your phone.
Very nice! It would also be fantastic to get the webassembly patches merged : http://qtandeverything.blogspot.fr/2017/06/qt-for-web-assembly.html
Well, I’m guessing the whole point of this is that you can have Qt running on the server, instead of compiling it into JavaScript for the client – with just the rendering happening in the browser.
Congrats !! Great work, Have you managed to use the QWebChannel (for full web functionalities support)?
It’s in my backlog 🙂 First, we need to redesign the server part. It will give us more flexibility.
you guys do an awesome job!
Is it possible to share a docker image with this demo as a low level entry for testing out this demo?
I don’t know how to do that 🙁 I will look into it. If I have something, I will definitely let you know!
“Everyone can agree that desktop applications are not intended to work with concurrent physical users, even if the window instances were different for all users.”
I disagree. Your basic MVC design pattern should alleviate this. I commented on the expresso blog post (proceeding this blog entry) and everything being written today should be using a microservices architecture.
Your Qt app, I think as most of your embedded customers want to do – is to use Qt for the local GUI, and Qt for a remote web GUI. The architecture you are creating in this post is one that will not adequately synchronize the two. Your local UI will always be connected. That may be a nativeGL or WebGL UI. The only logical conclusion is to create a REST microservice and native and WebGL clients that are synced.
So your statement is misaligned. It’s not desktop apps we want to create with this, it’s embedded UIs and Web UIs with the same stack.
Thank you for your comment.
No, there is no possibility to give different UIs to different users in the same process, there are some platform limitations… Maybe Qt 6 can bring some changes to that. The solution I’m going to provide soon is spawning different processes connected via web socket to different browsers. This solution will be more scalable (I tried it).
Are widget based applications also support (given that it is a platform plugin I’d assume so). How is the performance?
Widget applications are a problem here because they are written relying on rasterization. You can write a custom application using GLES2 calls directly, but not widgets, unfortunately.
If we implement support for widgets right now, the performance would be really poor.
I wrote WebGL backend for Qt a year ago .. I hit bottlenecks. Too much GL commands to flow over the web. Qt isn’t optimized painstakingly for this… or maybe now it is… cool to demo on localhost though.
WebGL streaming is not a perfect solution, it’s not meant to be used on the Internet to have interactive applications. I spent time reducing the query calls to the remote WebGL context. If a frame needs to be redrawn only draw calls are sent over the network, the query calls are cached on the server side, so the response time is immediate.
I already used the QML Presentation System in a presentation over the Internet recently and the times were more than acceptable.
Thanks for the reply. I got similar performance to typical VNC setup when not using too many animations in QML. I used Huffman coding to squeeze data about 2x (many zeros in calls). I don’t remember exactly, but I tried to buffer all verts and pixels on client side.. using VAO’s, PBO’s …
Anyway. Maybe it’s useful to try this kind of approach. Although I am looking for someone to port Qt to Emscripten for me 🙂
I have a pending patch with permessage-deflate support in our Web Socket implementation. When it’s done, it will save a lot of bandwidth for all calls, not just for submitting buffer data. Also, the new scenegraph is helping a lot with performance. If you consider contributing, it would be much appreciated 🙂
Unfortunately, I don’t find the most recent video I made … this was made soon after I got something rendering in a browser – https://drive.google.com/open?id=0B7nGK-1U4QjnM0xDR3ZNa0FfQzg
Thanks for sharing!
What is the idea behind QHttpServer? Is this developed in the public?
For the IoT world, we would need some sort of HTTP server. We are starting to research a bit on this area.
I’m not sure about the license yet. So stay tuned 🙂
Is the repository available to anyone? I assumed so since you have a ‘How To Try’ section, but I’m having trouble downloading it. These are my steps:
git clone –branch dev https://github.com/qt/qt5.git
cd qt5
./init-repository –module-subset=default
Everything seems to go OK with init-repository until it gets to qtwebglplugin:
Cloning into ‘qtwebglplugin’…
remote: Repository not found.
fatal: repository ‘https://github.com/qt/qtwebglplugin.git/’ not found
git clone –no-checkout https://github.com/qt/qtwebglplugin.git qtwebglplugin exited with status 32768 at ./init-repository line 198.
github is not the proper place to clone it.
You should use:
git clone https://codereview.qt-project.org/qt/qtwebglplugin
This would be really cool!
https://github.com/qt/qtwebglplugin/ works fine now.
Can I use WebGL in an application that uses Qt with Desktop OpenGL? I read what you write in the prerequisites that ES2 is needed instead of Desktop.
If you can build Qt with ES2 support. ES2 works on Linux and Windows (via ANGLE).
Some parts of the application need desktop OpenGL, so switching to ES2 is not an option.
Or are you suggesting to compile Qt with both desktop and ES2 support? Is that even possible?
Then the answer is no. You can use custom OpenGL command. But just ES2 commands.
Hej Jesús,
have you actually tested running an QtQuick application with webgl plugin on Windows/ANGLE?
If I do not set SG_RENDER_LOOP to ‘basic’ explicitly, the app crashes right away inside the scenegraph code.
When setting SG_RENDER_LOOP to ‘basic’, the application starts up, but crashes sometimes later (haven’t investigated further yet). So have you ever tried it on Windows/ANGLE and this is just a bug in my local setup, or could there be a general problem with this setup?
Hi Tobias,
I tried some time ago on Windows. Currently, I haven’t a Windows machine to test the plugin. Can you try using ‘threaded’ render loop?
I will try to debug it on Windows ASAP.
Many years ago I’ve wrote integrated web server into my business logic to generate GUI/CUI/CLI library. This library could works both as client and server, so this a way to allow running many applications by one client. We need also proxy, which run our first application (app written to use my library in client mode). This proxy is not real proxy, because is both server for tao-network-client and our remote application, but the way it’s working is the same – only difference is way it starts. You can read description at:
https://nintyfan.wordpress.com/2015/04/30/server-buildin-into-libgreattao-and-tao-network-client/
Source code for libgreattao and tao-network-client are on sourceforge.net.
Thank you!
Are there any plans to have both OpenGL and OpenGL ES supported at the same time, or rather have it as a runtime rather than compile time switch?
It makes no sense to support Desktop OpenGL. This plugin is designed to work with ES2.
Could this be used in conjunction with the Qt Wayland Compositor framework to stream a Wayland desktop to a WebGL client?
I have not enough info to answer this question.
I am try it on Windows 7 64bit with MSVC2017 and it is crash because QWebGLIntegration::createPlatformBackingStore(QWindow*) method always return nullptr.
Are you trying to run a widgets application?
Yes.
Widgets are not supported because of performance issues. We only support Qt Quick and QOpenGLWindows calling raw GLES2 commands.
Grief. I just wanted to try this with QtIFW installers 🙁
I’m sorry. The way widgets work does not fit here 🙁
I tried it on Windows 7 64bit with MSVC2015 and it crashes in void QOpenGLFunctions::initializeOpenGLFunctions()
I compared the stack with -platform webgl and without and I noticed a call of syncSceneGraph() but it looks like there is no context yet.
with -platform webgl
Qt5Guid.dll!qt_gl_functions(QOpenGLContext * context) <- crash because returned context is nullptr
Qt5Guid.dll!QOpenGLFunctions::initializeOpenGLFunctions()
Qt5Quickd.dll!QSGBatchRenderer::Renderer::Renderer(QSGDefaultRenderContext * ctx)
Qt5Quickd.dll!QSGDefaultRenderContext::createRenderer()
Qt5Quickd.dll!QQuickWindowPrivate::syncSceneGraph() <- sync without currentContext?
Qt5Quickd.dll!QSGWindowsRenderLoop::renderWindow(QQuickWindow * window)
Qt5Quickd.dll!QSGWindowsRenderLoop::render()
Qt5Quickd.dll!QSGWindowsRenderLoop::exposureChanged(QQuickWindow * window)
…
Qt5Guid.dll!QPlatformWindow::setVisible(bool visible)
without:
Qt5Guid.dll!qt_gl_functions(QOpenGLContext * context)
Qt5Guid.dll!QOpenGLFunctions::QOpenGLFunctions(QOpenGLContext * context)
Qt5Guid.dll!QOpenGLExtraFunctions::QOpenGLExtraFunctions(QOpenGLContext * context)
Qt5Guid.dll!QOpenGLExtensions::QOpenGLExtensions(QOpenGLContext * context)
Qt5Guid.dll!QOpenGLContext::functions()
Qt5Guid.dll!QOpenGLContext::makeCurrent(QSurface * surface)
Qt5Quickd.dll!QSGWindowsRenderLoop::show(QQuickWindow * window)
Qt5Quickd.dll!QQuickWindow::showEvent(QShowEvent * __formal)
…
Qt5Guid.dll!QWindow::setVisible(bool visible)
I’m currently investigating it the crash.
Can you try to set the “QSG_RENDER_LOOP” environment variable to “threaded” before running the application?
Now it works great. Thanks
HI Jesús,
Just started with Qt. I made a little calculation tool with PyQt5. Is it also possible to run this in a webbrowser using WebGL Streaming?
We don’t support PyQt5 🙁
Ah too bad! I think the best way to solve this would be to build the tool in QT Quick and use Qt WebGl do you agree? Will take some time but should be worth it.
Hi Jesús,
the Michael’s problem (context) solved for me with QSG_RENDER_LOOP=threaded, but I use MSVC2017.
Thank you Konstantin 🙂
“I’m going to provide soon is spawning different processes connected via web socket to different browsers” Why not re-implementing custom qabstracteventdispatcher to support epoll backend, so that already hand made code with qtcpsocket class could gain advantage this epoll and possibly iocp on windows?
What benefits do you think this could bring?