Mikhail Svetkin

QtHttpServer routing API

Published Friday February 1st, 2019
25 Comments on QtHttpServer routing API
Posted in Automation, C++, Dev Loop, Embedded, Internet of Things, Labs | Tags: , ,

Hi everybody. First of all, I want to say thank you for all of you for the comments
to the previous blog post.
Today, I’m going to talk about routing, how it works, and how we implement it.

Before I start, I want to clarify something. We’ve looked at many projects on GitHub, written in many languages – not only C++.
We found that most of the projects written in C++ are over-complicated, or very low level.
Sometimes, a “hello world” example takes 20-30 lines of code.
Projects written in other languages than C++ are very big, and they have big ecosystems. We aren’t able to reimplement
all of the features that they have already done.
That’s why we wanted to create something that is both simple and easily extensible at the same time.

Note: Big thank you to Flask. It inspired me to create that API.

What is routing and how does it work?

Routing is a process that is responsible for defining a callback for a specific request.
A route is a specific rule (callback) for a specific requested URL.

Let’s take a look at an example:


http://blog.qt.io/dev // blogs for Dev loop
http://blog.qt.io/biz // blogs for Biz Circuit
http://blog.qt.io/blog/2019/ // blogs for 2019 (dev loop + Biz Circuit)

Each address represents a specific request with a specific callback.
Also, you may have noticed that one of them has a parameter (http://blog.qt.io/blog/2019/).
This URL requests whole blogs for the year 2019, so the callback should be able to understand which year is requested.
Such a route is called dynamic, and other two static.

Static routing

I think this is a very simple topic. Let’s see how we can do it with QHttpServer.

QHttpServer server;

server.route("/dev/", [] () {
    return "All dev loop blogs";
});

server.route("/biz/", [] () {
    return "All biz loop blogs";
});

We just bound each path to its own callback. I think a more interesting topic is the dynamic routing.

Dynamic routing

First, I suggest splitting that task to sub-tasks. Then understand what we need to implement.

  • We need to have a mechanism to catch a parameter from a path in a url.
  • Convert the captured parameter to a type which we expect to have.

These are two main points, and the most important ones. I suggest going through each item in a row.

First, you might ask: “Is it so complicated? Can we just use a regular expression?”
I think, to understand it better, we should take the example from the top and try to solve it with a regexp.


/blog/(\\d+)/

Looks perfect. We can catch a parameter. How to convert the parameter to int?
The most common way is to just keep the parameter as a string, and then convert it manually.
However, this approach has several disadvantages:

  • If we have more than one parameter, we will need to do it for all of them.
  • Not everybody want to write regular expressions – especially for strings or floats.

How can we improve it? Let’s take a look at our colleagues from other languages.
Modern frameworks (Django, flask, ruby on rails) all do it in the same way.
For example, it can be like this:


/blog/<int:year>/

Looks pretty clear. int used as an alias to the regular expression, and also now we know the type.
year is used to bind a captured parameter to a callback argument.
This approach allows you to easily add your custom types, which could look like this:


/blog/<HexInt:year>/

Good, now we know enough about routing to start to implement it.

QHttpServer::route

I suggest we do something like this:

route("/blog/<int:year>", [] (const HttpRequest &request) {
    return blogs_by_year(request["year"].toInt());
});

route("/blog/<int:year>", [] (auto year) {
    return blogs_by_year(year.toInt());
});

It’s not bad, but we can still improve on it. If you look closely on the second case, you’ll see
that we don’t really need the name of the parameter year in the path pattern. The reason for this is that we cannot bind a captured parameter to the callback argument in C++. In addition, we can’t get a compile-time check of the path pattern type and the callback argument. So how can we improve this?

We can use static types. This way we get to use the compiler as a “controller”, making sure that the type we get is a type we support.
To solve our second sub-task (converting, remember?), we can use a QVariant which can easily convert parameters.
So what can we do? We can combine them, like this:

QHttpServer server;
server.route("/blog/", [] (int year) {
    return blogs_by_year(year);
});

What do we get?

  • We don’t need to duplicate the type name and an argument’s name in the path pattern and callback.
  • Types will be automatically converted.
  • Additional advantage: compile time check of supported types.

Isn’t this cool?!

Out of the box, we support several types: int, float, double, QString, QByteArray, QUrl, and a couple more…
You can also add support for your own types or even change the regex matching. Let me know in the comments if you want me to explain this in another blog post.

You might be asking: “How do I capture several parameters?”. For example, I want to show all blog posts posted in February 2019:

QHttpServer server;
server.route("/blog/<var>/<var>", [] (int year, int month) {
    return blogs_by_year_and_month(year, month);
});

You may already have spotted the keyword . It works almost exactly like QString::arg, but doesn’t support ordering.
That’s why we don’t use the same syntax as in QString::arg.

Also, we want to provide you with the possibility to create a REST API. For that, we need to split GET/POST/PUT/DELETE requests.
A small example:

server.route("/blog/", QHttpServerRequest::Method::Get, [] (int year) {
    return blogs_by_year(year);
});

or

server.route("/blog/", [] (int year, const QHttpServerRequest &req) {
    if (req.method() == QHttpRequest::Method::Get)
        return blogs_by_year(year);

    return QHttpServerResponder::StatusCode::NotFound;
});

Both of these are good, but I prefer the first one – it is shorter.

By default, QHttpServer::router works only with a path from a url. If you want to create a specific rule that works with a query,
you can inherit from QHttpServerRouterRule and pass it as a template argument to QHttpServer::route.

If you want to set up the HTTP response headers by yourself, then you can use the low-level API QHttpServerResponder.

QHttpServer server;
server.route("/blog/", [] (int year, QHttpServerResponder &&responder) {
    responder.write(blogs_by_year(year), "text/plain");
});

Note: QHttpServerResponder and QHttpServerRequest are special arguments that you can only use as the last argument of a callback.

There you have it: a simple web server that you can easily extend. If you’re interested in this project, don’t forget to clone it 🙂
And if you have suggestions or ideas on how to improve it further, you can consider contributing or provide some feedback in QTBUG-60105.

Thanks for reading!

Do you like this? Share it
Share on LinkedInGoogle+Share on FacebookTweet about this on Twitter

Posted in Automation, C++, Dev Loop, Embedded, Internet of Things, Labs | Tags: , ,

25 comments

Nicolás says:

Do the examples have the year parameter specified in the routing string? I suspect the blog software ate them because they look like HTML tags.

Jesús Fernández Jesús Fernández says:

Fixed!

Thank you 🙂

Shi says:

really great to see Qt has an official api for creating http servers. I have written one myself before https://github.com/shi-yan/swiftly

Jesús Fernández Jesús Fernández says:

Cool stuff

Mikhail Svetkin Mikhail Svetkin says:

Thank you, we open for patches 🙂

Dominik says:

I think what you implemented looks very similar to the API of the served project: https://github.com/meltwater/served

As you can see, served also provided easy and quick access to the fields you introduced. Good work. Were you inspired by served, or did you get your ideas from somewhere else?

PS: I am not involved in served in any way.

Dominik says:

Btw, you should add this to Qt 5.1x, so that you get feedback. With this feedback you can even improve it for Qt6.

Jesús Fernández Jesús Fernández says:

It does not require Qt 6. It’s usable with (at least) 5.12.

Mikhail Svetkin Mikhail Svetkin says:

I was inspired by Flask (http://flask.pocoo.org/) and as I said all modern frameworks does almost the same.
Served keeps parameters as a string and then you need to manually convert them to the type. You don’t need to do it with QHttpServer.

Andrew Dolby says:

This is excellent. Routing design is important.

Alexander Yudaev says:

This is great news! HTTP server in Qt is not enough. Constantly have to reinvent the wheel or take something third-party.
Will the product be released only under the GPL license? I think this is not the best choice for web service. Flask comes under BSD, TreeFrog under BSD, served under MIT, Cutelyst under the LGPL, etc.

Mikhail Svetkin Mikhail Svetkin says:

It will be under GPL and commercial license.

Peter says:

It looks like the Qt company is trying to move the Qt libraries into a commercially unfriendly open source product. There are many members of the Qt community that write commercial software and contribute back to Qt through the careful development of bug reports, fixes and test cases, but use the LGPL licence. Extending Qt as a whole to include GPL and LGPL libraries will fragment the Qt eco-system so that other libraries that operate within the LGPL will have to include separate solutions for the growing GPL sections of Qt, e.g. KDE and the KDE Frameworks. It would have been more useful for Qt to adopt one of the myriad of Qt-based http server solutions, rather than invent a library that is not able to be used by everyone, nor benefit from everyone contributing to fixing and improving the product.

> is trying to move the Qt libraries into a commercially unfriendly open source product

If by that you mean having the choice only between GPL and commercial license, then it’s been like this for a while now, and it certainly didn’t start in secret. Most of the upcoming new components in Qt are likely to be GPL/commercial only.

Leaving aside for a bit such oxymoron as “commercially unfriendly open source product”, the choice is quite clear: you either support Open Source or you don’t. If you do, then GPL should not be a problem; and if you don’t, then there is a commercial license available.

Drone says:

I don’t think it’s as easy as this. The model of contributing back to Qt while using the LGPL license has been a good compromise for many smaller teams and part-time projects. Also, the GPL pretty much forces your own project to also be GPL which might be a showstopper. The new GPL/Commercial model makes it pretty certain that those users won’t use the GPL-only modules, and also certainly won’t contribute anything back to them.

Just look at the GPL module Charts – it’s not particularily good (imo), has stagnated, and nobody is contributing back to it. I’m not even using it in my commercially-licensed projects. For me, this is worrying for a http server, especially with regards to security. Less users, less testing, and less bug fixes contributed back.

How making a project to be GPL can be a showstopper for someone who supports Open Source? Is it because of the requirement to share all the… oh wait 🙂
Also how does GPL/commercial make users not to use the GPL-only modules, not to say about preventing them from contributing back? The GPL option is clearly there and it is an Open Source license.

I admit though that Qt Charts haven’t seen any love for quite a long time. And I can tell that the module is not forgotten, and hopefully you’ll hear some news in that regard in the nearest future (although that concerns only the Qt Company’s effort).
At the same time, I can pick at least several Qt modules available under LGPL which have close-to-zero amount of community contributions, so I don’t think the type of license (GPL or LGPL) plays much of a role here.

Drone says:

In a commercial setting, writing closed-source software, the GPL is a no-go. Still, we use lots of open-source libraries whenever possible, and also do contribute back to them. Take mapbox-gl-native, for example, or sqlite, or the Bullet physics engine.

Open source does not exclusively mean using the GPL license. If you want to limit open source to those who do, fine, but you get less exposure, testing and contributions back.

Qt itself uses lots of 3rd party open source libraries, none of which are purely GPL. So I assume the Qt project has intentionally avoided pure-GPL libraries themselves.
https://doc.qt.io/qt-5/licenses-used-in-qt.html

Tham says:

I would like them to make this module LGPL or BSD license too, but we must know that Qt company need money to live.
If their strategies can help them earn more money, keep them alive, hire more developers to improve the quality of Qt libraries,
why should we complain?

I like free things, but the free things are often the most expensive

Pius says:

So very soon I can easily pick up Qt and say yes! I am developing a website today! good old work

Dima says:

There is a modern system of routing paths, which perfectly maps to URL tree structure. You can spy on the idea at Pyramid framework:

* https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/hellotraversal.html
* https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/traversal.html
* async version https://github.com/zzzsochi/aiohttp_traversal

Also it would be great to make support for asynchronous request handler

Mikhail Svetkin Mikhail Svetkin says:

Thank you, I will take a look

Vladyslav Stelmakhovskyi says:

jquery in Qt/C++. brilliant!

sadeq says:

does the router support callbacks to member functions ?

Mikhail Svetkin Mikhail Svetkin says:

No, It does not, but I am working on a extension, which will allow you to register a class.

Commenting closed.

Get started today with Qt Download now