Posted on by

The BLUEsat Off World Robotics Software team is rebuilding our user interface, in an effort to address maintenance and learning curve problems we have with our existing glut/opengl based gui. After trying out a few different options, we’ve settled on a combination of QT and QML. We liked this option as it allowed us easy maintenance, with a reasonable amount of power and flexibility. We decided to share with you a simple tutorial we made for working with QT and ROS.

In part one of this article we go through the process of setting up a ROS package with QT5 dependencies and building a basic QML application. In the next instalment we will then look at streaming a ROS sensor_msgs/Image video into a custom QML component. The article assumes that you have ROS kinetic setup, and some understanding of how ROS works. It does not assume any prior knowledge of QT.

Full sources for this tutorial can be found on BLUEsat’s github.

Setting up the Environment

First things first, we need to setup Qt, and because one of our criteria for GUI solutions is ease of use, we also need to setup qt-creator so we can take advantage of its visual editor features.

Fortunately there is a ROS plugin for qt-creator (which you can find more about here). To setup we do the following (instructions for Ubuntu 16.04, for other platforms see the source here):


sudo add-apt-repository ppa:beineri/opt-qt571-xenial
sudo add-apt-repository ppa:levi-armstrong/ppa
sudo apt-get update && sudo apt-get install qt57creator-plugin-ros

We also need to install the ROS QT packages, these allow us to easily setup some of the catkin dependencies we will need later (note: unfortunately these packages are currently designed for QT4, so we can’t take full advantage of them)


sudo apt-get install ros-kinetic-qt-build

Setting up our ROS workspace

  1. We will use qt-creator to create our workspace, so start by opening qt-creator.
  2. On the welcome screen select “New Project”. Then chose “Import Project>Import ROS Workspace”.
    The QT-Creator new project dialogue display the correct selection for creating a new ros project.
  3. Name the project “qt-gui” and set the workspace path to a new folder of the same name. An error dialogue will appear, because we are not using an existing workspace, but that is fine.
  4. Then click “Generate Project File”The QT Import Existing ROS Project Window
  5. Click “Next”, choose your version control settings then click “Finish”
  6. For this project we need a ROS package that contains our gui node. In the project window, right click on the “src” folder, and select “add new”.
  7. Select “ROS>Package” and then fill in the details so they match the screenshot below. We’ll call it “gui” and  the Catkin dependencies are “qt_build roscpp sensor_msgs image_transport” QT Creator Create Ros Package Window
  8. Click “next” and then “finish”
  9. Open up the CMakeLists.txt file for the gui package, and replace it with the following file.
    
    ##############################################################################
    # CMake
    ##############################################################################
    
    cmake_minimum_required(VERSION 2.8.0)
    project(gui)
    
    ##############################################################################
    # Catkin
    ##############################################################################
    
    # qt_build provides the qt cmake glue, roscpp the comms for a default talker
    find_package(catkin REQUIRED COMPONENTS qt_build roscpp sensor_msgs image_transport)
    set(QML_IMPORT_PATH "${QML_IMPORT_PATH};${CATKIN_GLOBAL_LIB_DESTINATION}" )
    set(QML_IMPORT_PATH2 "${QML_IMPORT_PATH};${CATKIN_GLOBAL_LIB_DESTINATION}" )
    include_directories(${catkin_INCLUDE_DIRS})
    # Use this to define what the package will export (e.g. libs, headers).
    # Since the default here is to produce only a binary, we don't worry about
    # exporting anything. 
    catkin_package()
    
    ##############################################################################
    # Qt Environment
    ##############################################################################
    
    # this comes from qt_build's qt-ros.cmake which is automatically 
    # included via the dependency ca ll in package.xml
    #rosbuild_prepare_qt4(QtCore QtGui QtQml QtQuick) # Add the appropriate components to the component list here
    find_package(Qt5 COMPONENTS Core Gui Qml Quick REQUIRED)
    
    ##############################################################################
    # Sections
    ##############################################################################
    
    file(GLOB QT_RESOURCES RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} resources/*.qrc)
    file(GLOB_RECURSE QT_MOC RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} FOLLOW_SYMLINKS include/gui/*.hpp)
    
    QT5_ADD_RESOURCES(QT_RESOURCES_CPP ${QT_RESOURCES})
    QT5_WRAP_CPP(QT_MOC_HPP ${QT_MOC})
    
    ##############################################################################
    # Sources
    ##############################################################################
    
    file(GLOB_RECURSE QT_SOURCES RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} FOLLOW_SYMLINKS src/*.cpp)
    
    ##############################################################################
    # Binaries
    ##############################################################################
    
    add_executable(gui ${QT_SOURCES} ${QT_RESOURCES_CPP} ${QT_FORMS_HPP} ${QT_MOC_HPP})
    qt5_use_modules(gui Quick Core)
    target_link_libraries(gui ${QT_LIBRARIES} ${catkin_LIBRARIES})
    target_include_directories(gui PUBLIC include)
    install(TARGETS gui RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION})
    

    Note: This code is based on the auto generated CMakeList.txt file provided by the qt-create ROS package.
    Lets have a look at what this file is doing:

    cmake_minimum_required(VERSION 2.8.3)
    project(gui)
    
    ...
    
    find_package(catkin REQUIRED COMPONENTS qt_build roscpp sensor_msgs image_transport)
    include_directories(${catkin_INCLUDE_DIRS})
    

    This part is simply setting up the ROS package, as you would expect in a normal CMakeLists.txt file.

    find_package(Qt5 COMPONENTS Core Qml Quick REQUIRED)
    

    In this section we setup catkin to include Qt5, and tell it we need the Core, QML, and Quick components. This differs from a normal qt-build CMakeList.txt, because we need QT5 rather than QT4.

    file(GLOB QT_RESOURCES RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} resources/*.qrc)
    file(GLOB_RECURSE QT_MOC RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} FOLLOW_SYMLINKS include/ros_video_components/*.hpp)
    
    QT5_ADD_RESOURCES(QT_RESOURCES_CPP ${QT_RESOURCES})
    QT5_WRAP_CPP(QT_MOC_HPP ${QT_MOC}
    

    This section tells cmake where to find the QT resource files, and where to find the QT header files so we can compile them using the QT precompiler.

    file(GLOB_RECURSE QT_SOURCES RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} FOLLOW_SYMLINKS src/*.cpp)
    

    And this tells cmake where to find all the QT (and ROS) source files for the project

    add_executable(gui ${QT_SOURCES} ${QT_RESOURCES_CPP} ${QT_FORMS_HPP} ${QT_MOC_HPP})
    qt5_use_modules(gui Quick Core)
    target_link_libraries(gui ${QT_LIBRARIES} ${catkin_LIBRARIES})
    target_include_directories(gui PUBLIC include)
    install(TARGETS gui RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION})
    

    Finally we setup a ROS node (executable) target called “gui” that links the C++ source files with the Qt MOC/Header files, and the QT resources.

  10. Next we need to fix our package.xml file, the qt-creator plugin has a bug where it puts all the ROS dependencies in one build_depends and run_depends tag, rather than putting them separately. You need to separate them like so:
     
    <buildtool_depend>catkin</buildtool_depend>
      <buildtool_depend>catkin</buildtool_depend>
      <build_depend>qt_build</build_depend>
      <build_depend>roscpp</build_depend>
      <build_depend>image_transport</build_depend>
      <build_depend>sensor_msgs</build_depend>
      <build_depend>libqt4-dev</build_depend>
      <run_depend>qt_build</run_depend>
      <run_depend>image_transport</run_depend>
      <run_depend>sensor_msgs</run_depend>
      <run_depend>roscpp</run_depend>
      <run_depend>libqt4-dev</run_depend> 
    
  11. Create src/, include/<package name> and resources/ folders in the package. (Note: it doesn’t seem possible to do this in qt-creator you have to do it from the terminal or file browser).

Our ROS workspace is now setup and ready to go. We can start on our actual code.

Creating a Basic QML Application using the Catkin Build System

We’ll start by creating a basic ROS node, that displays a simple qml file.

  1. Right click on the src folder in the gui package, and select “Add New”
  2. Select “ROS>Basic Node” and then click choose
  3. Call it “guiMain” and click Finish. You should end up with a new file open in the editor that looks like this:Qt creator window displaying a main function with a basic ros "Hello World" in it
  4. We’ll come back to this file later, but first we need to add our QML Application. In order to call ros::spinOnce, without having to implement threading we need to subclass the QQmlApplicationEngine class so we can a Qt ‘slot’ to trigger it in the applications main loop  (more on slots later). So to start we need to create a new class: right click on the src directory again and select “Add New.”
  5. Select “C++>C++ Class “, then click “Choose”
  6. Set the “Class Name” to “MainApplication”, and the “Base Class” to “QQmlApplicationEngine.”
  7. Rename the header file so it has the path “../include/gui/MainApplication.hpp” This allows the CMakeLists.txt file we setup earlier to find it, a run the MOC compiler on it.
  8. Rename the source file so that it is called “MainApplication.cpp”. Your dialog should now look like this:Qt Creator "C++ Class" dialogue showing the settings described above.
  9. Click “Next”, then “Finish”.
  10. Change you MainApplication.hpp file to match this one:
    #ifndef MAINAPPLICATION_H
    #define MAINAPPLICATION_H
    
    #include <ros/ros.h>
    #include <QQmlApplicationEngine>
    
    class MainApplication : public QQmlApplicationEngine {
        Q_OBJECT
        public:
            MainApplication();
            //this method is used to setup all the ROS functionality we need, before the application starts running
            void run();
            
        //this defines a slot that will be called when the application is idle.
        public slots:
            void mainLoop();
    
       private:
            ros::NodeHandle nh;
    };
    
    #endif // MAINAPPLICATION_H
    

    Two important parts here, we add the line “Q_OBJECT” below the class declaration. This tells the QT MOC compiler to do QT magic here in order to make this into a valid QT Object.
    Secondly, we add the following lines:

    public slots:
        void mainLoop();
    

    What does this mean? Well QT uses a system of “slots” and “signals” rather than the more conventional “listener” system used by many other gui frameworks. In layman’s terms a slot acts similarly to a callback, when an event its been “connected” to occurs the function gets called.

  11. Now we want to update the MainApplication.cpp file. Edit it so it looks like the following:
    #include "gui/MainApplication.hpp"
    #include <QTimer>
    
    MainApplication::MainApplication() {
        
    }
    
    void MainApplication::run() {
        
        //this loads the qml file we are about to create
        this->load(QUrl(QStringLiteral("qrc:/window1.qml"))); 
        
        //Setup a timer to get the application's idle loop
        QTimer *timer = new QTimer(this);
        connect(timer, SIGNAL(timeout()), this, SLOT(mainLoop()));
        timer->start(0);
    }
    

    The main things here are we load a qml file, at the resource path “qrc:/window1.qml”. In a moment we will create this file. The other important thing is the set of timer code. Basically how this works is we create a timer object, we create a connection between the timer object’s “timeout” event (signal), and our “mainLoop” slot which we will create in a moment. We then set the timeout to 0, causing this event to trigger whenever the application is idle.

  12. Finally we want to add the the mainLoop function to the end of our MainApplication code, it simply calls ros::SpinOnce to get the latest ROS messages whenever the application is idle.
    void MainApplication::mainLoop() {
        ros::spinOnce();
    }
    
  13. In our guiMain.cpp we need to add the following lines at the end of our main function:
        QGuiApplication app(argc, argv);
        MainApplication engine;
        engine.run();
        
        return app.exec();
    

    This initialises our QML application, calls our run function, then enters QT’s main loop.

  14. You will also need to add these two #includes, to the top of the guiMain.cpp file
    #include <QGuiApplication>
    #include <gui/MainApplication.hpp>
    
  15. We now have all the C++ code we need to run our first demo. All that remains is writing the actual QT code. Right click on the “resources” folder, and select “Add New.”
  16. In the New File window, select “Qt” and then “QML File (Qt Quick 2)”, and click “Choose.”
  17.  Call the file “window1” and finish.
  18. We want to create a window rather than an item, so change the qml file so it looks like this:
    import QtQuick 2.0
    import QtQuick.Window 2.2
    
    Window {
        id: window1
        visible: true
    
    }
    
  19. Now we will use the visual editor to add a simple image to the window. With the QML file open, click on the far left menu and select “Design” mode. You should get a view like this:
    QT Creator QML Design View
  20. From the left hand “QML Types” toolbox drag an image widget onto the canvas. You should see rectangle with “Image” written in it on your canvas.
    Qt Creator Design Window with an image added
  21. We need to add an image for it to use. To do this we need a resource file, so switch back to edit mode. Right click on the resources folder and select “Add New.”
  22. Select “Qt>Qt Resource File” and then click “Choose”
  23. Call the resource file “images,” and finish.
  24. This should open the resource file editor, first you need to add a new prefix. Select “add>New Prefix”
    QT Creator Resource File Editor: Select Add>Add Prefix
  25. Change the “prefix” to “/image”.
  26. We now want to add an image. Find an image file on your computer than is appropriate then click “Add Files” and navigate to it. If the file is outside your project, qt-creator will prompt you to save it to your resources folder, which is good. You should now have a view that looks like this:
    QT Creator Resource File Editor with an image added
  27. Switch back to the qml file and design view.
  28. Click the image, on the right hand side will be a drop down marked “source”, select the image you just added from it. (Note if the designer has auto filled this box but the image preview is not appearing you may need to select another image, and then reselect the one you want). I used the BLUEsat logo as my image:
    I used the BLUEsat logo for my example
  29. Now we just need to put the qml somewhere we can find it. As in steps 21 to 26, create a new resource file in the resources folder called qml and add the qml window1.qml to it under the “/” prefix.
  30. At this point you should be able to build your project. You can build using catkin_make as you normally would, or by clicking the build button in the bottom left corner of the ide.
  31. To run your ROS node you can either run it as you would normally from the command line using “rosrun”, or you can run it from the ide. To setup running it from the ide select “Project” from the left hand menu, then under “desktop” select run.
  32. Under the run heading, select “Add Run Step>rosrun step.” You should get a screen like this.new/prefix1QT Creator - Project settings screen
  33. Set the package to “gui”, and the target should auto fill with “gui” as well.
  34. Press run in the bottom left corner. You should get something like this (note: depending on where you places the image you may need to resize the window to see it). Important: as always with ROS, roscore needs to be running for nodes to start correctly. Window Displaying BLUEsat Logo
  35. You have a working gui application now, compiled with catkin and running a ROS Spin Loop at the appropriate loop, but this is a bit useless without using some information from other ROS nodes in the gui. In our next article, we will look at streaming video from ROS into Qt so stay tuned!