Hardware Accelerated Video with Qt 6 on the Raspberry Pi
Getting hardware acceleration into Qt eglfs is tricky. Doing so on a Raspberry Pi is, unfortunately, still tricky after many years. Qt claimed to have reimplemented the Qt Multimedia module entirely, and one of their target was getting hardware acceleration where possibile. So, I thought I could start with a quick look.
Qt 5
Since Raspberry Pi was born, I had to solve the problem of hardware accelerated video in Qt. At the beginning, I wrote POT (PiOmxTextures) to solve this problem: https://github.com/carlonluca/pot. It used OpenMAX to stream decoded video into an OpenGL texture, which was then showed through a custom backend of Qt Multimedia in Qt 5. This approach worked fine, but won’t work on Pi 4/Qt 6. On the other hand, there is another component in the same repo, that includes a custom Qt Quick item to render video through omxplayer. This is the most performant approach, but has its limitations.
Qt Multimedia
I quickly tested Qt Multimedia in Qt 6 on the rpi. My build from this article should support gstreamer. All I got was a a warning on the console. I didn’t investigate further. Maybe I’ll spend more time on this in the future.
POTVL
As the classical POT is no more usable on Raspberry Pi 4, I started to have a look at POTVL, which is very simple to port to Qt 6. With a small patch, it is possible to build it. You’ll find updates on the repo.
Demo
The video is a 1080p video. As you can see, the framerate is acceptable up to a certain weight of the graphics. It seems that Qt 6 OpenGL backend still is a bit less performant than Qt 5 in this specific demo, as you can see from this test from a previous article:
so the result may even improve in the future.
The benchmark app can be found here: https://github.com/carlonluca/Fall.
Unfortunately POTVL is still not future proof, but it is the simplest and most efficient element to port to Qt 6. I may try something better in the near future.
Have fun! Bye 😉
18 thoughts on “Hardware Accelerated Video with Qt 6 on the Raspberry Pi”
Leave a Reply
Related Post
Raspbian Stretch and Cross Toolchain for Linux and Mac OS
Raspbian Stretch is out for Raspberry Pi. I needed a cross toolchain with the same [...]
Hi Luca,
in potvl are there any custom settings to apply to tty? Because if I start omxplayer with layer -128 I can listen audio but I can’t see the video, maybe it’s hidden beyond the console?
Regards,
Mattia
-128 may be hidden. What happens if you use a higher level? Also check the geometry. You can also list the dispmanx layers to verify.
If I use -127 the video is visible but then the QtApplication is under the omxplayer layer.
I’m trying to use Fall with my Qt 5.15.2 cross-compiled but if I use potvl as argument I can’t see the video in background.
I will try to list the layers.
Are you using fkms and vc4?
yes!
vcgencmd dispmanx_list:
display:2 format:RGB565 transform:0 layer:-127 1920×1080 src:0,0,1920,1080 dst:0,0,1920,1080 cost:889 lbm:0
So I don’t have layer -128 by default, anyway if I set POT_LAYER to -127 I see omxplayer upon Fall.
I do not clearly remember, the project is pretty old, however you need to see the Qt layer above the omxplayer layer. This should be possible, or at least it was.
p.s:
i forgot to mention that i use raspbian bullseye, python3.9
Hi,
how did you install Qt6 , Qt6Quick and … on raspberry pi ??
and can you help me how to run pyside6 app on raspberry pi?
I copied the files to the board in the location set in the prefix.
No, I’m sorry, I don’t use python.
hi , how did you install all of qt6 and qtquick and … on raspberry pi ??
I copied the files.
Hi Luca,
I builded Qt6 with QtMultimedia based on GStreamer 1.0, but QMediaPlayer not play audio file . I have added GST_DBG and the log prints these msg:
0:02:50.864485075 1406 0xa7f0b850 WARN baseparse gstbaseparse.c:3676:gst_base_parse_loop: error: Internal data stream error.
0:02:50.864714740 1406 0xa7f0b850 WARN baseparse gstbaseparse.c:3676:gst_base_parse_loop: error: streaming stopped, reason not-linked (-1)
QMediaPlayer slot error is triggered with “Invalid media type”.
I obtains whetever file I try to play (also wave) , but if I try to launch a gstreamer pipeline outside Qt it works.
Have you faced the same issue?
Hello,
I’m sorry but I did not investigate Qt Multimedia in any way. All I got was a segfault and I tested nothing more. Qt Multimedia was never useful in any way on the Pi, so I did not invest time on it.
Thank you Luca,
anyway if anyone is interested I opened a more detailed post here:
https://stackoverflow.com/questions/70668976/qt6-multimedia-module-doesnt-play-audio-on-raspberry-pi-4
I am interested to know if qtmultimedia works or not at least for audio. With qt5 I have always used qtmultimedia for audio and bus controlled omxplayer for video, but I would like to start doing some tests with vlc (using kms instead of fkms), since qtmultimedia is not yet usable.
Hi,
Is the source of POTVL/ PiOmxTexturesVideoLayer available? I can only find one mention of it on github, in your fall app
Hello,
yes, it is open source. Source code is here: https://github.com/carlonluca/pot/tree/master/piomxtextures_videolayer. However, I have never presented it publicly nor documented it. It was only used in commercial projects.
You can freely use it.
Note that it needs a specific omxplayer binary that is added to the repo. This binary sends specific dbus signals to sync with the UI.
Regards.
Cool, thanks!