Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (01.05.2015)
Many complained about an horizontal tearing issue when playing videos on the Pi using the PiOmxTextures driver. The issue was caused by a missing switch of buffers backing the textures on the output of the renderer component. I took a little time to quickly improve the rendering and the new result is interesting. I also fixed an OpenMAX setup that caused my Pi to crash when stressed (have a look at the commit logs). These were the videos showing the old implementation:
Note that video was only 720p when shaders were applied. I couldn’t do it on 1080p.
Consider that in these videos animations involving both 1080p graphics and 1080p source video were very difficult. In fact you can see that video is sometimes only 720p.
After recent changes I tried again the same sample apps and created a small demo of what you can achieve with Pi. Now source video, images and UI are all 1080p resolution.
Source code can be found in the usual github repo. At the moment the latest changes are all committed to the branch no_tearing. Keep in mind that those changes may break the rest of the implementation, so some work is needed if you really want to use it in your project. The modifications were only made to show what performance Pi can achieve. You’re welcome to send patches to fix the branch as well if you’re interested.
22 thoughts on “Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (01.05.2015)”
Leave a Reply
Related Post
Raspberry OS Bullseye and Cross Toolchain for Linux and Mac OS 10.2.1
Raspberry OS based on Debian bullseye was recently published. The compiler provided with this new [...]
I was able to loop the playlist as in your piomxtextures_pocplayer/qml/main_loop.qml. At the moment, it's enough for me.
Luca thanks for the answer!
Minimal sample: https://pastebin.com/ksifMv6v
Log: https://pastebin.com/s0BG38wU
Can you provide a minimal sample showing the problem?
Hi Nicolás and Luca! I know that this is an old topic, but I can not loop the playlist. After the first video playlist goes to the next video without playing the video and so on.
How can I fix this?
Thank you in advance.
"but then you get a lag of about 5 seconds between loops (I think it is due to the data interchange between CPU and GPU)"
There was a wrong sequence in the setup of the omx graph when opening a video. The time to start is now a little lower. I'll push this change shortly.
This comment has been removed by the author.
Hi Atron! That's kind of the solution I have right now, but maybe it's not a good control as the one I can have trough mediaPlayer… maybe I'm doing it wrong…
Can you share your project or some part of it?
Right now I'm trying to solve the memory leak problem, but I'm slow at that because I have little experience… I'm Nightmare204 at https://github.com/carlonluca/pi/issues/13
Hi Nicolás! If you are in a hurry, this summer I made a similar project. You could try to control Omxplayer from qt through QProcess and using "transparent" as background color in your app . It worked well to me.
It would be awesome if people fork the project and upload their improvements.
Yes, probably the same here: https://github.com/carlonluca/pi/issues/13. You can maybe ask if someone fixed it.
Those macros are very useful!
I'm trying to make a video banner like application. That's why I need the loop property.
However I manage to overcome this using two mediaplayer components that switch one into the other, with fading effect using the opacity property. The switch is activated 3 seconds before the ending of the video being played in order to overcome the lag I told you before.
Now I'm testing the performance to assure a 24/7 working duty of the banner…. and I found a bug… It seems that the GPU doesn't dispose the textures correctly, and as I change the video being played it starts to fill the memory. It doesn't matter if the video is long (20min) or short (20 sec) it can only achieve an specific amount of loops.
If I use only one mediaplayer component it's about 93 video changes, but as I'm using two the number decreases to about 52 video changes (128Mbts memory split to GPU).
I'm getting into your code to find a way to assure the GPU frees the memory resources… any help would be appreciated…
These are some of the reports I got using the command sudo vcdbg reloc
– Just after reboot: http://pastebin.com/BEW0gcfC
– After 1 play of Big Buck Bunny : http://pastebin.com/jrTsQuAJ
– When GPU memory gets filled up: http://pastebin.com/TQgfr4Um
It is possible that the loop feature does not work. I never bothered to implement that. You can probably fix that and patch easily.
All the logs are made up using https://github.com/carlonluca/LightLogger. Have a look in the header and you'll see that there are macros (BUILD_LOG_LEVEL_*) to switch on and off logs efficiently.
Thanks Luca, I didn't have a monitor with audio output from the HDMI, but I tried in another one and I got audio… this means that audio through HDMI is forced by the Openmaxil backend.
I have other 2 observations:
– The MediaPlayer component doesn't follows the loops property if I use the Openmaxil Backend. In order to get a loop I can use the onPlaybackStateChanged, but then you get a lag of about 5 seconds between loops (I think it is due to the data interchange between CPU and GPU)
– Is there any way to avoid the console logs that the Openmaxil Backends deliver?
I'm sorry. Can'the help you with that. I never used that audio output.
I'm sorry. Can'the help you with that. I never used that audio output.
I tried again with the video suggested and with the POCPlayer… there's no audio output via the 3.5mm jack. (I even forced the audio output via raspi-config)…
I used your binaries from 12/2014, and also compiled Qt 5.3.2 from Git and added the OpenMaxil Backend as you described in your posts…
I'm working on a Raspberry Pi 2, I don't know if there's something related to the kernel version that could make the difference to get the audio.
If I place again the libgstmediaplayer.so in order to make Qt use the gstreamer plugin I got normal audio, but obviously not Hardware Accelerated Video…
Any Ideas?
Did you try the standard big buck bunny mov format from the official Web site with POCPlayer (sources in the same repo)?
No video… I know its really weird…
To Discard other possibilities I tried using OmxPlayer, aplay. etc…. I get audio via the 3.5mm jack with all of them, just can't get audio with QT…
No video or just a specific video? There is no particular action you should take to play audio.
Hi Again Luca
It's really awesome what you have achieved with the raspberry pi.
Thanks again for helping me to make work the PiOmxTextures plugin. But I have an issue again, I don't get audio when reproducing a video with Qt using the PiOmxTextures Plugin…
Is there some extra configuration needed?
This is awesome.
Unfortunately this is just a demo of the performance/result. The changes broke many other "features" so it makes no sense to provide bins. It can't be used without working on it.
Great work once again! Will you be providing binary files for this fix? Your previously provided binaries work beautifully but I do have the tearing issue.
Thanks.