hascol
Posts: 5
Joined: Sun Dec 20, 2015 1:28 pm

FFMpeg + hello_video/video.c: movie and monitor resolution

Thu Jan 21, 2016 10:24 am

Playing .h264 low resolution movie on a HD display through HDMI port. I see the movie gets automatically scaled to full screen and I guess OpenMAX is responsible for that. So, I assume GPU resources are being used to zoom while the movie plays.
If my intro is correct, what does it happen if I scale the movie in advance with FFMpeg for example?
a) visual appearance improves/degrades
b) playing the FFMpeg-scaled movie reduces/leave unchanged GPU utilization

dickon
Posts: 356
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: FFMpeg + hello_video/video.c: movie and monitor resoluti

Thu Jan 21, 2016 7:10 pm

Scaling a video like that will (almost certainly) require a full transcode, which isn't pretty: you'd decode the video to the current, small frame size, then upscale it, then encode the full-size picture. This is *incredibly* resource-intensive, even if you get the GPU to do the decoding and encoding and just perform whatever your favourite upscaling method is on the CPU: you'd be transferring all the frames back and forth between the GPU and the CPU.

There's a resize OMX component available, which is probably the thing doing the actual scaling (albeit behind the scenes). You can invoke this yourself in your OMX pipeline; omxtx can do this.

The visual impact this all has is the tradeoff between your (hopefully better) upscaling technique than the resize component's, and the fact that you're going to be reencoding it all again for later playback, with all the generational issues that implies. The GPU usage impact is going to be a tradeoff where you've decided to increase the bitrate of the transcoded video to offset the losses in encoding, which means more data crossing the buses, being processed by the CPU and the GPU, and the larger amount of data being shunted around the GPU as it's dealing with inherently larger frames.

Basically: don't bother.

hascol
Posts: 5
Joined: Sun Dec 20, 2015 1:28 pm

Re: FFMpeg + hello_video/video.c: movie and monitor resoluti

Thu Jan 21, 2016 8:42 pm

dickon, thanks for being so exhaustive!
Actually, you are far beyond my intent. Maybe, the best answer to linuxstb » Wed May 29, 2013 10:15 pm in viewtopic.php?f=70&t=44852, for example.

I didn't mean to run FFMpeg runtime chunk by chunk but convert the whole video file to be passed to hello_video.bin as an argument. If the video resolution exactly matches the screen's, this should bring benefits in terms of GPU usage at run time, I guess. But, I am not sure. And I don't know about visual appearance improvement too. I suppose that a lot depends on how the conversion is carried out in FFMpeg. And perhaps, a different tool is more appropriate than FFMpeg.

dickon
Posts: 356
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: FFMpeg + hello_video/video.c: movie and monitor resoluti

Thu Jan 21, 2016 11:12 pm

Your two options ('->' being smallish bandwidth; '=>' greater):

low-res video -> decoder -> [implicit resize =>] render => screen

high-res video -> decoder => render => screen

In the former case, the GPU is doing more work as it's sending the low-res images to the scaling unit for resizing which then sends bigger pictures to the screen. In the latter case, you're sacrificing picture quality as you've re-encoded the bitstream (which inevitably reduces the quality), but you're punting more data between the decoder and the renderer than in the earlier case, and that may actually be more work for the GPU than the first case; I don't know.

linuxstb in that post seemed to be trying to use the scaler twice: once to take the camera image and rescale it to a viewfinder size, and once to take the input video and resize it to the display size (although unless I've missed something, this isn't entirely clear). Most of the time, GPU usage really isn't an issue; it's usually got much more capability for this sort of thing than the CPU has. Don't forget: most of these functions aren't running on the GPU as you'd expect them to run on a CPU: they're dedicated hardware macroblocks, so whether the thing appears in the pipeline or not doesn't massively impact on the performance of the GPU as a whole.

omxtx will happily run twice simultaneously on the same hardware: it just transcodes at half the speed it does normally, and, I assume, uses twice the GPU RAM to do it. Broadcom's OMX stack seems to be good enough for that.

If in doubt, overclock it. IME the GPU cores are quite happy at x2 their rated limits...

Return to “OpenMAX”