mattday wrote:Hope someone will be able to clarify something regarding the official driver. I'm aiming to perform some processing on a 1920x1080 stream as fast as possible (it really needs to be at least 10fps).
It seems the official driver can deliver frames at close to 30fps only when the pixel format is set to H264 or MJPG. Anything else such as YU12 or RGB3 and this drops to 5fps. Can anyone give an authoritative explanation as to why the uncompressed frame rates are so low? I can guess bandwidth limitations, but would love to know what is really going on and how I might get round it (trying to decode compressed frames on the CPU doesn't make much sense).
The imaging pipe always runs in YUV mode, and converting to any RGB format is also done as an extra step. There is almost zero performance difference between the YUV 4:2:0 formats (I420 aka YU12, YV12, NV12, and NV21). YUV 4:2:2 will be a little slower as there is more data to handle, and the interleaving of the chroma for the YUYV formats will have a very small additional overhead compared to the planar format.
Secondly, for performance reasons, the GPU always works with widths that are multiples of 32 and heights a multiple of 16. 1080 isn't a multiple of 16, so there is a postprocessing step to remove the extra padding and that takes memory bandwidth and time.
BGR3 also takes an extra step over and above RGB3 (My brain couldn't munging the coefficients at the time, so we convert I420 to RGB3, and then swap all the red and blue values. At some point I might revisit that, but very low priority).
I'll check later (camera module currently refusing to detect), but we have had on other threads reporting 2592x1944 I420 streaming at 15fps successfully.
for example, with the reported performance of
BGR3 - 9 fps
RGB3 - 11-12 fps
l420 - 15 fps
we also have 15fps 5MPix I420.
The key thing I suspect you are missing is setting the max_video_[width|height] when you load the driver ("sudo modprobe bcm2835-v4l2 max_video_width=2592 max_video_height=1944"). Without that it will be doing multiple stills captures, which include extra denoising and postprocessing to optimise image quality, and the sensor will be stopping and starting as the buffering isn't sufficient to keep the higher resolutions going continuously.
Got my camera working again.
Code: Select all
sudo modprobe bcm2835-v4l2 max_video_width=2592 max_video_height=1944
v4l2-ctl -p 15
v4l2-ctl --stream-mmap=3 --stream-count=1000 --stream-to=/dev/null
Works perfectly at 15fps for all four formats listed above. The framerate does drop if going up to 2592x1944, but you said you didn't need that.
I haven't checked what the field of view is. It may well be selecting the cropped field of view mode. Requesting 1920x1440 (4:3) should ensure you get the full field of view (full 5MPix resized), and a quick test implies that it still manages to deliver 15fps in all the above pixel formats.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.