jpsa
Posts: 15
Joined: Wed Feb 04, 2015 8:22 am

Mode 1 video processing & size rounding

Mon Jan 22, 2018 11:54 am

Hello,

I'm trying to perform live processing on a video stream using mode=1 (documented as being 1920x1080 pixels from the middle of the sensor) on a camera V2.1. I'm starting with the code

Code: Select all

#!/usr/bin/env python3

import time
import picamera

class FrameHandler(object):
    def write(self, buf):
        pass

    def flush(self):
        pass

with picamera.PiCamera(sensor_mode=1, resolution='1920x1080', framerate=30) as camera:
    time.sleep(2) # let the camera warm up

    camera.start_recording(FrameHandler(), 'yuv')
    camera.wait_recording(10)
    camera.stop_recording()
which is adapted from that given in http://picamera.readthedocs.io/en/relea ... om-outputs.

When I run this code, I get the warning message
/usr/lib/python3/dist-packages/picamera/encoders.py:544: PiCameraResolutionRounded: frame size rounded up from 1920x1080 to 1920x1088
width, height, fwidth, fheight)))
Which sounds as if it will cause the image to be rescaled slightly before it's passed to my frame handler, which I don't want to happen.

How can I get un-scaled mode 1 data?

Thanks for any hints!
John

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24188
Joined: Sat Jul 30, 2011 7:41 pm

Re: Mode 1 video processing & size rounding

Mon Jan 22, 2018 12:35 pm

It's not scaled. The ISP requires frames that are multiple of 16 in X and 32 in Y (I think that is the right way round) so 1080 is rounded up to 1088. The extra lines are added, IIRC by making the sensor area slightly larger (might be wrong there), not by scaled up the 1080.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

ethanol100
Posts: 587
Joined: Wed Oct 02, 2013 12:28 pm

Re: Mode 1 video processing & size rounding

Mon Jan 22, 2018 12:38 pm

It is the packaging of YUV values. This needs to be a multiple of 32 for height, as 1080 is not a multiple of 32 it selects next higher number 1088, you can just ignore the last 8 pixel, which will not contain valid data.

See picamera documentation:
It is also important to note that when outputting to unencoded formats, the camera rounds the requested resolution. The horizontal resolution is rounded up to the nearest multiple of 32 pixels, while the vertical resolution is rounded up to the nearest multiple of 16 pixels. For example, if the requested resolution is 100x100, the capture will actually contain 128x112 pixels worth of data, but pixels beyond 100x100 will be uninitialized.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7575
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Mode 1 video processing & size rounding

Mon Jan 22, 2018 1:19 pm

jamesh wrote:
Mon Jan 22, 2018 12:35 pm
It's not scaled. The ISP requires frames that are multiple of 16 in X and 32 in Y (I think that is the right way round) so 1080 is rounded up to 1088. The extra lines are added, IIRC by making the sensor area slightly larger (might be wrong there), not by scaled up the 1080.
No, wrong way around.
stride must be a multiple of 32 pixels, and buffer height a multiple of 16. 1080 is not a multiple of 16, therefore is rounded up to 1088.

In the case of mode 1 on the V2.1 sensor, exactly 1920x1080 pixels are read off the sensor.

Why are you so afraid of resizing? Under many circumstances it actually helps out in reducing noise slightly.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jpsa
Posts: 15
Joined: Wed Feb 04, 2015 8:22 am

Re: Mode 1 video processing & size rounding

Mon Jan 22, 2018 2:34 pm

Thank you for the replies.

I don't want to resize, as I'm keen to retain as much resolution as possible, and any resizing seems likely to introduce at least a little blurring (either that or aliasing artefacts, which would also be bad).

I may also be performance limited, so if there were any CPU cost to resizing, I'd want to avoid that too. But I think from what I've read that's done by the video processor, so probably isn't actually a concern.

It sounds as if what I need to do is exactly what I am doing, then, and just ignore the warning message.

Cheers,
John

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7575
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Mode 1 video processing & size rounding

Mon Jan 22, 2018 3:13 pm

jpsa wrote:
Mon Jan 22, 2018 2:34 pm
I don't want to resize, as I'm keen to retain as much resolution as possible, and any resizing seems likely to introduce at least a little blurring (either that or aliasing artefacts, which would also be bad).
It's a Bayer camera, so by the time you've run it through demosaicing and similar processing steps, resize isn't going to create a huge amount of blurring.
jpsa wrote:I may also be performance limited, so if there were any CPU cost to resizing, I'd want to avoid that too. But I think from what I've read that's done by the video processor, so probably isn't actually a concern.
Correct - all done in hardware. The hardware is limited to reading or writing one pixel per clock cycle, so downscaling by large factors would limit you, but most use cases will have a very small difference (clock speed is 250MHz by default).
jpsa wrote:It sounds as if what I need to do is exactly what I am doing, then, and just ignore the warning message.
Yes, just ignore the warning, but look at "addr + (1920*1088)" for the U plane, and "addr + (1920*1088) + (1920*1088)/4" for the V plane.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jpsa
Posts: 15
Joined: Wed Feb 04, 2015 8:22 am

Re: Mode 1 video processing & size rounding

Mon Jan 22, 2018 3:21 pm

Actually, I only want the y plane, so that's easy :)
Thanks again!

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24188
Joined: Sat Jul 30, 2011 7:41 pm

Re: Mode 1 video processing & size rounding

Mon Jan 22, 2018 3:32 pm

6by9 wrote:
Mon Jan 22, 2018 1:19 pm
jamesh wrote:
Mon Jan 22, 2018 12:35 pm
It's not scaled. The ISP requires frames that are multiple of 16 in X and 32 in Y (I think that is the right way round) so 1080 is rounded up to 1088. The extra lines are added, IIRC by making the sensor area slightly larger (might be wrong there), not by scaled up the 1080.
No, wrong way around.
stride must be a multiple of 32 pixels, and buffer height a multiple of 16. 1080 is not a multiple of 16, therefore is rounded up to 1088.
Doh!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Return to “Camera board”