ZuLuuuuuu
Posts: 26
Joined: Thu May 23, 2013 6:30 am

capture_sequence captures first 3 images before "yield"

Mon Mar 02, 2015 7:03 pm

Hello,

I have a projector that I use to project several images onto a scene. I project several photos onto the scene sequentially and try to take 1 picture of every different image projected onto the scene. I am trying to capture photos of the scene using capture_sequence function and I enable video_port parameter to take photos faster.

I have a generator function to yield a new stream object to the function for every photo I take. This generator function takes almost 1 second to yield a new stream since I project a different image onto the scene in between frames and it takes 1 second to project a new photo onto the scene and save the capture onto the disk.

The weird thing is that the first 3 images show the same image projected onto the scene. I change the image between every frame using the generator function (and I do this synchronously). I even put sleeps of several seconds to make sure the new image is shown before yield statement returns but it doesn't matter, the first 3 photo is still the same.

If I disable video_port parameter, everything works fine. Every photo taken shows a different image projected onto the scene.

What might be the problem? Is there some buffering going on in video port when I initialize the capture_sequence function? What can I do to overcome this problem?

You can find the relevant parts of the code:

Code: Select all

    def capture(self):
        self.__pi_camera.capture_sequence(self._feed_capture(), format='yuv', use_video_port=True)

    def _feed_capture(self):
        frame = 0

        while frame < 10:
            self.__projector.show_image(get_next_image_to_project())
            # Wait until image is shown
            while self.__projector.image_shown() is False:
                time.sleep(0.05)

            stream = io.BytesIO()
            yield stream

            stream.seek(0)
            captured_grayscale_array = numpy.fromstring(stream.getvalue(), dtype=numpy.uint8,
                                                        count=yuv_width * yuv_height).reshape(
                (yuv_height, yuv_width))

                self.save_array_as_image(captured_grayscale_array, str(frame) + ".jpg")

            frame += 1

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7542
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: capture_sequence captures first 3 images before "yield"

Mon Mar 02, 2015 8:36 pm

You're using the video port, so entirely expected.
In video mode the system will do its utmost to maintain the requested frame rate and fill up all buffering it has before dropping frames. The hope is that whatever is downstream will have got going before frames actually get dropped.
There are 3 input buffers to accept data from the sensor for further processing. You've allowed those to be filled and then stalled the entire imaging pipe as you're not collecting the resulting frames.
If you want frames at a slower rate from the video port, then you have to drop the frame rate that you are asking from it. Or use the stills port as that is designed to grab frames on demand.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ZuLuuuuuu
Posts: 26
Joined: Thu May 23, 2013 6:30 am

Re: capture_sequence captures first 3 images before "yield"

Mon Mar 02, 2015 9:01 pm

6by9 wrote:You're using the video port, so entirely expected.
In video mode the system will do its utmost to maintain the requested frame rate and fill up all buffering it has before dropping frames. The hope is that whatever is downstream will have got going before frames actually get dropped.
There are 3 input buffers to accept data from the sensor for further processing. You've allowed those to be filled and then stalled the entire imaging pipe as you're not collecting the resulting frames.
If you want frames at a slower rate from the video port, then you have to drop the frame rate that you are asking from it. Or use the stills port as that is designed to grab frames on demand.
Slowing down the FPS wouldn't be the best option for me since I want it to be as fast as possible (even 0.1 seconds is important for me) so I cannot give an ideal FPS number.

1) Maybe one option is to grab 3 frames every time, would that be as slow as grabbing via still port or is there an advantage of video port besides the buffering?

2) Is there a way to tell camera api to flush the video buffers and grab a fresh frame?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7542
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: capture_sequence captures first 3 images before "yield"

Tue Mar 03, 2015 7:51 am

ZuLuuuuuu wrote:Slowing down the FPS wouldn't be the best option for me since I want it to be as fast as possible (even 0.1 seconds is important for me) so I cannot give an ideal FPS number.

1) Maybe one option is to grab 3 frames every time, would that be as slow as grabbing via still port or is there an advantage of video port besides the buffering?

2) Is there a way to tell camera api to flush the video buffers and grab a fresh frame?
Well each to their own.

1) Try it. Video port also disables a chunk of extra processing.

2) No.

Picamera isn't supported by the Foundation - it's a library done by waveform80. The source is available, so feel free to go and play.
I have no idea how Picamera actually drives the GPU, but if it enabled and disabled capturing on the video port between each request then the GPU would get on with the extra frames as preview frames, thereby flushing the queue.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”