PyKakapo
Posts: 5
Joined: Fri Oct 04, 2019 2:43 am

Camera missing frames

Fri Oct 04, 2019 2:50 am

Hi there,
I am kind of a noob with Raspberry Pi so I guess it is probably a dumb question. Still, I have lost so much time upon it that any idea would be greatly appreciated.
I would like to record a 10 sec video at 60 fps. The code is below:

Code: Select all

import picamera
import time

input('Press enter to start recording')
with picamera.PiCamera() as camera:
    camera.resolution = (640,480)
    camera.framerate=30
#    camera.start_preview()
#    input('Press enter to start recording')
#    camera.stop_preview()
    camera.start_recording('my_video.h264')
    camera.wait_recording(10)
    camera.stop_recording()

    print('Record done')
The issue is that, although in the preview everything is fine, the final product has missing frames. My feeling is that every sec or so the camera loses a frame, kind of like it is trying to autocheck itself or similar.
Am I perhaps missing something basic?

Thanks!

mohit1
Posts: 10
Joined: Sat Nov 10, 2018 10:23 am

Re: Camera missing frames

Fri Oct 04, 2019 5:30 am

Which version of pi ?

eniovasconcellos
Posts: 4
Joined: Tue Oct 01, 2019 2:36 pm

Re: Camera missing frames

Fri Oct 04, 2019 11:18 am

is your sdcard class10? looks like the problem is about writing speed .

User avatar
HermannSW
Posts: 1610
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Camera missing frames

Fri Oct 04, 2019 12:48 pm

If you have a v2 camera you can write 10 seconds 640x480 video even at 150fps without frameskips. v1 camera can do same at 90fps. Use raspivid "-pts file.pts" option and analyze the timestamps written by "./ptsanalyze file.pts 0":
https://github.com/Hermann-SW/userland/ ... ptsanalyze

Here is an example video I took with v2 NoIR camera where I recorded 640x480@150fps recently without a single frame skip ( with only 50µs shutter time, lit by a very bright 48led IR light). Please @25fps on youtube, 6× slower than real:
https://www.youtube.com/watch?time_cont ... pu8OzF8xrc

In case ptsanalyze confirms no frame skips with 90/150 frames per second with raspivid, it is a picamera issue.
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

mohit1
Posts: 10
Joined: Sat Nov 10, 2018 10:23 am

Re: Camera missing frames

Sun Oct 06, 2019 5:22 am

If its old version of pi
Solution is overclocking gpu and ram
core_freq=500
#h264_freq=300
#gpu_freq=500
sdram_freq=600
over_voltage=6
gpu_mem=150

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Camera missing frames

Sun Oct 06, 2019 9:01 am

mohit1 wrote:
Sun Oct 06, 2019 5:22 am
If its old version of pi
Solution is overclocking gpu and ram
core_freq=500
#h264_freq=300
#gpu_freq=500
sdram_freq=600
over_voltage=6
gpu_mem=150
???! Where did you get that one from?
The GPU is quite capable of doing VGA30 video encode at stock clocks and voltages. It's capable of 1080P30 at them too.
Some Pi0's have seen issues of undervoltage when driving the camera hard, but that is very different.

My guess would be that they have either a slow sd card (as has already been suggested), or a dodgy cable to the camera module (too many CRC errors will drop the frame).

The OP hasn't been back since 7 minutes after posting his question, so I suspect this will be a dead thread.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

PyKakapo
Posts: 5
Joined: Fri Oct 04, 2019 2:43 am

Re: Camera missing frames

Sun Oct 06, 2019 7:35 pm

eniovasconcellos wrote:
Fri Oct 04, 2019 11:18 am
is your sdcard class10? looks like the problem is about writing speed .
Yes, it is. You are probably right, that might be the issue then. Thank you!

PyKakapo
Posts: 5
Joined: Fri Oct 04, 2019 2:43 am

Re: Camera missing frames

Sun Oct 06, 2019 7:36 pm

mohit1 wrote:
Fri Oct 04, 2019 5:30 am
Which version of pi ?
Pi 3/ Camera v1

User avatar
HermannSW
Posts: 1610
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Camera missing frames

Sun Oct 06, 2019 7:50 pm

Please run the raspivid tests I mentioned, then you will know whether it is a SD card problem, or a picamera problem -- as I stated there is no problem to capture 640x480@90fps with zero frame skips for your v1 camera with raspivid, even over a longer period of time.
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

PyKakapo
Posts: 5
Joined: Fri Oct 04, 2019 2:43 am

Re: Camera missing frames

Mon Oct 07, 2019 2:08 am

HermannSW wrote: Please run the raspivid tests I mentioned, then you will know whether it is a SD card problem, or a picamera problem -- as I stated there is no problem to capture 640x480@90fps with zero frame skips for your v1 camera with raspivid, even over a longer period of time.
Thank you, Hermann.

I did download your code but it is not clear how to run it (sorry pretty newbie with Linux too). I did download it on the Desktop, create the executable i.e. chmod +x ptsanalyze and try : ./ ptsanalyze my_video.h264
Is that what I am supposed to do?

Since I need to post-process frame by frame ( I need to locate the movement of some particles), I was wondering if the best strategy is:
1) Recording a full video, then break it into frames with Python and analyze it
2) Capture and process each single image

In case 1, I feel the bottleneck is how big the file can be before I lose frames (I intend to record at minimum 30 Hz). However, based on what you said it seems achievable.
In case 2, I see 2 challenges:
- capture an image fast enough (it seems there is a preview time)
- manipulate the image at GPU level and save on the sd card only the results of the calculation

What would you suggest?

Thanks a lot

User avatar
HermannSW
Posts: 1610
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Camera missing frames

Mon Oct 07, 2019 9:05 am

Please run these commands, one after the other (you missed "0" as 2nd arg to ptsanalyze):

Code: Select all

$ raspivid -md 7 -w 640 -h 480 -fps 90 -pts file.pts -t 20000 -o tst.h264
$ ./ptsanalyze file.pts 0
I would always prefer option 1 if that is feasible, then you have enough time to process the frames.

On option two I did write a gstreamer plugin (redfilter):
https://www.raspberrypi.org/forums/view ... 3&t=247034

It depends on how much time your analysis will take per frame.
Minimal framerate is 42fps for 640x480 mode 6 and v2 camera:
https://www.raspberrypi.org/documentati ... /camera.md

You do have less than 24ms time to proces a frame.

In case you use raspivid, you have to deal with .h264 frames, for processing raspividyuv is easier to deal with. It gives you Y, U and V planes, and if processing grey frames is needed, you can just ignore U and V and process the widthxheight Y frame:
https://www.raspberrypi.org/forums/view ... 2#p1479262
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

PyKakapo
Posts: 5
Joined: Fri Oct 04, 2019 2:43 am

Re: Camera missing frames

Thu Oct 10, 2019 10:05 pm

HermannSW wrote: Please run these commands, one after the other (you missed "0" as 2nd arg to ptsanalyze):

Code: Select all

$ raspivid -md 7 -w 640 -h 480 -fps 90 -pts file.pts -t 20000 -o tst.h264
$ ./ptsanalyze file.pts 0
I would always prefer option 1 if that is feasible, then you have enough time to process the frames.

On option two I did write a gstreamer plugin (redfilter):
https://www.raspberrypi.org/forums/view ... 3&t=247034

It depends on how much time your analysis will take per frame.
Minimal framerate is 42fps for 640x480 mode 6 and v2 camera:
https://www.raspberrypi.org/documentati ... /camera.md

You do have less than 24ms time to proces a frame.

In case you use raspivid, you have to deal with .h264 frames, for processing raspividyuv is easier to deal with. It gives you Y, U and V planes, and if processing grey frames is needed, you can just ignore U and V and process the widthxheight Y frame:
https://www.raspberrypi.org/forums/view ... 2#p1479262
Dear Hermann,
thank you very much for this! As you said, no frame was skipped:

Code: Select all

pi@raspberrypi:~/Desktop/Record $ raspivid -md 7 -w 640 -h 480 -fps 90 -pts file.pts -t 20000 -o tst.h264
pi@raspberrypi:~/Desktop/Record $ ./ptsanalyze file.pts 0
creating tstamps.csv
1805 frames were captured at 90fps
frame delta time[us] distribution
      1 11068
      1 11071
      1 11072
      2 11073
      1 11074
      1 11076
      3 11077
      2 11078
      3 11080
      2 11082
      1 11083
      5 11084
      1 11086
      4 11087
     68 11088
    208 11089
    249 11090
    668 11091
    257 11092
    209 11093
     80 11094
      9 11095
      3 11096
      3 11098
      2 11099
      4 11101
      1 11102
      2 11103
      2 11104
      2 11105
      1 11107
      2 11109
      2 11110
      1 11111
      1 11114
after skip frame indices (middle column)
0 frame skips (0%)
When looking at the video still it seems odd, with some frames missing. Then, I sent it and watch it on my laptop (instead of the pi) and actually it looked perfect. So I wonder if simply the pi has some limitations when displaying videos.

Thank you so much for your help, that's greatly appreciated!

User avatar
HermannSW
Posts: 1610
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Camera missing frames

Fri Oct 11, 2019 2:24 am

> When looking at the video still it seems odd, with some frames missing.
> Then, I sent it and watch it on my laptop (instead of the pi) and actually it looked perfect.
> So I wonder if simply the pi has some limitations when displaying videos.
>
With which software on the Pi?
(mplayer, vlc, gstreamer, /opt/vc/src/hello_pi/hello_video/hello_video.bin, ...)
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

mohit1
Posts: 10
Joined: Sat Nov 10, 2018 10:23 am

Re: Camera missing frames

Sat Oct 12, 2019 5:54 am

Try this and it will fix it
I spend lots of time asking them qs and trying to debug the issue
What worked is
core_freq=500
#h264_freq=300
#gpu_freq=500
sdram_freq=600
over_voltage=6
gpu_mem=150

sdram_freq=600 because i was storing video in it in tmpfs mount
You may not need it

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Camera missing frames

Sat Oct 12, 2019 8:22 am

mohit1 wrote:
Sat Oct 12, 2019 5:54 am
Try this and it will fix it
I spend lots of time asking them qs and trying to debug the issue
What worked is
core_freq=500
#h264_freq=300
#gpu_freq=500
sdram_freq=600
over_voltage=6
gpu_mem=150

sdram_freq=600 because i was storing video in it in tmpfs mount
You may not need it
You've quoted this twice but I still don't see why. Vga capture and h264 is a very light task for the imaging pipeline and the arm cores, so there should be no need to over clock or over volt anything. It might be worth the op giving it a try, but I'm sceptical that it will help.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”