6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Tue Aug 07, 2018 2:39 pm

zim wrote:
Tue Aug 07, 2018 2:37 pm
Thank you for looking into it, and nice to know that there is a fix coming along!
No problem.
Thank you for the report, and sorry if I was initially dismissive. We do try our best to avoid regressions, but testing can never be totally exhaustive.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5341
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Official V4L2 driver

Wed Aug 08, 2018 5:19 pm

6by9 wrote:
Tue Aug 07, 2018 9:52 am
I don't know when the next rpi-update is going to be done, but the fix should be in there. I'll be doing some more testing in the meantime.
Fix should be present in latest rpi-update firmware.

judovana
Posts: 1
Joined: Tue Oct 23, 2018 4:04 pm

Re: Official V4L2 driver

Tue Oct 23, 2018 4:06 pm

Hi!

Is there a way I can build this driver for fedora? Or generally, was somebody able to make official cammera running on fedora? Is there better place to ask? (i'm running f29 on pi3 B+ aarch64)

dmatthews
Posts: 2
Joined: Thu Dec 06, 2018 3:18 am

Re: Official V4L2 driver

Thu Dec 06, 2018 3:41 am

Is it possible to access Bayer image data directly, without stuffing it into the EXIF data on a JPEG?

Earlier in this discussion, I found a passing mention of Bayer data getting transferred directly to memory before being read out again to the ISP. Is it possible to access the data at that point? Or to just pass the data through the GPU unchanged?

My application is pretty atypical, and I'm hoping to access full-sensor raw data faster than the 1-2 FPS I'm getting from the Python picamera module. If this is not possible, I would greatly appreciate it if someone could help me understand what's preventing it so I can stop banging my head against the wall.

Thanks for all your work!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Thu Dec 06, 2018 3:02 pm

dmatthews wrote:
Thu Dec 06, 2018 3:41 am
Is it possible to access Bayer image data directly, without stuffing it into the EXIF data on a JPEG?

Earlier in this discussion, I found a passing mention of Bayer data getting transferred directly to memory before being read out again to the ISP. Is it possible to access the data at that point? Or to just pass the data through the GPU unchanged?

My application is pretty atypical, and I'm hoping to access full-sensor raw data faster than the 1-2 FPS I'm getting from the Python picamera module. If this is not possible, I would greatly appreciate it if someone could help me understand what's preventing it so I can stop banging my head against the wall.

Thanks for all your work!
Pretty much a duplicate of viewtopic.php?f=43&t=52203&p=1400929#p1400929

And there's the minor point that there is no way to get the Bayer data through the bcm2835-v4l2 driver (JPEG+RAW is not supported).

There is a very basic kernel driver for ov5647 (v1 camera) running in raw mode, but it only supports 640x480 at a fixed frame rate. That can be used with the bcm2835-unicam driver, but you're better off looking at raspiraw, as has already been referenced in the linked thread.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hazeii
Posts: 31
Joined: Sat Apr 25, 2015 8:33 am

Re: Official V4L2 driver

Fri Apr 05, 2019 1:44 pm

Having recently switched to a new release of Raspbian, some custom software that reads stills from the camera via the V4L2 interface is going a lot slower. It appears to be because the camera is going into 'stills mode' at a lower resolution than before, despite the presence of 'max_video_height' and 'max_video_width' settings.

Here's how the module is being loaded:-

sudo modprobe bcm2835-v4l2 max_video_width=3280 max_video_height=2464

On an older system (4.4.13-v7+ #894), dmesg reports the following:-

Code: Select all

[   12.747648] media: Linux media interface: v0.10
[   12.764754] Linux video capture interface: v2.00
[   12.810751] bcm2835-v4l2: scene mode selected 0, was 0
[   12.811003] bcm2835-v4l2: V4L2 device registered as video0 - stills mode > 3280x2464 <--- Correct
On the new system (4.19.32-v7+ #1210, after an 'rpi-update' to see if that sorted the issue):

Code: Select all

[    4.203845] bcm2835-codec bcm2835-codec: Device registered as /dev/video10
[    4.203861] bcm2835-codec bcm2835-codec: Loaded V4L2 decode
[    4.208662] bcm2835-codec bcm2835-codec: Device registered as /dev/video11
[    4.208688] bcm2835-codec bcm2835-codec: Loaded V4L2 encode
[    4.227512] bcm2835-codec bcm2835-codec: Device registered as /dev/video12
[    4.227526] bcm2835-codec bcm2835-codec: Loaded V4L2 isp
[    4.239165] bcm2835-v4l2: scene mode selected 0, was 0
[    4.240636] bcm2835-v4l2: V4L2 device registered as video0 - stills mode > 1280x720 <--- Not as requested
[    4.250138] bcm2835-v4l2: Broadcom 2835 MMAL video capture ver 0.0.2 loaded.
Any suggestions on how to fix this, so we get 'video mode' (i.e. faster stills capture) as higher resolution, e.g. 1640x1232?

TIA!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Fri Apr 05, 2019 2:30 pm

hazeii wrote:
Fri Apr 05, 2019 1:44 pm
Having recently switched to a new release of Raspbian, some custom software that reads stills from the camera via the V4L2 interface is going a lot slower. It appears to be because the camera is going into 'stills mode' at a lower resolution than before, despite the presence of 'max_video_height' and 'max_video_width' settings.

Here's how the module is being loaded:-

sudo modprobe bcm2835-v4l2 max_video_width=3280 max_video_height=2464

On an older system (4.4.13-v7+ #894), dmesg reports the following:-

Code: Select all

[   12.747648] media: Linux media interface: v0.10
[   12.764754] Linux video capture interface: v2.00
[   12.810751] bcm2835-v4l2: scene mode selected 0, was 0
[   12.811003] bcm2835-v4l2: V4L2 device registered as video0 - stills mode > 3280x2464 <--- Correct
On the new system (4.19.32-v7+ #1210, after an 'rpi-update' to see if that sorted the issue):

Code: Select all

[    4.203845] bcm2835-codec bcm2835-codec: Device registered as /dev/video10
[    4.203861] bcm2835-codec bcm2835-codec: Loaded V4L2 decode
[    4.208662] bcm2835-codec bcm2835-codec: Device registered as /dev/video11
[    4.208688] bcm2835-codec bcm2835-codec: Loaded V4L2 encode
[    4.227512] bcm2835-codec bcm2835-codec: Device registered as /dev/video12
[    4.227526] bcm2835-codec bcm2835-codec: Loaded V4L2 isp
[    4.239165] bcm2835-v4l2: scene mode selected 0, was 0
[    4.240636] bcm2835-v4l2: V4L2 device registered as video0 - stills mode > 1280x720 <--- Not as requested
[    4.250138] bcm2835-v4l2: Broadcom 2835 MMAL video capture ver 0.0.2 loaded.
Any suggestions on how to fix this, so we get 'video mode' (i.e. faster stills capture) as higher resolution, e.g. 1640x1232?
bcm2835-v4l2 is a platform driver now, so is automatically loaded at boot time. Your modprobe line is doing nothing.

Either unload bcm2835-v4l2 first, or create a file (name not important, but typically something like bcm2835_v4l2.conf) in /etc/modprobe.d/ containing

Code: Select all

options bcm2835-v4l2 max_video_width=3280
options bcm2835-v4l2 max_video_height=2464
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hazeii
Posts: 31
Joined: Sat Apr 25, 2015 8:33 am

Re: Official V4L2 driver

Fri Apr 05, 2019 4:12 pm

Many thanks for the speedy reply, that solved it!

hazeii
Posts: 31
Joined: Sat Apr 25, 2015 8:33 am

Re: Official V4L2 driver

Mon Apr 08, 2019 9:29 am

Following on from the above, we're having problems grabbing big frames (inc. kernel crashes) with the latest firmware unless we increase GPU memory. There are varying symptoms depending on video format and frame size but possibly all related to some underlying memory issue. Easiest way I've found to reproduce is as follows:-

1. Stock install of 2018-11-13-raspbian-stretch-lite on Pi 3 B+ with Sony camera (with or without rpi-update)
2. Load video module with max_video_width/height set to 3280 and 2464
3. Build the capture.c example from v4l2.
4. Set video format: v4l2-ctl --set-fmt-video=width=3280,height=2464,pixelformat=YUV420
5. Run 'capture' example; result here is " VIDIOC_STREAMON error 1, Operation not permitted\n"' (whether root or not)
6. 'dmesg' reports error (see below).
7. Set video format: v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=YUV420
8. Run 'capture' again; generally results in system lockup.

Setting gpu_mem to 256 (from stock 128) and all works as expected, hence possible memory allocation issue? We're tight on memory so we don't want to have to give up the extra memory if it can be avoided.

This is on 4.19.32-v7+ #1210; the older version 4.4.13-v7+ #894 works as expected. Also reproducible by moving the SD card with the later version into older, known good systems (running the older version in the new systems also works as expected).

FWIW problems start showing up at 1640x1232, and YUYV seems to be more of an issue than YU12, e.g. setting max_video_height and width to 1920x1080 is ok for YU12 but hangs (without dmesg error) for YUYV. With these settings trying to capture at 3280x2484 (which would be in stills mode) works (slowly) for YU12 but fails for YUYV.

Here's the relevant bits from 'dmesg':

Code: Select all

[    5.247131] bcm2835-v4l2: scene mode selected 0, was 0
[    5.247548] bcm2835-v4l2: V4L2 device registered as video0 - stills mode > 3280x2464
[    5.253652] bcm2835-v4l2: Broadcom 2835 MMAL video capture ver 0.0.2 loaded.

Code: Select all

[  165.693646] bcm2835-v4l2: Failed to enable capture port - error -1. Disabling camera port again
[  165.720695] ------------[ cut here ]------------
[  165.720757] WARNING: CPU: 3 PID: 649 at drivers/media/common/videobuf2/videobuf2-core.c:1331 vb2_start_streaming+0xe4/0x160 [videobuf2_common]
[  165.720763] Modules linked in: cmac bnep hci_uart btbcm serdev bluetooth ecdh_generic evdev brcmfmac brcmutil sha256_generic cfg80211 bcm2835_codec(C) rfkill bcm2835_v4l2(C) v4l2_mem2mem v4l2_common bcm2835_mmal_vchiq(C) videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 snd_bcm2835(C) videobuf2_common raspberrypi_hwmon snd_pcm hwmon snd_timer videodev media snd vc_sm_cma(C) fixed uio_pdrv_genirq uio ip_tables x_tables ipv6
[  165.720981] CPU: 3 PID: 649 Comm: capture-v4l Tainted: G         C        4.19.32-v7+ #1210
[  165.720986] Hardware name: BCM2835
[  165.721020] [<80111e8c>] (unwind_backtrace) from [<8010d414>] (show_stack+0x20/0x24)
[  165.721038] [<8010d414>] (show_stack) from [<807ed5c0>] (dump_stack+0xd4/0x118)
[  165.721059] [<807ed5c0>] (dump_stack) from [<80120638>] (__warn+0x104/0x11c)
[  165.721075] [<80120638>] (__warn) from [<80120788>] (warn_slowpath_null+0x50/0x58)
[  165.721115] [<80120788>] (warn_slowpath_null) from [<7f1e96b0>] (vb2_start_streaming+0xe4/0x160 [videobuf2_common])
[  165.721188] [<7f1e96b0>] (vb2_start_streaming [videobuf2_common]) from [<7f1eb01c>] (vb2_core_streamon+0x130/0x16c [videobuf2_common])
[  165.721244] [<7f1eb01c>] (vb2_core_streamon [videobuf2_common]) from [<7f20f254>] (vb2_streamon+0x40/0x60 [videobuf2_v4l2])
[  165.721290] [<7f20f254>] (vb2_streamon [videobuf2_v4l2]) from [<7f20f2c4>] (vb2_ioctl_streamon+0x50/0x54 [videobuf2_v4l2])
[  165.721426] [<7f20f2c4>] (vb2_ioctl_streamon [videobuf2_v4l2]) from [<7f10fe3c>] (v4l_streamon+0x28/0x2c [videodev])
[  165.721623] [<7f10fe3c>] (v4l_streamon [videodev]) from [<7f1127a0>] (__video_do_ioctl+0x268/0x528 [videodev])
[  165.721821] [<7f1127a0>] (__video_do_ioctl [videodev]) from [<7f11643c>] (video_usercopy+0x218/0x600 [videodev])
[  165.722016] [<7f11643c>] (video_usercopy [videodev]) from [<7f116844>] (video_ioctl2+0x20/0x24 [videodev])
[  165.722209] [<7f116844>] (video_ioctl2 [videodev]) from [<7f10e144>] (v4l2_ioctl+0x4c/0x60 [videodev])
[  165.722320] [<7f10e144>] (v4l2_ioctl [videodev]) from [<802abc10>] (do_vfs_ioctl+0xbc/0x804)
[  165.722339] [<802abc10>] (do_vfs_ioctl) from [<802ac39c>] (ksys_ioctl+0x44/0x6c)
[  165.722355] [<802ac39c>] (ksys_ioctl) from [<802ac3dc>] (sys_ioctl+0x18/0x1c)
[  165.722371] [<802ac3dc>] (sys_ioctl) from [<80101000>] (ret_fast_syscall+0x0/0x28)
[  165.722379] Exception stack(0xb2985fa8 to 0xb2985ff0)
[  165.722392] 5fa0:                   0001202c 00000000 00000003 40045612 7ef5d650 40045612
[  165.722404] 5fc0: 0001202c 00000000 000107cc 00000036 00000000 00000000 76f6d000 7ef5d604
[  165.722413] 5fe0: 000226c8 7ef5d5e4 00010984 76eb0c7c
[  165.722422] ---[ end trace f315b2021c204c4e ]---

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Mon Apr 08, 2019 10:26 am

YUYV, being a YUV422 format, uses bigger buffers than YUV420 formats (16bpp vs 12bpp).

8MPix images are big!

Bayer raw 10 they're ~10MB each, and for video mode you need 3, or 30MB.
YUV422 frames are 16MPix each, and again video mode wants 3 for internal allocations, or 48MB.
Each buffer passed back to the ARM requires an equivalent sized buffer, so in the case of capture.c you are asking for 4 buffers in your request to VIDIOC_REQBUFS, so that's another 64MB if YUYV.

Total 142MB just in camera buffers, therefore it will fail if gpu_mem is 128M. You may get away with gpu_mem=160M. (You need to allow for an 8MB framebuffer, and some extra working memory for use cases).
Set gpu_mem to 256M and run your test case. "sudo vcdbg reloc" will tell you the amount of heap still free, so you can reduce gpu_mem by that amount.

It is very uncommon to use YUV422 for video mode in typical use cases, therefore I suspect the internal allocation isn't allowing for that. You may be able to fudge it by asking for "max_video_height=3286" (2464*16/12).

The jump from 4.4 to 4.19 is HUGE - that's well over 2 years worth of work. Please test on 4.9 and 4.14 if you want to try and track down vaguely with which step the requirements have changed. I don't recall any significant changes in that area.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hazeii
Posts: 31
Joined: Sat Apr 25, 2015 8:33 am

Re: Official V4L2 driver

Mon Apr 08, 2019 12:44 pm

We're actually using YUV420 (for our purposes we only need Y), YUYV just made the issue easier to reproduce. Having done a bit of poking about with 'vcdbg reloc' now, it looks like we were just making it with GPU_MEM=128 on the old version (we're using 3 capture buffers rather than the 4 in the sample) and since the version appears to use a bit more memory it tipped it over the edge. That suggests we'll be able to get away with just a small increase in GPU_MEM.

Getting some impressive performance on the Pi 3 B+ now, streaming video 1280x720x8 video at 40FPS out via the LAN (~295Mbps), at higher res. obviously the network interface means rate/resolution tradeoffs (thinking about simple compression schemes to work around that).

Thanks for the explanation, detail and speedy reply!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Mon Apr 08, 2019 2:18 pm

hazeii wrote:
Mon Apr 08, 2019 12:44 pm
We're actually using YUV420 (for our purposes we only need Y), YUYV just made the issue easier to reproduce. Having done a bit of poking about with 'vcdbg reloc' now, it looks like we were just making it with GPU_MEM=128 on the old version (we're using 3 capture buffers rather than the 4 in the sample) and since the version appears to use a bit more memory it tipped it over the edge. That suggests we'll be able to get away with just a small increase in GPU_MEM.

Getting some impressive performance on the Pi 3 B+ now, streaming video 1280x720x8 video at 40FPS out via the LAN (~295Mbps), at higher res. obviously the network interface means rate/resolution tradeoffs (thinking about simple compression schemes to work around that).

Thanks for the explanation, detail and speedy reply!
I'd need to check through exactly how the buffers are allocated. I suspect YUYV will almost always wedge at full resolution as the internal buffers aren't big enough.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Serveurperso
Posts: 7
Joined: Tue May 21, 2019 1:30 pm
Contact: Website

Re: Official V4L2 driver

Tue May 21, 2019 1:55 pm

Hello,

I use V4L2 API to get low latency H.264 30FPS stream - PI 2/3/3B+ and the V1 Sensor module - for a fleet of robots with a total success. (you can test them online at domain "vigibot.com" there is NO ads.)

V4L2 is very cool because I can set the bitrate on the fly to keep the latency very low even in 3G+/4G network problem.

But I can't get more than 640x480 on sensor when I look the video quality.... I can upscale to 800x600 or 1296x972 but it's only a crap upscaling:(

I'm interested with this hardware mode because it match the sensor, I need only 30FPS, not more :
1296x972 4:3 1-42fps x Full 2x2

But I always get this one, upscaled :
640x480 4:3 42.1-60fps x Full 4x4

I tested with GPU_MEM=128 and GPU_MEM=256 without success

I also tested with /etc/modprobe.d/bcm2835-v4l2.conf to set video/still switch max resolution without success :

options bcm2835-v4l2 max_video_width=1296
options bcm2835-v4l2 max_video_height=972

(Parameter are succefuly displayed on /var/log/syslog when the v4l2 driver is started)



I tested on many raspbery PI 2/3B/3B+ with always the same problem : a 640x480 upscaled stream.


Can you help?

Thanks.
https://www.vigibot.com
Robotic fleet cloud (no ads)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Tue May 21, 2019 5:03 pm

Serveurperso wrote:
Tue May 21, 2019 1:55 pm
But I can't get more than 640x480 on sensor when I look the video quality.... I can upscale to 800x600 or 1296x972 but it's only a crap upscaling:(

I'm interested with this hardware mode because it match the sensor, I need only 30FPS, not more :
1296x972 4:3 1-42fps x Full 2x2

But I always get this one, upscaled :
640x480 4:3 42.1-60fps x Full 4x4
How have you determined that it's the wrong mode?
Please provide the output from "v4l2-ctl -V -P" after you've set everything up.
Yes, you will need to increase the max_video_[width|height] parameters.

I can't think of an easily parsable way to confirm the sensor mode chosen, however "vcgencmd set_logging level=0x400", run your app, and then "sudo vcdbg log msg" will dump the logging buffer from the camera subsystem.
On my system with a v2 sensor attached

Code: Select all

v4l2-ctl -v width=1296,height=972,pixelformat=I420
v4l2-ctl --stream-mmap=3 --stream-count=10000 --stream-to=/dev/null
and dumping the log I get the full log including

Code: Select all

1447062.826: camplus_isp_callback [0]: input_lines_consumed=1232, num=972, isp_lines_done=972, flags=3
Being on a V2, I'm in the 1640x1232 mode, and I'd chosen 1296x972 as the output resolution. From the log line, 1232 is the number of input lines consumed, and 972 is the number of output lines produced from those input lines. flags=3 is end of row and end of frame. You will get other very similar callbacks with smaller numbers (but not flags=3) as the input frame is fed into the ISP as stripes as they are received from the sensor.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Serveurperso
Posts: 7
Joined: Tue May 21, 2019 1:30 pm
Contact: Website

Re: Official V4L2 driver

Wed May 22, 2019 6:24 am

FYI Increase of the the max_video_[width|height] parameters + reboot + testing with, or without this confuguration have NO impact on this test, it's, strange.

Result from "v4l2-ctl -V -P" @ my best interesting resolution = 1296 x 972

Code: Select all

(root|~) v4l2-ctl -V -P
Format Video Capture:
        Width/Height      : 1296/972
        Pixel Format      : 'H264'
        Field             : None
        Bytes per Line    : 0
        Size Image        : 1280512
        Colorspace        : SMPTE 170M
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Flags             :
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 30.000 (30000/1000)
        Read buffers     : 1
I get a 1296 x 972 stream with the same 640 x 480 quality but with an upscaled stream !
I can show pixel block on browser upscale video:( it's NOT a full resolution sensor-to-stream I get


But if I choose *by the same way* a 1920x1080 30FPS, I get an *awesome* quality but it's a crop from the center of sensor (near useless for robotics) :
1 1920x1080 16:9 1-30fps x Partial None

Code: Select all

(root|~) v4l2-ctl -V -P
Format Video Capture:
        Width/Height      : 1920/1080
        Pixel Format      : 'H264'
        Field             : None
        Bytes per Line    : 0
        Size Image        : 2088960
        Colorspace        : SMPTE 170M
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Flags             :
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 30.000 (30000/1000)
        Read buffers     : 1
Source doc for v1 module supported resolution is https://picamera.readthedocs.io/en/rele ... 2/fov.html

I extracted raw h.264 stream and encapsulated inside mp4 (without recompression) to show you the problem :

Please look standard quality, this is the factory configuration for all PI robots, 1 stream pixel = 1 sensor pixel :
https://www.serveurperso.com/temp/6by9/ ... efault.mp4

Good image quality (but useless crop mode), where we also get 1 stream pixel = 1 sensor pixel,
(please do not pay attention to the blur of the lens on corners of video) :
https://www.serveurperso.com/temp/6by9/ ... seless.mp4
It's the proof for me my software work but there is a driver or other problem:(This is why I ask help.

1296x972 video but there is PIXELS VISIBLE:'( it's the problem please help... There is like 2x2 pixels block on this stream) :
https://www.serveurperso.com/temp/6by9/ ... orMode.mp4
I can't think of an easily parsable way to confirm the sensor mode chosen, however "vcgencmd set_logging level=0x400", run your app, and then "sudo vcdbg log msg" will dump the logging buffer from the camera subsystem.
I do this soon and EDIT this post.

Thanks a lot for your help 6by9
https://www.vigibot.com
Robotic fleet cloud (no ads)

Serveurperso
Posts: 7
Joined: Tue May 21, 2019 1:30 pm
Contact: Website

Re: Official V4L2 driver

Wed May 29, 2019 8:05 am

Code: Select all

vcdbg log msg
There is segfault I think inside the vcdbg command itself, and not inside the log
Because this segfault don't appear without the high verbose level.

I get :

Code: Select all

145046.358: camplus_isp_callback [0]: input_lines_consumed=424, num=444, isp_lines_done=444, flags=1
It's very strange, 424x444 seem very low ! I don't understand this, because I requested for 1296 x 972 !



145042.616: camplus_device_callback [0]: num_lines=480, in_frames_done=1466, cb_flags=0, cdi_write_index=2
145042.645: camplus_device_callback [0]: Done, cdi_write_index=2, cdi_next_index=0
145042.665: camplus_preprocess_task_func: retrieved_events=1
145042.703: camplus_preprocess: exit prep_lines_available[0]=480 prep_lines_done[0]=480
145042.746: camplus_isp_process [0]: channel begin, isp_lines_pending=360 isp_lines_done=0 isp_wrap_line=0 output_lines_done=0
145042.787: camplus_isp_process [0]: channel end, isp_lines_pending=480 isp_wrap_line=0 output_lines_done=0
145045.414: camplus_device_callback [0]: num_lines=600, in_frames_done=1466, cb_flags=0, cdi_write_index=2
145045.445: camplus_device_callback [0]: Done, cdi_write_index=2, cdi_next_index=0
145045.464: camplus_preprocess_task_func: retrieved_events=1
145045.504: camplus_preprocess: exit prep_lines_available[0]=600 prep_lines_done[0]=600
145045.550: camplus_isp_process [0]: channel begin, isp_lines_pending=480 isp_lines_done=0 isp_wrap_line=0 output_lines_done=0
145045.595: camplus_isp_process [0]: channel end, isp_lines_pending=600 isp_wrap_line=0 output_lines_done=0
145046.358: camplus_isp_callback [0]: input_lines_consumed=424, num=444, isp_lines_done=444, flags=1
145046.390: camplus_postprocess_task_func: retrieved_events=1
145046.430: camplus_postprocess_run [0]: enter, postp_lines_available=0, postp_lines_done=0, strip_wrap_line=0, isp_lines_done=444
145048.191: camplus_device_callback [0]: num_lines=720, in_frames_done=1466, cb_flags=0, cdi_write_index=2
145048.218: camplus_device_callback [0]: Done, cdi_write_index=2, cdi_next_index=0
145048.233: camplus_preprocess_task_func: retrieved_events=1
145048.268: camplus_preprocess: exit prep_lines_available[0]=720 prep_lines_done[0]=720
145048.305: camplus_isp_process [0]: channel begin, isp_lines_pending=600 isp_lines_done=444 isp_wrap_line=0 output_lines_done=0
145048.345: camplus_isp_process [0]: channel end, isp_lines_pending=720 isp_wrap_line=0 output_lines_done=0
145049.589: camplus_stripe_ready: sending stripe out, height=400 offset=0
145049.613: camplus_get_ctrl [0]: enter, control=input_buffer
145049.633: camplus_get_ctrl [0]: exit
145049.664: camplus_grab_stripe: enter, flags=0
145049.681: camplus_grab_stripe: exit. stripe_request=4400
145049.703: camplus_stripe_ready: done, stripe_lines_done=400 postp_stripes_done=4399
145049.744: camplus_postprocess: Done. active_channel=0, postp_lines_available=444, postp_lines_done=400, stripe_lines_done=400, postp_frames_done=1466
145049.763: camplus_postprocess_task_func: retrieved_events=1
145049.803: camplus_postprocess_run [0]: enter, postp_lines_available=444, postp_lines_done=400, strip_wrap_line=0, isp_lines_done=444
145049.897: camplus_postprocess: Done. active_channel=0, postp_lines_available=444, postp_lines_done=400, stripe_lines_done=400, postp_frames_done=1466
145050.974: camplus_device_callback [0]: num_lines=840, in_frames_done=1466, cb_flags=0, cdi_write_index=2
145051.000: camplus_device_callback [0]: Done, cdi_write_index=2, cdi_next_index=0
145051.015: camplus_preprocess_task_func: retrieved_events=1
145051.047: camplus_preprocess: exit prep_lines_available[0]=840 prep_lines_done[0]=840
145051.085: camplus_isp_process [0]: channel begin, isp_lines_pending=720 isp_lines_done=444 isp_wrap_line=0 output_lines_done=0
145051.120: camplus_isp_process [0]: channel end, isp_lines_pending=840 isp_wrap_line=0 output_lines_done=0
145053.760: camplus_device_callback [0]: num_lines=960, in_frames_done=1466, cb_flags=0, cdi_write_index=2
145053.788: camplus_device_callback [0]: Done, cdi_write_index=2, cdi_next_index=0
145053.805: camplus_preprocess_task_func: retrieved_events=1
145053.838: camplus_preprocess: exit prep_lines_available[0]=960 prep_lines_done[0]=960
145053.878: camplus_isp_process [0]: channel begin, isp_lines_pending=840 isp_lines_done=444 isp_wrap_line=0 output_lines_done=0
145053.914: camplus_isp_process [0]: channel end, isp_lines_pending=960 isp_wrap_line=0 output_lines_done=0
145054.057: camplus_device_callback [0]: num_lines=972, in_frames_done=1466, cb_flags=16, cdi_write_index=2
145054.097: camplus_set_cdi_buffer [0]: cdi_write_index=0, cdi_next_index=4, prep_input_index=2, isp_input_index=2
115704.832: ���;��(0������0��(0��$0��
000648.323: �
������@G��� ���$p�����O-� ���L�M�
@��D0

1951596.644: pe error
`4�?�JH��G@����\g�b�t��﮿,�H�([@f"(��Ld@\š��#����WeRT@
Erreur de segmentation
https://www.vigibot.com
Robotic fleet cloud (no ads)

Serveurperso
Posts: 7
Joined: Tue May 21, 2019 1:30 pm
Contact: Website

Re: Official V4L2 driver

Thu Jun 06, 2019 6:43 am

This is not a V4L2 itself problem it's kernel/driver bug
When I do a h264 1296x972 video with Raspivid I get exactly the same bad quality:( For sure it's not a 1296x972 native pixel-to-pixel...

Code: Select all

raspivid -o raspivid-1296x972-30.h264 -w 1296 -h 972 -fps 30 -b 1500000 -t 10000 -rot 180
Raw result : https://www.serveurperso.com/temp/6by9/ ... 72-30.h264

Code: Select all

ffmpeg -i raspivid-1296x972-30.h264 -vcodec copy raspivid-1296x972-30.mp4
MP4 : https://www.serveurperso.com/temp/6by9/ ... 972-30.mp4

When I do a 1920x1080 it's perfect because pixels are not visible (Disregard the low dynamic range and bad focus especially on the right of the picture).
https://www.serveurperso.com/temp/6by9/ ... 080-30.mp4

I found same problem here : viewtopic.php?t=110757
https://www.vigibot.com
Robotic fleet cloud (no ads)

Serveurperso
Posts: 7
Joined: Tue May 21, 2019 1:30 pm
Contact: Website

Re: Official V4L2 driver

Thu Jun 06, 2019 7:10 am

With raspivid and "-md 2" to force mode 2 on 1296x972 30FPS it's good ! Here is the result :
https://www.serveurperso.com/temp/6by9/ ... 30-md2.mp4
GoodSensorMode2.jpg
GoodSensorMode2.jpg (110.64 KiB) Viewed 1948 times
------------------------

Versus the bad automatic mode from V4L2
https://www.serveurperso.com/temp/6by9/ ... 972-30.mp4
BadSensorMode.jpg
BadSensorMode.jpg (99.79 KiB) Viewed 1948 times
There is two way to solve this problem :
- solving the "autoselect mode" heuristic (driver?)
- adding a force sensor mode inside v4l2

Please help : how to force sensor mode 2 this is the best mode for video streaming this bug makes V4L2 API almost useless :'(
https://www.vigibot.com
Robotic fleet cloud (no ads)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Thu Jun 06, 2019 9:58 am

You quoted a thread where the cropping is ever so slightly different between modes, not image quality.
With raspivid and "-md 2" to force mode 2 on 1296x972 30FPS it's good ! Here is the result :
Mode 2 will only run at up to 15fps, so you can't have made a recording with those parameters.

Binning does lose some image quality due to the slight distortion of the Bayer pattern, but not that significant. Downscaling in the ISP is likely to give better quality images. The main reason for using it is that the sensor can't produce unbinned frames at 30fps (except for the cropped 1920x1080 mode).

If you're using 1296x972 then I surmise you're using an OV5647 based V1 sensor as a V2 sensor would be 1640x1232 for the binned mode. Raspberry Pi haven't sold that module since 2015, so I'd suspect you're using a third party version of the sensor. Quality of that isn't up to us.

I will do a quick comparison, but I'm not convinced we have a real issue.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Serveurperso
Posts: 7
Joined: Tue May 21, 2019 1:30 pm
Contact: Website

Re: Official V4L2 driver

Mon Jun 10, 2019 9:04 pm

I deleted my previous post it's a mistake ! I tested @ 15 FPS on v4l2 the heuristic select the correct mode 2.
If I force mode 2 raspivid ignore the 30FPS parameter ! and result a 2* acceleration on my raspivid version this completely fooled me!!!!

They say 42 FPS for 1296x972 on v1 sensor, this fooled me also:
https://picamera.readthedocs.io/en/rele ... 2/fov.html
4 1296x972 4:3 1-42fps x Full 2x2

Also aftermarket-clone v1 camera are the best for small robotics because fisheyes wide angle / larger aperture for sensitivity / electro-mechanical IR block filter for good day color as night vision.
Stock v2 camera are too "Zoomy" this make the First Person View piloting hard, and less immersive/very unpleasant/risky for piloting a fast robot or flying drone.

These camera are a beautiful piece of technology with a game quality FOV, and there is no very wide angle PCB glass lens made before with sufficient focal space for a motorized IR cut filter.
Sans titre.jpg
Sans titre.jpg (46.16 KiB) Viewed 1846 times
https://www.vigibot.com
Robotic fleet cloud (no ads)

Serveurperso
Posts: 7
Joined: Tue May 21, 2019 1:30 pm
Contact: Website

Re: Official V4L2 driver

Tue Jun 11, 2019 6:32 am

No, there is always a bug, image quality is better with raspivid and mode2 even all @ 15 FPS...
Always need a force sensor mode in v4l2
https://www.vigibot.com
Robotic fleet cloud (no ads)

xss
Posts: 2
Joined: Sat Jun 29, 2019 8:15 am

Re: Official V4L2 driver

Sat Jun 29, 2019 8:43 am

Hi gurus,
As a newbie, I don't know what is the different bwtween v4l2 and picamera. I don't mean the language but how to deal with camera data.
I steal a photo from picamera project :
Image

When we use a csi camera, the VPU will handle with this video stream. And picamera will get this data or communication with vpu by VCHI. And picamera actually deal with every thing by MMAL.

But how about V4L2, what is the different part? And if there is a poosible to use v4l2 to develop a driver for new camera sense like imx290/327.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Sat Jun 29, 2019 1:02 pm

xss wrote:
Sat Jun 29, 2019 8:43 am
Hi gurus,
As a newbie, I don't know what is the different bwtween v4l2 and picamera. I don't mean the language but how to deal with camera data.
I steal a photo from picamera project :
When we use a csi camera, the VPU will handle with this video stream. And picamera will get this data or communication with vpu by VCHI. And picamera actually deal with every thing by MMAL.

But how about V4L2, what is the different part?
The v4l2 driver also just sits on top of MMAL.
And if there is a poosible to use v4l2 to develop a driver for new camera sense like imx290/327.
Not using the processing path that sits solely on the vpu.

There is the bcm2835-unicam driver which allows you to capture the raw data off any sensor that you can find or write a driver for. That will generally get you Bayer data, so you need some form of processing to convert to yuv or similar. Parts of the isp are also exposed, but there are no control loops, so ae and awb are totally up to you.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

nutron
Posts: 8
Joined: Sat Apr 28, 2018 3:36 am

Re: Official V4L2 driver

Wed Aug 28, 2019 8:24 pm

Could somebody tell me what the /dev/video10 video11 video12 devices are for ?

Are they generated by v4l2 for some purpose ?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7420
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Wed Aug 28, 2019 8:37 pm

nutron wrote:
Wed Aug 28, 2019 8:24 pm
Could somebody tell me what the /dev/video10 video11 video12 devices are for ?

Are they generated by v4l2 for some purpose ?
Yes, they are created for a purpose. They are all memory to memory (M2M) devices for the video decoder, encoder, and ISP resize respectively.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”