Official V4L2 driver

[unromeo]

Hi everybody,

First, please excuse my English.

I need to take advantage of the long exposure capabilities of this camera (not more than 2-3 seconds exposure, for an astronomical autoguiding system). This is possible with raspistill, but the application works only with the V4L2 driver.

Would it be even possible to get those long exposure with the official V4L2 driver ?

Thank you.

[Vdl]

Hello All,

Racer asked me for some help and we investigated this problem together. The result is issue I've raised: https://github.com/raspberrypi/linux/issues/1196

We tried to use suggested solutions:
1) call ctrl_set_awb_mode from start_streaming doesn't help at all
2) bm2835_mmal_set_all_camera_controls doesn't help also

To reproduce this issue, easiest way is change white balance to something crazy, like very "reddish" values. Correct result would be red picture then, incorrect - nice looking images taken with AWB.
My test script looks like:

Code: Select all

v4l2-ctl -v width=1024,height=768,pixelformat=I420
v4l2-ctl --set-ctrl=white_balance_auto_preset=0,red_balance=5000,blue_balance=1500
v4l2-ctl --set-ctrl=brightness=55,contrast=15,saturation=0,horizontal_flip=1,vertical_flip=1,power_line_frequency=1,sharpness=0,color_effects=0,rotate=0,color_effects_cbcr=32896
v4l2-ctl --set-ctrl=auto_exposure=1,exposure_time_absolute=300,auto_exposure_bias=12,iso_sensitivity=1
v4l2-ctl -p 10

v4l2-ctl --stream-mmap=3 --stream-skip=2 --stream-to=./test.yuv --stream-count=4
split -d -b $((3*1024*768/2)) ./test.yuv ./test-
for i in ./test-??; do mv $i $i.yuv;convert -size 1024x768 -depth 8 -quality 100 $i.yuv $i.jpg;done

If you run it, the result will be incorrect. But if you turn on overlay (preview) before, it works correctly. So this behave is overlay/preview dependent.
Running without overlay gives ugly results in system log:

Code: Select all

[ 9119.801453] CPU: 1 PID: 1573 Comm: v4l2-ctl Not tainted 4.1.7-v7+ #817
[ 9119.801466] Hardware name: BCM2709
[ 9119.801511] [<80018440>] (unwind_backtrace) from [<80013e0c>] (show_stack+0x20/0x24)
[ 9119.801542] [<80013e0c>] (show_stack) from [<80558548>] (dump_stack+0x98/0xe0)
[ 9119.801571] [<80558548>] (dump_stack) from [<80026a4c>] (warn_slowpath_common+0x8c/0xc8)
[ 9119.801600] [<80026a4c>] (warn_slowpath_common) from [<80026b44>] (warn_slowpath_null+0x2c/0x34)
[ 9119.801634] [<80026b44>] (warn_slowpath_null) from [<7f2f5274>] (__vb2_queue_cancel+0xf8/0x15c [videobuf2_core])
[ 9119.801695] [<7f2f5274>] (__vb2_queue_cancel [videobuf2_core]) from [<7f2f74dc>] (vb2_internal_streamoff+0x40/0xd8 [videobuf2_core])
[ 9119.801739] [<7f2f74dc>] (vb2_internal_streamoff [videobuf2_core]) from [<7f2f75b0>] (vb2_streamoff+0x3c/0x60 [videobuf2_core])
[ 9119.801784] [<7f2f75b0>] (vb2_streamoff [videobuf2_core]) from [<7f2f7624>] (vb2_ioctl_streamoff+0x50/0x54 [videobuf2_core])
[ 9119.801840] [<7f2f7624>] (vb2_ioctl_streamoff [videobuf2_core]) from [<7f2c1b7c>] (v4l_streamoff+0x28/0x2c [videodev])
[ 9119.801915] [<7f2c1b7c>] (v4l_streamoff [videodev]) from [<7f2c4ed4>] (__video_do_ioctl+0x270/0x30c [videodev])
[ 9119.801993] [<7f2c4ed4>] (__video_do_ioctl [videodev]) from [<7f2c4870>] (video_usercopy+0x180/0x550 [videodev])
[ 9119.802071] [<7f2c4870>] (video_usercopy [videodev]) from [<7f2c4c5c>] (video_ioctl2+0x1c/0x24 [videodev])
[ 9119.802136] [<7f2c4c5c>] (video_ioctl2 [videodev]) from [<7f2c06d8>] (v4l2_ioctl+0xb0/0xd0 [videodev])
[ 9119.802182] [<7f2c06d8>] (v4l2_ioctl [videodev]) from [<801566cc>] (do_vfs_ioctl+0x420/0x618)
[ 9119.802208] [<801566cc>] (do_vfs_ioctl) from [<80156908>] (SyS_ioctl+0x44/0x6c)
[ 9119.802234] [<80156908>] (SyS_ioctl) from [<8000f980>] (ret_fast_syscall+0x0/0x54)
[ 9119.802249] ---[ end trace d0054e83a240323d ]---

After long investigation we found the reason. It is described in RaspiCam documentation (RaspiStill works fine):

The preview display is optional, but can be used full screen or directed to a specific rectangular area on the display. If preview is disabled, the null_sink component is used to 'absorb' the preview frames. It is necessary for the camera to produce preview frames even if not required for display, as they are used for calculating exposure and white balance settings.

Generally bcm2835-v42l behaves incorrectly in no-preview state. It connects preview output port of camera component to NULL value. But it should redirect it to NULL_SINK component. There is no such component in current implementation in use.

I've prepared patch to this module which solves AWB and memory problems.

Regards,
Piotr

[Crymes]

Hi, I want to stream video from the RasPI Camera to an other computer. The project is written in C/C++.
Can anyone suggest a book or a tutorial how to use V4L2 to capture h.264 frames/video ?
All tutorials I found only describe how to capture images.

[6by9]

Hello All,

Racer asked me for some help and we investigated this problem together. The result is issue I've raised: https://github.com/raspberrypi/linux/issues/1196

We tried to use suggested solutions:
1) call ctrl_set_awb_mode from start_streaming doesn't help at all
2) bm2835_mmal_set_all_camera_controls doesn't help also
....

Sorry, I never replied here as the discussion was on the issue raised on github instead.
This was resolved at the end of November 2015.

[6by9]

Hi, I want to stream video from the RasPI Camera to an other computer. The project is written in C/C++.
Can anyone suggest a book or a tutorial how to use V4L2 to capture h.264 frames/video ?
All tutorials I found only describe how to capture images.

From V4L2's point of view, there is no difference between capturing images and H264 frames - they're just different pixel formats to it, and it'll pass buffers of any sort around.

The simplest tool for V4L2 is v4l2-ctl (source at http://git.linuxtv.org/v4l-utils.git/tr ... s/v4l2-ctl)

Code: Select all

v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=4
v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.264

will stream 100 frames of H264 elementary stream to somefile.h264.

For actual streaming, you may be better off using something like GStreamer which will use V4L2 to get source images, and then has RTP and other stream parsing structures. One recent thread on the topic is viewtopic.php?f=98&t=142592&p=942381#p946157 where the poster is trying to stream the two cameras from a Compute Module as an RTP stream. That doesn't fit quite what you're doing, but may give you pointers for search terms to look for.

[racer993]

I recently tried different methods for streaming low latency video over a lan.

The best results I got with

Stream from the Raspberry Pi

raspivid -n -t 0 -w 1280 -h 720 -vf -ih -fps 30 -o - | nc -k -l 8554


View with mplayer on a linux pc

nc 192.168.1.144 8554 | mplayer -fps 30 -cache 1024 -


Maybe it is possible to substitute raspivid with v4l2 (not sure what the syntax should look like)

@6by9 In december, after the AWB was fixed, I encountered an issue with the v4l2 driver. Somehow the "brightness or contrast" of the image depends on the object in the image. I discovered it in a series of pictures that I took. One picture had a large dark object in the center and a in second picture the dark object was replaced by a white object. The strange thing was that the background changed in "brightness or contrast". Which seemed to depend on the object in the center of the picture. At the time I didn't have time to investigate or report it here. But what I did do, was taking the same pictures with raspistill and these pictures were perfectly alright. Unfortunately I didn't keep the pictures to demonstrate and I have not yet been able to reproduce the issue.

Despite all controls being set to manual in the v4l2 driver something still seemed to be in "auto mode" (brightness? contrast?).

I know it sounds vague, but I don't have the time now to investigate / reproduce, I just hope it might be ringing a bell. Otherwise, if I find the time to conduct some tests I'll post the results here.

[Crymes]

I have studied the V4L2 specification and have got a few questions left:
- Is it right that the fps of my video capture are only controlled by queuing the buffers via ioctl to the kernel ?

- For example I want to capture 10 fps. In this case more than 1 buffer wouldn't give me any advantage because as soon as I enqueue a buffer it gets filled up with an image and has to wait for dequeuing which gives me an older image as if I would enqueue --> wait a little time ( less than 1/30 second ? ) --> dequeue --> wait 1/10s --> enqueue ... . Am I right ?

[6by9]

I have studied the V4L2 specification and have got a few questions left:
- Is it right that the fps of my video capture are only controlled by queuing the buffers via ioctl to the kernel ?

- For example I want to capture 10 fps. In this case more than 1 buffer wouldn't give me any advantage because as soon as I enqueue a buffer it gets filled up with an image and has to wait for dequeuing which gives me an older image as if I would enqueue --> wait a little time ( less than 1/30 second ? ) --> dequeue --> wait 1/10s --> enqueue ... . Am I right ?

Not really.

For any raw pixel format (YUV or RGB) then you would be approximately right, but not reliably.
For MJPEG or H264 you'd get the GPU filling up an internal FIFO, and then start causing frames to be dropped.
For JPEG currently it'll misbehave a little bit as a new capture will be taken immediately the last one completed, but can't be delivered until a buffer has been delivered.

If you want reliable frame rate control, then you're looking for VIDIOC_S_PARM where you can specify the timeperframe (reciprocal of frame rate). You can then provide the buffers as fast as possible, and the sensor/GPU will fill them at the requested rate.
If using v4l2-ctl, then "-p <fps>" is what you're after.

[Crymes]

Thank you for your reply.
The program runs fine if I set up the mmap mode with only one buffer. However, the v4l2-requestbuffers structure contains the value 3 for 'count' after the ioctl 'VIDIOC_REQBUFS' function.
What happens with the other two buffers which I do not enqueue ?
Is it safe to run v4l2 this way ?
What would you recommend on how many buffers do I need ? ( I think two would be perfect but I also saw 4 in an example or should I take the three which the driver allocates automatically ?)

I mainly want to stream video in 720p with 20-30 fps if that matters.

[6by9]

Thank you for your reply.
The program runs fine if I set up the mmap mode with only one buffer. However, the v4l2-requestbuffers structure contains the value 3 for 'count' after the ioctl 'VIDIOC_REQBUFS' function.
What happens with the other two buffers which I do not enqueue ?

They get allocated and then sit there unused and unloved, until they get released.

Is it safe to run v4l2 this way ?

Safe in this case, but suboptimal.
On some devices they will require a new buffer to be queued before it can complete and return the current one (otherwise it has to stop the sensor). Compare it to double-buffering in graphics rendering. In that case your system would totally stall.

What would you recommend on how many buffers do I need ? ( I think two would be perfect but I also saw 4 in an example or should I take the three which the driver allocates automatically ?)

I mainly want to stream video in 720p with 20-30 fps if that matters.

The driver gets to specify a minimum number of buffers that it views as required. This driver sets that to 3 (https://github.com/raspberrypi/linux/bl ... era.c#L261): one buffer being filled by the camera, one being displayed, and one in flight.

Assuming you're selecting H264 natively, then you're actually fighting the fact that there is a 2MB FIFO within the codec, so pulling frames out slowly is not going to have a clean effect on framerate. Personally I'd stick with 3 buffers allocated/queued to allow free circulation of data - stalling any real-time pipe by starving it of resources is generally a bad plan.

[sparkie777]

View with mplayer on a linux pc

nc 192.168.1.144 8554 | mplayer -fps 30 -cache 1024 -

no need for 'nc' there. Mplayer can read the stream directly. As a welcome side effect this avoids fiddling with mplayer -cache size argument.

My favorite:

camera server:

Code: Select all

raspivid -v -n -fps 30 -ih -t 0 -o - | nc -k -l -p 8554

client viewer:

Code: Select all

mplayer -fps 30 -demuxer h264es ffmpeg://tcp://192.168.1.144:8554

for real 'low latency' you may want to give 'mplayer' a higher framerate than on the server side. Say '-fps 100' in this example. This reduces buffering on the receive side to a minimum but of course introduces judder.

[Luckk93]

Hello, I'm having the same problem as racer993, I can't remove auto white balance, and if I pick the frames from the v4l2 buffer when overlay is active the frames are completely black.
Exist any way to remove (also completely) the auto white balance setting, so I can at least do a manual white balance on my program?

[6by9]

Hello, I'm having the same problem as racer993, I can't remove auto white balance, and if I pick the frames from the v4l2 buffer when overlay is active the frames are completely black.
Exist any way to remove (also completely) the auto white balance setting, so I can at least do a manual white balance on my program?

I had tried it using QV4L2 a month or so back - https://github.com/raspberrypi/linux/issues/1196
Found an issue and fixed it. Assuming R4C3R on Github is the same person as racer993 here, then he confirmed it worked.
If you're still seeing issues then please ensure your firmware is more recent than December 2015 ("vcgencmd version").

[Luckk93]

Hello, I'm having the same problem as racer993, I can't remove auto white balance, and if I pick the frames from the v4l2 buffer when overlay is active the frames are completely black.
Exist any way to remove (also completely) the auto white balance setting, so I can at least do a manual white balance on my program?

I had tried it using QV4L2 a month or so back - https://github.com/raspberrypi/linux/issues/1196
Found an issue and fixed it. Assuming R4C3R on Github is the same person as racer993 here, then he confirmed it worked.
If you're still seeing issues then please ensure your firmware is more recent than December 2015 ("vcgencmd version").

yes, sorry you are right, was that the problem,
I was sure to have done an upgrade in february, but actually was not so.
Thank you.

[Crymes]

Does anyone know how I can control the h.264 Codec data in V4L2?

I have got the following source of my sample program:

Code: Select all

#include <iostream>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <sys/mman.h>	//mmap
#include <fcntl.h>
#include <unistd.h>
#include <linux/videodev2.h>

using namespace std;

#define numbuffers 3

struct picturebuffer
{
	void *startadress;
	size_t length;
};

//array in which the buffer pointer are beeing stored
picturebuffer pb[numbuffers];

int main()
{
	//open camera
	int fd;
	fd = open("/dev/video0", O_RDWR);
	if(fd < 0)
	{
		cout << "error during opening the camera device!";
		cout.flush();
	}
	cout << "camera opened";

	//read capabilities
	struct v4l2_capability caps;
	if(ioctl(fd, VIDIOC_QUERYCAP, &caps) < 0)
	{
		cout << "error while reading the capabilities!";
		cout.flush();
	}
	cout << "Capabilities " << caps.capabilities << endl;
	//ToDo: check for required capabilities

	//set image data
	struct v4l2_format format;
	format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	format.fmt.pix.pixelformat = V4L2_PIX_FMT_H264;
	format.fmt.pix.width = 1920;
	format.fmt.pix.height = 1080;
	if(ioctl(fd, VIDIOC_S_FMT, &format) < 0)
	{
		cout << "error in the image format";
	}
	cout << "Bilddaten eingestellt" << endl;
	//Todo: check if width and height fit together (VIDIOC_ENUM_FRAMESIZES)

	//set extended Controls
	struct v4l2_ext_controls ecs1;
	struct v4l2_ext_control ec1;
	memset(&ecs1, 0, sizeof(ecs1));
	memset(&ec1, 0, sizeof(ec1));
	ec1.id = V4L2_CID_MPEG_VIDEO_BITRATE_MODE;
	ec1.value = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR;
	ec1.size = 0;
	ecs1.controls = &ec1;
	ecs1.count = 1;
	ecs1.ctrl_class = V4L2_CTRL_CLASS_MPEG;
	if(ioctl(fd, VIDIOC_S_EXT_CTRLS, &ecs1) < 0)
	{
		cout << "error in extended controls bitrate mode";
		cout.flush();
	}



	struct v4l2_ext_controls ecs;
	struct v4l2_ext_control ec;
	memset(&ecs, 0, sizeof(ecs));
	memset(&ec, 0, sizeof(ec));
	ec.id = V4L2_CID_MPEG_VIDEO_BITRATE_PEAK;
	ec.value = 50000;
	ec.size = 0;
	ecs.controls = &ec;
	ecs.count = 1;
	ecs.ctrl_class = V4L2_CTRL_CLASS_MPEG;
	if(ioctl(fd, VIDIOC_S_EXT_CTRLS, &ecs) < 0)
	{
		cout << "error in extended controls bitrate";
		cout.flush();
	}

	//allocate buffer in the kernel
	struct v4l2_requestbuffers req;
	req.count = numbuffers;
	req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	req.memory = V4L2_MEMORY_MMAP;
	if(ioctl(fd, VIDIOC_REQBUFS, &req) < 0)
	{
		cout << "errro while allocating buffer";
		cout.flush();
	}
	cout << "number of buffers: " << req.count << endl;
	cout.flush();

	//map buffers into userspace
	for(int i=0; i<numbuffers; i++)
	{
		struct v4l2_buffer bufferinfo;
		memset(&bufferinfo, 0, sizeof(bufferinfo));
		bufferinfo.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
		bufferinfo.memory = V4L2_MEMORY_MMAP;
		bufferinfo.index = i;
		if(ioctl(fd, VIDIOC_QUERYBUF, &bufferinfo) < 0)
		{
			cout << "error while quering bufferinfo";
			cout.flush();
		}
		pb[i].startadress = mmap(NULL, bufferinfo.length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, bufferinfo.m.offset);
		pb[i].length = bufferinfo.length;
		if(pb[i].startadress == MAP_FAILED)
		{
			cout << "error during mmap" << endl;
		}
		memset(pb[i].startadress, 0, bufferinfo.length);
		cout << "size of buffer: " << bufferinfo.length << endl;
	}
	cout << "buffers mapped into userspace" << endl;
	cout.flush();

	//queue in the buffers
	for(int i=0; i<numbuffers; i++)
	{
		struct v4l2_buffer bufferinfo;
		memset(&bufferinfo, 0, sizeof(bufferinfo));
		bufferinfo.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
		bufferinfo.memory = V4L2_MEMORY_MMAP;
		bufferinfo.index = i;
		if(ioctl(fd, VIDIOC_QBUF, &bufferinfo) < 0)
		{
			cout << "error while queueing the buffers in" << endl;
		}
	}

	//since that point the driver starts capturing the pics
	int type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	if(ioctl(fd, VIDIOC_STREAMON, &type) < 0)
	{
		cout << "error while starting the stream" << endl;
	}

	int file;
	if((file = open("/home/pi/image.h264", O_WRONLY | O_CREAT, 0660)) < 0)
	{
		cout << "error while writing the file";
	}

	//loop for managing the pics
	for(int i=0; i<100; i++)
	{
		struct v4l2_buffer bufferinfo;
		memset(&bufferinfo, 0, sizeof(bufferinfo));
		bufferinfo.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
		bufferinfo.memory = V4L2_MEMORY_MMAP;
		if(ioctl(fd, VIDIOC_DQBUF, &bufferinfo) < 0)
		{
			cout << "error while getting the buffer!" << endl;
		}

		//do anything with the pic
		char buf[pb[bufferinfo.index].length];
		memcpy(&buf, pb[bufferinfo.index].startadress, pb[bufferinfo.index].length);
		cout << bufferinfo.index << endl;
		cout.flush();

		//write picture into the file
		write(file, pb[bufferinfo.index].startadress, pb[bufferinfo.index].length);

		if(ioctl(fd, VIDIOC_QBUF, &bufferinfo) < 0)
		{
			cout << "error while enqueuing the buffer" << endl;
		}
	}
	close(file);
	if(ioctl(fd, VIDIOC_STREAMOFF, &type) < 0)
	{
		cout << "error while stopping the stream" << endl;
	}

	//clean up
	for(int i=0; i<numbuffers; i++)
	{
		if(munmap(pb[i].startadress, pb[i].length) < 0)
		{
			cout << "error during unmap";
		}
	}

	//close camera file
	close(fd);
	cout << "!!!Hello World!!!" << endl;
	cout.flush();
	return 0;
}

It is all about the two blocks after the comment "//set extended controls" (I think I have to do it via the extended controls in V4L2).
Actually my ioctl fails and the error gets printed but last week when I tested the program it always produces a output file with the same suze no matter what bitrate I set.

- Is it right to use the extended controls api ?
- Does anyone know what I did wrong ?

The c4l2-ctl --all output prints the following lines:

Codec Controls

video_bitrate_mode (menu) : min=0 max=1 default=0 value=0 flags=update
video_bitrate (int) : min=25000 max=25000000 step=25000 default=10000000 value=10000000
repeat_sequence_header (bool) : default=0 value=0
h264_i_frame_period (int) : min=0 max=2147483647 step=1 default=60 value=60
h264_level (menu) : min=0 max=11 default=11 value=11
h264_profile (menu) : min=0 max=4 default=4 value=4

That are the parameters which I can change, right?

[racer993]

Hello, I'm having the same problem as racer993, I can't remove auto white balance, and if I pick the frames from the v4l2 buffer when overlay is active the frames are completely black.
Exist any way to remove (also completely) the auto white balance setting, so I can at least do a manual white balance on my program?

I had tried it using QV4L2 a month or so back - https://github.com/raspberrypi/linux/issues/1196
Found an issue and fixed it. Assuming R4C3R on Github is the same person as racer993 here, then he confirmed it worked.
If you're still seeing issues then please ensure your firmware is more recent than December 2015 ("vcgencmd version").

I indeed confirmed that the AWB issue has been fixed and the official v4l2 is almost perfect thanks to your efforts. However the latest issue I mentioned above is a different issue. Which I have not been able to demonstrate yet, I guess the issues become more subtle and harder to detect as the driver matures. The brightness seemed to fluctuate with color of the object in the picture, despite everything being in manual mode.

For my purpose I need full manual control, any automatic setting impacts negatively on the results that I get. User Vdl fixed it for me and recompiled a modified driver in December shortly after the AWB fix, but I'm not a programmer and not sure how he fixed it. I would like to get the fix as well in the official firmware.

[6by9]

Does anyone know how I can control the h.264 Codec data in V4L2?
<snip>
It is all about the two blocks after the comment "//set extended controls" (I think I have to do it via the extended controls in V4L2).
Actually my ioctl fails and the error gets printed but last week when I tested the program it always produces a output file with the same suze no matter what bitrate I set.

- Is it right to use the extended controls api ?
- Does anyone know what I did wrong ?

The c4l2-ctl --all output prints the following lines:

Codec Controls

video_bitrate_mode (menu) : min=0 max=1 default=0 value=0 flags=update
video_bitrate (int) : min=25000 max=25000000 step=25000 default=10000000 value=10000000
repeat_sequence_header (bool) : default=0 value=0
h264_i_frame_period (int) : min=0 max=2147483647 step=1 default=60 value=60
h264_level (menu) : min=0 max=11 default=11 value=11
h264_profile (menu) : min=0 max=4 default=4 value=4

That are the parameters which I can change, right?

I think you've just chosen the wrong IOCTL. Use VIDIOC_S_CTRL instead of VIDIOC_S_EXT_CTRLS. All those parameters that you've listed should be available via VIDIOC_S_CTRL.

I haven't checked the history, but I believe S_EXT_CTRLS was added relatively recently to try and support the Android Camera2 API where it wants sets of parameters to be applied atomically. S_CTRL didn't allow this hence the new command. However it looks like no upstreamed implementation actually supports it

[fnoop]

Hi, I can't seem to get h264 out of v4l2src in gstreamer, it hangs with a kernel entry:
May 13 07:52:33 raspberrypi kernel: [ 263.336191] bcm2835-v4l2: error 0 waiting for frame completion

x-raw and image/jpeg v4l2src pipelines work just fine, including passing x-raw through to omxh264enc and successfully getting h264 out the other end. So it's specific to v4l2 and h264, which seems to be a bit of a problem as it's the most efficient way to stream data (at least using streamer).

Taking a quite search around it looks like quite a few other people have hit the same problem, and there is an open github issue:
https://github.com/raspberrypi/linux/issues/849

Any suggestions for workarounds, settings or debugging, or is this just some low level bug that we have to wait until someone has time to look at and fix?

[6by9]

Hi, I can't seem to get h264 out of v4l2src in gstreamer, it hangs with a kernel entry:
May 13 07:52:33 raspberrypi kernel: [ 263.336191] bcm2835-v4l2: error 0 waiting for frame completion

x-raw and image/jpeg v4l2src pipelines work just fine, including passing x-raw through to omxh264enc and successfully getting h264 out the other end. So it's specific to v4l2 and h264, which seems to be a bit of a problem as it's the most efficient way to stream data (at least using streamer).

Taking a quite search around it looks like quite a few other people have hit the same problem, and there is an open github issue:
https://github.com/raspberrypi/linux/issues/849

Any suggestions for workarounds, settings or debugging, or is this just some low level bug that we have to wait until someone has time to look at and fix?

I haven't investigated this one at all.
H264 from V4L2 is a pretty rare thing - the Logitech C920 was the only thing I knew of that did it, so it may well be a GStreamer bug again. v4l2-ctl streams quite happily with H264, so I know there's nothing fundamentally wrong with the driver.

edit Done some debugging - see the github issue.

[j79]

Hi,

I try to take a picture with the original RPI camera using this two lines :

Code: Select all

v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=./testaujourdhui.jpg

But the first command return

VIDIOC_ENUM_FMT: failed: Invalid argument

If I try without

pixelformat=3

everything ok

So my first question is about why this argument is refused and the second is about the use of pixelformat. I've made some searches about pixelformat and its different value but I do not find my answers.

Thanks for your help,
Jérôme

[6by9]

I try to take a picture with the original RPI camera using this two lines :

Code: Select all

v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=./testaujourdhui.jpg

But the first command return

VIDIOC_ENUM_FMT: failed: Invalid argument

If I try without

pixelformat=3

everything ok

So my first question is about why this argument is refused and the second is about the use of pixelformat. I've made some searches about pixelformat and its different value but I do not find my answers.

There are 14 different image formats that the camera can produce. If you run "v4l2-ctl --list-formats-ext" it should list them all out. pixelformat is specifying which of those 14 modes to use, and also show the resolutions that each can support.
Why it is rejecting pixelformat=3 is an interesting question. Do you have a USB webcam plugged in as well, as v4l2-ctl will use /dev/video0 by default, and that would likely be that webcam. Otherwise the output from "v4l2-ctl --list-formats-ext" would be very helpful to see what it thinks it is asking for.

[kehall]

Hi,

Just been pulling hair out a little bit...

Using standard Motion with the /dev/video0 device and width/height of 640x480, autoexposure works perfectly.

Changing only the width/height to 1024x768 or 1280x1024, it simply does nothing at all - exposure (and WB seemingly) stays as it was when the device was first opened... This obviously causes problems with exposure and white balance when motion is started at bright or dark times resulting in underexposure or overexposure and WB issues (most often green tint, please see attachment).

Device Opened when dark

WBandOverexposed.jpg (39.48 KiB) Viewed 7674 times

Device Re-opened

AfterDeviceReopened.jpg (47.71 KiB) Viewed 7674 times

Happy to do more tests but would love this issue to be resolved in some way other than manually having to restart Motion every time there is a need for significant exposure change.

Kind regards,

Keith.

[sparkie777]

Using standard Motion with the /dev/video0 device and width/height of 640x480, autoexposure works perfectly.

very interesting to hear that standard Motion with v4l2-auto_brightness works at all for any resolution.

I tested with 1920x1080 only yet and as you say auto_brightness of v4l2 appears to work only at the time of device open. BTW: the Motion-'auto_brightness' internal option of Motion does not work either.

as a workaroud I restart Motion every other 10min per cron:

Code: Select all

0-59/10 * * * * root pgrep motion > /dev/null && systemctl restart motion

This at least helps to get around the worst effects of current static 'one-shot-brightness-control' implementation of v4l2.

Does Motion open the v4l2 device with wrong options to cause this problem?

I hope I can toss this ugly workaround one day...

cheers
sparkie777

[6by9]

Using standard Motion with the /dev/video0 device and width/height of 640x480, autoexposure works perfectly.

very interesting to hear that standard Motion with v4l2-auto_brightness works at all for any resolution.

I tested with 1920x1080 only yet and as you say auto_brightness of v4l2 appears to work only at the time of device open. BTW: the Motion-'auto_brightness' internal option of Motion does not work either.

It's the longstanding issue that stills mode locks AE and AWB. V4L2 takes anything above 1280x720 (in either direction) to be a still capture.
Set the module parameters max_video_width and max_video_height to 2592 and 1944 respectively when loading bcm2835-v4l2, and it'll treat all captures as video mode.

[sparkie777]

Set the module parameters max_video_width and max_video_height to 2592 and 1944 respectively when loading bcm2835-v4l2, and it'll treat all captures as video mode.

Great! I thought I once already tested this (but with no success) after reading this thread:

Raspberry Pi • View topic - Captured frames get progressively darker

but maybe I'm wrong and/or my system was outdated to that time. I now reconfigured three RPIs as follows:

Code: Select all

modprobe bcm2835-v4l2 max_video_width=2592 max_video_height=1944
more /sys/module/bcm2835_v4l2/parameters/*
::::::::::::::
/sys/module/bcm2835_v4l2/parameters/max_video_height
::::::::::::::
1944
::::::::::::::
/sys/module/bcm2835_v4l2/parameters/max_video_width
::::::::::::::
2592

/var/log/messages:
Jun  2 13:18:46 rpi-0 kernel: [1741556.083639] bcm2835-v4l2: scene mode selected 0, was 0
Jun  2 13:18:46 rpi-0 kernel: [1741556.084128] bcm2835-v4l2: V4L2 device registered as video0 - stills mode > 2592x1944
Jun  2 13:18:46 rpi-0 kernel: [1741556.093044] bcm2835-v4l2: Broadcom 2835 MMAL video capture ver 0.0.2 loaded.

I will report you the results (til tomorrow)

cheers
sparkie777