jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23698
Joined: Sat Jul 30, 2011 7:41 pm

Re: Official V4L2 driver

Sat Dec 28, 2013 7:50 pm

OK, thanks for the clarification. For historical purposes, its worth noting that raspistill and raspivid were only ever intended as demo applications from which others could write their own stuff (and someone did write a V4L2 userland driver).

It was always intended that a Foundation V4L2 would be released, but its actually taken much longer than expected.

Also worth noting that I think that there will always be advantages to the raspistill/vid code in certain areas, as there are with the V4L2 in others (and that the V4L2 driver uses the same basic interface underneath)

I doubt anyone has had a chance to look at the issue reported yet. It being Xmas and everyone's on holiday. 6x9 will be back on the 2nd I think, and he's the expert, although may be looking in from time to time.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7312
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Sat Dec 28, 2013 8:15 pm

jamesh wrote:Also worth noting that I think that there will always be advantages to the raspistill/vid code in certain areas, as there are with the V4L2 in others (and that the V4L2 driver uses the same basic interface underneath)
Indeed. These apps allow for doing anything that we can do on the GPU, whereas the V4L2 driver has to fit within what V4L2 can do.
jamesh wrote:I doubt anyone has had a chance to look at the issue reported yet. It being Xmas and everyone's on holiday. 6x9 will be back on the 2nd I think, and he's the expert, although may be looking in from time to time.
I'm lurking at the moment, enjoying far too much alcohol, and getting stuck in Christmas traffic :(
New Year's Eve is probably the soonest I'd have a chance to have a look, and whilst I'm back in the office on the 2nd, this driver is a backburner project again for me. I only got the MJPEG support (I thought) done as there was little point in starting something new on my last day before the break. I need to enable a load of extra logging to work out why you're seeing that timeout - it was working for me from v4l2-ctl.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hawk_eye
Posts: 1
Joined: Sun Dec 29, 2013 7:07 pm

Re: Official V4L2 driver

Sun Dec 29, 2013 7:20 pm

Hi everyone,

I'm having a problem using this driver in gstreamer as a source. It works fine when I take JPEG stills with:
v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=somefile.jpg
...or capture video to a file.

HOWEVER, doesn't work when I use the following pipeline (posted by towolf) or any other of my own in g_stremer:
(driver is loaded)
towolf wrote::
Sender:

Code: Select all

gst-launch-1.0 -e v4l2src do-timestamp=true ! video/x-h264,width=640,height=480,framerate=30/1 ! h264parse ! rtph264pay config-interval=1 ! gdppay ! udpsink host=192.168.178.20 port=5000
It shows the following error when I use above pipeline:

Code: Select all

pi@doc ~ $ gst-launch-1.0 -v v4l2src do-timestamp=true ! video/x-h264,width=640,height=480,framerate=30/1 ! h264parse ! rtph264pay config-interval=1 ! gdppay ! udpsink host=192.168.1.105 port=5000
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-h264, framerate=(fraction)30/1, width=(int)640, height=(int)480, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, framerate=(fraction)30/1, width=(int)640, height=(int)480, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, framerate=(fraction)30/1, width=(int)640, height=(int)480, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, framerate=(fraction)30/1, width=(int)640, height=(int)480, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2812): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 51023465 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Another error with a test pipeline:

Code: Select all

pi@Delicious ~ $ gst-launch-1.0 'v4l2src ! xvimagesink'
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Could not initialise Xv output
Additional debug info:
xvimagesink.c(1291): gst_xvimagesink_xcontext_get (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
Could not open display
Setting pipeline to NULL ...
Freeing pipeline ...
What could be causing the problem? Any solution?

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Mon Dec 30, 2013 12:49 pm

6by9 wrote:
shuckle wrote:Does not seem to work with motion:
$ motion
[...]
[1] frames=4
[1] 0 length=307200
[1] 1 length=307200
[1] 2 length=307200
[1] 3 length=307200
[1] Using V4L2
[1] Resizing pre_capture buffer to 1 items
Segmentation fault


If I first use working palette YU12 (motion 8), I get different error:
$ motion
[0] Processing thread 0 - config file motion.conf
[0] Motion 3.2.12 Started
[0] ffmpeg LIBAVCODEC_BUILD 3482368 LIBAVFORMAT_BUILD 3478785
[...]
[1] Test palette MJPG (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Timer expired
[1] Config palette index 2 (MJPG) doesn't work.
[1] Supported palettes:
[1] 0: YU12 (4:2:0, packed YUV)
[...]
[1] 5: MJPG (MJPEG)
[1] Selected palette YU12

But it does not work and I soon get the same timer expired.
When trying to get motion to work with JPEG I was also seeing seg faults. If you run it via gdb, I was seeing it somewhere in a JPEG DCT function, which sort of implies that something is wrong with the JPEG decode. When selecting JPEG though, I was also getting very small buffers being passed out, but I hadn't had a chance to work out why. Trying to rebuild Motion to debug further was how I screwed up ffmpeg on my Pi :(

Any chance you could run it via gdb to see where it is blowing up please? ("gdb motion", "r<return>", and then probably "c<return>" a couple of times until it hits the seg fault). Whilst we'll do our best supporting this, it is still a spare-time project, so the more info you can feed us the better.

Second time through appears to be selecting MJPEG first, and then YU12 - that's a bit strange if you are sure you've set your config correctly to request YU12. I know for certain that Motion with YU12 was working, and the code to support that hasn't changed recently.
The driver isn't as polished as it could be, and if we get things too wrong then it can get the GPU into a weird state that the error handling code can't recover from. Can you try YU12 from a clean boot? It's possible that the selection of MJPEG and got us into a weird state. Thanks.

BTW Motion itself seems to have been updated since the version that is available via apt-get. YU12 is now format 17, not 8! The niggle is that you need to have all the relevant source files around for ffmpeg and the like to be able to rebuild it.
Not sure if this helps much:

Code: Select all

$ gdb motion
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/motion...(no debugging symbols found)...done.
(gdb) r
Starting program: /usr/bin/motion 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0xb53f35e0 in ?? () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
(gdb) c
Continuing.
Cannot access memory at address 0x0

Program received signal SIGILL, Illegal instruction.
0xb53f35e8 in ?? () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
(gdb) c
Continuing.
Cannot access memory at address 0x0
[0] Processing thread 0 - config file /home/juha/ohjelmat/webkamera/motion.conf
[0] Motion 3.2.12 Started
[0] ffmpeg LIBAVCODEC_BUILD 3482368 LIBAVFORMAT_BUILD 3478785
[0] Thread 1 is from /home/juha/ohjelmat/webkamera/motion.conf
[New Thread 0xb509e440 (LWP 2561)]
[1] Thread 1 started
[1] cap.driver: "bm2835 mmal"
[1] cap.card: "mmal service 15.1"
[1] cap.bus_info: "platform:bcm2835-v4l2"
[1] cap.capabilities=0x85000005
[1] - VIDEO_CAPTURE
[1] - VIDEO_OVERLAY
[1] - READWRITE
[1] - STREAMING
[New Thread 0xb489e440 (LWP 2562)]
[1] Test palette MJPG (1920x1088)
[0] motion-httpd/3.2.12 running, accepting connections
[0] motion-httpd: waiting for data on port TCP 8080
[1] Using palette MJPG (1920x1088) bytesperlines 1920 sizeimage 2088960 colorspace 00000008
[1] found control 0x00980900, "Brightness", range 0,100 
[1] 	"Brightness", default 50, current 50
[1] found control 0x00980901, "Contrast", range -100,100 
[1] 	"Contrast", default 0, current 0
[1] found control 0x00980902, "Saturation", range -100,100 
[1] 	"Saturation", default 0, current 0
[1] mmap information:
[1] frames=4
[1] 0 length=2088960
[1] 1 length=2088960
[1] 2 length=2088960
[1] 3 length=2088960
[1] Using V4L2
[1] Resizing pre_capture buffer to 1 items

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb509e440 (LWP 2561)]
0xb6f92958 in jpeg_idct_16x16 () from /usr/lib/arm-linux-gnueabihf/libjpeg.so.8
(gdb) c
Continuing.
[Thread 0xb489e440 (LWP 2562) exited]
[Thread 0xb509e440 (LWP 2561) exited]

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
When I first successfully run with motion palette 8 and then start again with 2, I get the timeout.

Code: Select all

$ gdb motion
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/motion...(no debugging symbols found)...done.
(gdb) r
Starting program: /usr/bin/motion 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0xb53f35e0 in ?? () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
(gdb) c
Continuing.
Cannot access memory at address 0x0

Program received signal SIGILL, Illegal instruction.
0xb53f35e8 in ?? () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
(gdb) c
Continuing.
Cannot access memory at address 0x0
[0] Processing thread 0 - config file /home/juha/ohjelmat/webkamera/motion.conf
[0] Motion 3.2.12 Started
[0] ffmpeg LIBAVCODEC_BUILD 3482368 LIBAVFORMAT_BUILD 3478785
[0] Thread 1 is from /home/juha/ohjelmat/webkamera/motion.conf
[New Thread 0xb509e440 (LWP 2732)]
[1] Thread 1 started
[1] cap.driver: "bm2835 mmal"
[1] cap.card: "mmal service 15.1"
[1] cap.bus_info: "platform:bcm2835-v4l2"
[1] cap.capabilities=0x85000005
[1] - VIDEO_CAPTURE
[1] - VIDEO_OVERLAY
[1] - READWRITE
[1] - STREAMING
[New Thread 0xb489e440 (LWP 2733)]
[1] Test palette MJPG (640x480)
[0] motion-httpd/3.2.12 running, accepting connections
[0] motion-httpd: waiting for data on port TCP 8080
[1] Error setting pixel format VIDIOC_S_FMT: Timer expired
[1] Config palette index 2 (MJPG) doesn't work.
[1] Supported palettes:
[1] 0: YU12 (4:2:0, packed YUV)
[1] 1: YUYV (4:2:2, packed, YUYV)
[1] 2: BGR3 (RGB24 (BE))
[1] 3: JPEG (JPEG)
[1] 4: H264 (H264)
[1] 5: MJPG (MJPEG)
[1] Selected palette YU12
[1] Test palette YU12 (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Timer expired
[1] VIDIOC_TRY_FMT failed for format YU12: Timer expired
[1] Unable to find a compatible palette format.
[1] ioctl (VIDIOCGCAP): Inappropriate ioctl for device
[1] Could not fetch initial image from camera
[1] Motion continues using width and height from config file(s)
[1] Resizing pre_capture buffer to 1 items
[1] Started stream webcam server in port 8081
[1] Retrying until successful connection with camera
[1] cap.driver: "bm2835 mmal"
[1] cap.card: "mmal service 0.-709564288"
[1] cap.bus_info: "platform:bcm2835-v4l2"
[1] cap.capabilities=0x85000005
[1] - VIDEO_CAPTURE
[1] - VIDEO_OVERLAY
[1] - READWRITE
[1] - STREAMING
[1] Test palette MJPG (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Timer expired
[1] Config palette index 2 (MJPG) doesn't work.
[1] Supported palettes:
[1] 0: YU12 (4:2:0, packed YUV)
[1] 1: YUYV (4:2:2, packed, YUYV)
[1] 2: BGR3 (RGB24 (BE))
[1] 3: JPEG (JPEG)
[1] 4: H264 (H264)
[1] 5: MJPG (MJPEG)
[1] Selected palette YU12
[1] Test palette YU12 (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Timer expired
[1] VIDIOC_TRY_FMT failed for format YU12: Timer expired
[1] Unable to find a compatible palette format.
[1] ioctl (VIDIOCGCAP): Inappropriate ioctl for device
[1] Retrying until successful connection with camera
[1] cap.driver: "bm2835 mmal"
[1] cap.card: "mmal service 0.-709564288"
[1] cap.bus_info: "platform:bcm2835-v4l2"
[1] cap.capabilities=0x85000005
[1] - VIDEO_CAPTURE
[1] - VIDEO_OVERLAY
[1] - READWRITE
[1] - STREAMING
[1] Test palette MJPG (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Timer expired
[1] Config palette index 2 (MJPG) doesn't work.
[1] Supported palettes:
[1] 0: YU12 (4:2:0, packed YUV)
[1] 1: YUYV (4:2:2, packed, YUYV)
[1] 2: BGR3 (RGB24 (BE))
[1] 3: JPEG (JPEG)
[1] 4: H264 (H264)
[1] 5: MJPG (MJPEG)
[1] Selected palette YU12
[1] Test palette YU12 (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Timer expired
[1] VIDIOC_TRY_FMT failed for format YU12: Timer expired
[1] Unable to find a compatible palette format.
The timeout then comes from all commands until system is rebooted:
$ v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=5
VIDIOC_S_FMT: failed: Timer expired

and dmesg shows:
[ 1178.770640] bcm2835_v4l2: error 0 waiting for sync completion
[ 1179.770683] bcm2835_v4l2: error 0 waiting for sync completion
[ 1179.770722] bcm2835-v4l2: vidioc_s_fmt_vid_cap: failed to setup mmal components: -62


But it is interesting that I can produce mjpeg file from it and that file works in another linux box with vlc! So maybe this is motion bug after all?

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Mon Dec 30, 2013 3:41 pm

I run into a hard hang issue. After I do these commands, the whole raspberry hangs and only boot will free it.
Seems to happen every time. If I first use pixelformat 4, which is the h264, it works without problems.

Code: Select all

$ v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=5
$ v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.mjpeg
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30 fps
<<<<<<
$ v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.mjpeg
VIDIOC_STREAMON: failed: Operation not permitted
$ v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.mjpeg
VIDIOC_STREAMON: failed: Operation not permitted
$ v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=5
$ v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.mjpeg
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30 fps
<<<<<<
My versions:
Linux raspberry2 3.10.25+ #616 PREEMPT Mon Dec 23 18:13:02 GMT 2013 armv6l GNU/Linux
Dec 23 2013 18:26:32
Copyright (c) 2012 Broadcom
version e6f70fa5f721f7583c8cf863f8bab6c8ac8285bd (clean) (release)

uski
Posts: 13
Joined: Sat Oct 27, 2012 12:41 pm

Re: Official V4L2 driver

Wed Jan 01, 2014 12:18 am

Hi,

For what it's worth, I managed to get MJPEG streaming working at 1280x720 10fps with mjpg_streamer.
I didn't manage to reproduce my earlier issues - not sure if I did something wrong or if there was some kind of other issue.
Now all is working for me.

PS: it's possible that my issues were caused by concurrent access to the video device.
If the driver does not support simultaneous accesses, maybe it would be worth it to deny concurrent access ?

Thanks for this driver ! And happy new year ;)
uski

andlinux
Posts: 18
Joined: Fri Dec 13, 2013 12:08 pm
Location: Belgium

Re: Official V4L2 driver

Wed Jan 01, 2014 3:10 pm

@uski: can you write down what you did to make it work with mjpg_streamer ?
I'm also trying to make this work with mjpg_streamer.
I can make a picture with the v4l2-ctl command and I can start mjpg_streamer but I'm unable to combine those two. :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7312
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Wed Jan 01, 2014 6:01 pm

I've just had quick look at the issue of trying to run a second MJPEG stream failing from v4l2-ctl.
The codec has bit-rotted further than I expected. We have two methods of allocating memory, one good for allocating small chunks, and one better for large chunks. This codec is allocating a "large" lump using the small method, and on the second attempt this alloc is failing.
I'll try out a GPU change to improve this, but it'll need a bit of testing before release (and I don't know when Dom is around again to do a new release). I'll try and sort out my end and push it to Dom.

This is independent of any issues with Motion etc. I've still got to rebuild my image so that ffmpeg works before I can look at that one again.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: Official V4L2 driver

Wed Jan 01, 2014 9:13 pm

Thanks for the good work @6by9.

I wanted to confirm that I was having same problem as @jbeale. modprobe would hang and dmesg would show a "bcm2835_v4l2: error 0 waiting for sync completion" error. Rebooting the device and outputting a single image with raspistill managed to get the driver loaded with modprobe. I'm running the following kernel "Linux raspberrypi 3.10.25+ #616 PREEMPT Mon Dec 23 18:13:02 GMT 2013 armv6l GNU/Linux" and my board has version 2011.12 and my camera is revision 1.3 (might help if anyone looks at the driver code in future).
6by9 wrote:
jbeale wrote:Very interesting. "bcm2835_v4l2: error 0 waiting for sync completion" is what I consistently got (even with reboot) until I power-cycled and then ran raspistill before attempting modprobe bcm2835-v4l2, only after that did it work.
I know there are a few error handling cases that aren't perfect, but I haven't seen that specific one. It is probably due to ffmpeg and avconv using V4L2 in subtly different way to other apps. Not necessarily wrong, just different.
We can investigate, but please remember this is a spare time project for me now, so it may be a little while.

Generally on any failure (particularly the current issue with 'motion' and JPEG that I'm looking at), "rmmod bcm2835-v4l2" does unload the driver cleanly and frees the camera on the GPU. There may be issues that can't be cleared that way, but it works for me. "modprobe bcm2835-v4l2" after to reload the driver, and the pair of commands are quicker than a reboot :)

As a little background, Raspistill/yuv/vid and this driver are all talking to the GPU over the same IPC mechanism (VCHIQ). The V4L2 driver is a different as we can't use our normal userspace MMAL library as the interface, so it's been rerolled in a more direct mode. There are minor differences between the two which have caused a few issues during development, and there are likely to be more lurking.

The V4L2 driver opens the sensor driver when it is loaded to confirm that the sensor is connected. It then powers it all back down again to allow others to access it. When you ask it to stream, it all gets powered back up again and starts grabbing images as requested. Stop streaming and we power it all back down again. Things can change whilst it is all powered down (you could have unplugged the sensor for a start - just don't at risk of blowing up the Pi or sensor!), so the reopen could fail. It's quite possible that the handling for that is missing, but that's only a guess at the moment.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Thu Jan 02, 2014 9:24 am

uski wrote:Hi,

For what it's worth, I managed to get MJPEG streaming working at 1280x720 10fps with mjpg_streamer.
I didn't manage to reproduce my earlier issues - not sure if I did something wrong or if there was some kind of other issue.
Now all is working for me.

uski
Thanks a lot uski! mjpg_streamer indeed seems to work fine. Doing 1280x720 10fps with very little cpu usage is impressive!
Looks like it even can do higher, but it takes so much resources from the other end, that my test laptop can't show the picture...

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23698
Joined: Sat Jul 30, 2011 7:41 pm

Re: Official V4L2 driver

Thu Jan 02, 2014 12:10 pm

6by9 wrote:
jamesh wrote:Since you are here..

Been trying to get raspivid working with mjpg. Am getting output of about the right size, but omxplayer doesnt like it, and vlc just shows the first frame. Have tried MJPG, MJPA, MJPB, all similar results (although VLC did show differently for B, just one frame).

Interestingly, fbi will load the video as as single frame jpg, so there data is definitely in there..

I'm sure you said omxplayer would play the stream back...

James
Odd. Are you just saving the data, no container or anything?
Just try out the V4L2 capture to confirm that it is working first:

Code: Select all

v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=5
v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.mjpg
omxplayer somefile.mjpg
I don't know what omxplayer is using to determine the codec/container type, and of course you have no headers so file extension is probably the key. I think I used .mjpg, and not .mjpeg - not got the work laptop powered up to check, but you've got the email, and my other files are on cam-a-b4 in a place you should be able to find quite easily.
Just tried this and omxplayer doesn't like the stream. Something odd somewhere.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7312
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Thu Jan 02, 2014 2:04 pm

bsdz wrote:I wanted to confirm that I was having same problem as @jbeale. modprobe would hang and dmesg would show a "bcm2835_v4l2: error 0 waiting for sync completion" error. Rebooting the device and outputting a single image with raspistill managed to get the driver loaded with modprobe. I'm running the following kernel "Linux raspberrypi 3.10.25+ #616 PREEMPT Mon Dec 23 18:13:02 GMT 2013 armv6l GNU/Linux" and my board has version 2011.12 and my camera is revision 1.3 (might help if anyone looks at the driver code in future).
I've good news for you - in doing a clean install (having broken ffmpeg) I hit exactly the same problem. A little time with the GPU debugger and we now know what is going on, and I have a fix for this instance. Quite why running raspistill first should solve the issue is a little odd, as that has the potential to hit the same issue.
I'll hold off on a push for now as I'll have a look at the changes needed to the MJPEG codec on memory allocation.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23698
Joined: Sat Jul 30, 2011 7:41 pm

Re: Official V4L2 driver

Thu Jan 02, 2014 3:53 pm

And in other good news it appears that raspivid can now output raw mjpeg as well. Note that to get this to play in things like VLC a container needs to be added (ffmpg or avconv), not sure how that would affect plumbing in to webpages.

I'll release it sometime in the next few days or so. It uses the same firmware as the v4l so no change needed there.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7312
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Thu Jan 02, 2014 5:10 pm

jamesh wrote:And in other good news it appears that raspivid can now output raw mjpeg as well. Note that to get this to play in things like VLC a container needs to be added (ffmpg or avconv), not sure how that would affect plumbing in to webpages.

I'll release it sometime in the next few days or so. It uses the same firmware as the v4l so no change needed there.
Busy day then.
I've just got olifre's issue with "avconv -f video4linux2 -list_formats all -i /dev/video0" sorted, so it is now listing JPEG and MJPEG as supported formats (not H264 for some reason - I guess it wasn't enabled at build time).
I haven't managed to get it to successfully use the stream though - it seems to be the same issue that it can't detect it correctly and therefore objects in the same way that omxplayer and vlc object. I'll get my side tidied up (it's only a couple of lines changed) and pushed, then someone else who knows avconv may be able to get further.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Thu Jan 02, 2014 5:32 pm

Just a data point even before the raw/compressed flag was added I could stream and store H264 with Gstreamer. Not sure why that guy above had a negotiation error, my example was with the newest gst-launch on Arch.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23698
Joined: Sat Jul 30, 2011 7:41 pm

Re: Official V4L2 driver

Thu Jan 02, 2014 5:54 pm

6by9 wrote:
jamesh wrote:And in other good news it appears that raspivid can now output raw mjpeg as well. Note that to get this to play in things like VLC a container needs to be added (ffmpg or avconv), not sure how that would affect plumbing in to webpages.

I'll release it sometime in the next few days or so. It uses the same firmware as the v4l so no change needed there.
Busy day then.
I've just got olifre's issue with "avconv -f video4linux2 -list_formats all -i /dev/video0" sorted, so it is now listing JPEG and MJPEG as supported formats (not H264 for some reason - I guess it wasn't enabled at build time).
I haven't managed to get it to successfully use the stream though - it seems to be the same issue that it can't detect it correctly and therefore objects in the same way that omxplayer and vlc object. I'll get my side tidied up (it's only a couple of lines changed) and pushed, then someone else who knows avconv may be able to get further.
More like a busy Xmas eve...
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7312
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Thu Jan 02, 2014 6:15 pm

towolf wrote:Just a data point even before the raw/compressed flag was added I could stream and store H264 with Gstreamer. Not sure why that guy above had a negotiation error, my example was with the newest gst-launch on Arch.
It seems that avconv is fussy about all the flags being set correctly so that it can compare against its internal lists of codecs and supported formats.
gstreamer may be a little less fussy as your pipeline is explicitly stating the format for each stage.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5331
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Official V4L2 driver

Thu Jan 02, 2014 9:27 pm

There's a few V4L fixes by 6x9, plus a couple of raspicam fixes by jamesh in latest rpi-update firmware.
https://github.com/Hexxeh/rpi-firmware/ ... 676aaee295

olifre
Posts: 3
Joined: Mon Dec 23, 2013 11:49 pm

Re: Official V4L2 driver

Fri Jan 03, 2014 12:22 am

6by9 wrote:I've just got olifre's issue with "avconv -f video4linux2 -list_formats all -i /dev/video0" sorted, so it is now listing JPEG and MJPEG as supported formats (not H264 for some reason - I guess it wasn't enabled at build time).
Thanks a lot for your work, this is a great success!
With this new firmware, I had partial success on Raspbian.

Code: Select all

avserver -f ffserver.conf

Code: Select all

avconv -r 30 -s 640x480 -f video4linux2 -input_format mjpeg -i /dev/video0 -vcodec copy http://0.0.0.0:9999/webcam.ffm
With ffserver.conf at http://pastebin.com/ccPJ332Y I get the following output from avconv:

Code: Select all

avconv version 0.8.6-6:0.8.6-1+rpi1, Copyright (c) 2000-2013 the Libav developers
  built on Mar 31 2013 13:58:10 with gcc 4.6.3
[video4linux2 @ 0x1fa7d20] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2, from '/dev/video0':
  Duration: N/A, start: -140459753.786405, bitrate: N/A
    Stream #0.0: Video: mjpeg, yuvj420p, 640x480 [PAR 1:1 DAR 4:3], -5 kb/s, 30 tbr, 1000k tbn, 30 tbc
Output #0, ffm, to 'http://0.0.0.0:9999/webcam.ffm':
  Metadata:
    encoder         : Lavf53.21.1
    Stream #0.0: Video: mjpeg, yuvj420p, 640x480 [PAR 1:1 DAR 4:3], q=2-31, -5 kb/s, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press ctrl-c to stop encoding
And it keeps running, shows up on the status.html page of avserver, and I can even try to play the stream with ffplay / mplayer on another machine.
I see some seconds of video arriving but played back at very wrong (far too much fps) speed until the stream suddenly dies. The framerate (tbc = 1000k, also shows up on status.html page) looks very wrong, I have no idea where avconv is getting inspiration from to do that.

Maybe this can help somebody with more avserver / avconv experience as a starting point?

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Fri Jan 03, 2014 9:11 am

$ v4l2-ctl -V
Format Video Capture:
Width/Height : 1280/720
Pixel Format : 'MJPG'
Field : None
Bytes per Line: 1280
Size Image : 921600
Colorspace : SRGB

What is the "Field" value? Id it digital zoom? Can it be changed?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7312
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Fri Jan 03, 2014 10:39 am

shuckle wrote:$ v4l2-ctl -V
Format Video Capture:
Width/Height : 1280/720
Pixel Format : 'MJPG'
Field : None
Bytes per Line: 1280
Size Image : 921600
Colorspace : SRGB

What is the "Field" value? Id it digital zoom? Can it be changed?
Field is for interlaced content. See enum v4l2_field in include/uapi/linux/videodev2.h

Code: Select all

enum v4l2_field {
	V4L2_FIELD_ANY           = 0, /* driver can choose from none,
					 top, bottom, interlaced
					 depending on whatever it thinks
					 is approximate ... */
	V4L2_FIELD_NONE          = 1, /* this device has no fields ... */
	V4L2_FIELD_TOP           = 2, /* top field only */
	V4L2_FIELD_BOTTOM        = 3, /* bottom field only */
	V4L2_FIELD_INTERLACED    = 4, /* both fields interlaced */
	V4L2_FIELD_SEQ_TB        = 5, /* both fields sequential into one
					 buffer, top-bottom order */
	V4L2_FIELD_SEQ_BT        = 6, /* same as above + bottom-top order */
	V4L2_FIELD_ALTERNATE     = 7, /* both fields alternating into
					 separate buffers */
	V4L2_FIELD_INTERLACED_TB = 8, /* both fields interlaced, top field
					 first and the top field is
					 transmitted first */
	V4L2_FIELD_INTERLACED_BT = 9, /* both fields interlaced, top field
					 first and the bottom field is
					 transmitted first */
Nothing at all to do with zoom, and no you can't change it (not sure why you'd want to).

Digital zoom is set by defining a cropping rectangle - see http://hverkuil.home.xs4all.nl/spec/media.html#crop and http://hverkuil.home.xs4all.nl/spec/med ... ection-api Neither are implemented yet for this driver though.
(From discussions we've had, I believe this is also the approach needed to get around our restrictions of stride/bufferheight being multiples of 32/16 respectively, but I need to check that one out more fully.)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7312
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Fri Jan 03, 2014 7:05 pm

A quick update for motion and MJPEG.
With the prebuilt copy it fails with a seg fault.
Manually rebuild by (from memory so hopefully not too many errors/omissions):
  • - download source from http://sourceforge.net/projects/motion/files/ for 3.2.12 and extract (for me into ~/motion-3.2.12)
    - sudo apt-get install libv4l-dev libjpeg62-dev libjpeg62
    - sudo cp /usr/include/libv4l1-videodev.h /usr/include/linux/videodev.h
    - cd motion-3.2.12
    - edit motion.h to include <linux/videodev2.h> as well as <linux/videodev.h>
    - ./configure
    - make
    - cp motion-dist.conf motion.conf
    - edit motion.conf to set "v4l2_palette 2"
    - v4l2-ctl -p 2 (sets framerate to 2fps - motion appears to fail to do this and gets swamped)
    - motion -c motion.conf -d 9
and it works with no seg fault, so I guess the libJPEG API has changed or something, and the standard copy hasn't caught up.

Setting the framerate manually is a little quirky. The fact that the function v4l2_set_fps in video2.c is surrounded by #if 0 / #endif tells me that they've had some issue with it! Why the app not dealing with buffers causes the MJPEG encoder to totally stall is another issue.

I'm not sure you're going to get the behaviour you want from this anyway, for two main reasons.
  • - It looks like the framerate parameter in motion.conf is just setting how frequently a frame is passed into the motion detection code. It does not vary the rate that the source is delivering. If you ask the V4L2 driver for an MJPEG stream, it will attempt to maintain the stream at the framerate specified and has a fairly meaty FIFOs to cope with some latency in the downstream handling the frames. If motion is then looking at the buffers "infrequently", it'll be seeing the contents of the FIFO and not just getting a frame at a later point in time. Asking for YU12 frames has much less buffering (it's too expensive in memory terms), so you'll get the frames at closer to realtime. If you want framerate control, then you need to control the V4L2 source as well.
    - The desire with this was to reduce the load on the ARM. I'm not sure you'll achieve this, as it now has to decode the JPEG to do motion detection on, and if it detects motion it then applies a time/datestamp overlay and reencodes it. Surely that has increased the ARM CPU loading, or is there an option to turn this off? Even the stream sent over the HTTP connection has this overlay on, so no gain there either
Motion with JPEG still fails. For some reason the buffers produced are tiny (2-3kB), which considering there should be a full EXIF header and thumbnail in there seems very wrong. It is a little strange that we don't see that anywhere else so I will pursue it, but it will have to wait a bit. Likewise why the MJPEG encoder totally wedges if it runs out of FIFO space would be a useful one to solve, but that codec is not used anywhere else these days, so won't be a priority.

Hopefully this'll make some people happy, and if you can give me runes to go into motion.conf for more funky things, then I'll happily give them a quick whirl to see if we can improve matters further.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Sun Jan 05, 2014 7:48 am

Thanks, 6by9.
I was able to run motion with your instructions and indeed
v4l2-ctl -p 4
was a mandatory setting. But running with MJPG did not help to the CPU bottle neck issue and I was not able to go higher than with the previous version. Limited to 4fps with 1280x720. Even when I turned motion detection off and also disabled the timestamp.
So this test failed and motion did not run any better with direct MJPEG code, which is strange in my opinion.

I can however now run mjpg-streamer with 1280x720 at 10fps with very limited CPU usage.

So, thanks a lot for the hard work, but I think it is not worth you to invest more time to motion.

Alex111
Posts: 32
Joined: Sun Oct 07, 2012 6:56 am

Re: Official V4L2 driver

Sun Jan 05, 2014 12:00 pm

Thanks for the Driver. It works for me well in h264 (piexelformat 4) but I have Problems with mjpeg (pixelformat 5).

I'm trying to read the raw stream directly from the /dev/video0 device. As mentioned this really works perfect with h264, but when using MJPEG the stream stops after about 1 Minute. I also tried different FPS rates from 30, 10, 4. It seems to happen more frequently on a higher rate.
With the stream stops I mean that there is simply no more data available on the stream. A read from /dev/video0 just does not return anymore.

I'm using C# to read the stream but the same happens, when I use this command:
v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=5
v4l2-ctl --stream-mmap=3 --stream-to=somefile.mjpeg
It simply stops after a few seconds without error. Camera led stays red. I can return to shell with ctrl-c...

Also it seems to happen more frequently, when there are much changes on the camera e.g. a camera move

Any suggestions?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23698
Joined: Sat Jul 30, 2011 7:41 pm

Re: Official V4L2 driver

Sun Jan 05, 2014 12:23 pm

We'll have a look Monday. Might be some bugs in there, the codec was pretty old and had to be ported up to run on the Raspi, so there may be problems with it we haven't seen yet.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Return to “Camera board”