silvanmelchior
Posts: 76
Joined: Mon Nov 25, 2013 5:56 pm

RaspiMJPEG

Mon Nov 25, 2013 6:28 pm

Important!!!

This description is outdated. All versions of RaspiMJPEG >= 4.0 use a config-file instead of parameters. A working demonstration can be found here: http://www.raspberrypi.org/phpBB3/viewt ... 91#p468491

Hi

I'd like to share my new application: RaspiMJPEG. A first impression can be gained with my uploaded video: http://www.youtube.com/watch?v=k4CiDx53EJc
I wrote it because I googled a long time but wasn't able to find an application that can stream MJPEG from RPi Cam with many frames per second and record it in Full HD at the same time.

Demonstration

I wrote a web interface to demonstrate all features of RaspiMJPEG, you can find it here: http://www.raspberrypi.org/phpBB3/viewt ... 43&t=63276

Description / Usage

RaspiMJPEG is an OpenMAX-Application based on the mmal-library, which is comparable to RaspiVid. Both applications save the recording formated as H264 into a file. But instead of showing the preview on a screen, RaspiMJPEG streams the preview as MJPEG into a file. The update-rate and the size of the preview are customizable with parameters and independent of the recording, which is always 1080p 30fps. Once started, the application receives commands with a unix-pipe and showes its status on stdout and writes it into a status-file. The program terminates itself after receiving a SIGINT or SIGTERM

Usage as MJPEG-Streamer: raspimjpeg -w 320 -h 240 -d 5 -of /path/to/image.jpg

This command creates the image "path/to/image.jpg" with the size 320x240 px and updates it every time, 5 frames are captured, which gives (30/5=) 6fps. This way used, RaspiMJPEG serves as MJPEG-streamer and the preview can be showed on a website for example. To achieve a high update-rate (e.g. "-d 1"), it is recommended to save the file into the ram and not on the sd-card (e.g. "-of /dev/shm/image.jpg").

Usages as MJPEG-Streamer with capture: raspimjpeg -w 320 -h 240 -d 5 -of /path/to/image.jpg -cf /path/to/pipe -vf /path/to/video.h264

This command does the same as the one above, but it is also listening on the pipe, defined with the parametet -c. If another program writes "ca 1" into the pipe (shell: echo "ca 1" > /path/to/pipe), the application continues with the MJPEG-stream, but starts also a H264-capture 1080p 30fps into the defined file. The capture is stopped with the command "ca 0" via pipe. A new capture overwrites the defined file. To make MJPEG and H264 possible, it is recommended to write the image file only into the ram, as described above.

Usage as MJPEG-streamer with capture and status-output: raspimjpeg -w 320 -h 240 -d 5 -of /path/to/image.jpg -cf /path/to/pipe -vf /path/to/video.h264 -sf /path/to/textfile.txt

Until now, RaspiMJPEG wrote its status into stdout/stderr. With this new command, the status is also written into a textfile (no logging, just the newest status). Possible messages and their meanings are: ready --> MJPEG is streaming, not capturing capture --> MJPEG is streaming and H264 is capturing errror --> An error occured and the application terminated

How to download
I created a git fork from the Pi userland, so use this commands:

Code: Select all

sudo apt-get install cmake
git clone https://github.com/silvanmelchior/userland.git
cd userland
./buildme
The compilation on the Pi itself takes about 20 minutes and afterwards, raspimjpeg is installed in /opt/vc/bin and with

Code: Select all

sudo ln -s /opt/vc/bin/raspimjpeg /usr/bin/raspimjpeg
is it available from the shell without the path.

If you've found a bug or have any questions/comments, feel free to post. And of course do not hesitate do edit my source code for your own projects.

Silvan

Edit: I updated my application, it is now also possible to take images in full resolution and to record timelapses with max. 30fps. Detail in the readme on github.

Edit II: It's now possible to change the camera settings (brightness, white balance, exposure mode, ...) on the fly with a command into the control-pipe. The newest version is available via github, all information are in the readme.
Last edited by silvanmelchior on Thu Oct 23, 2014 5:08 am, edited 8 times in total.

poing
Posts: 1131
Joined: Thu Mar 08, 2012 3:32 pm

Re: RaspiMJPEG

Mon Nov 25, 2013 9:00 pm

It sounds interesting. Would it also be possible to look at the stream and then when sending the signal have raspistill save a high def photo?

silvanmelchior
Posts: 76
Joined: Mon Nov 25, 2013 5:56 pm

Re: RaspiMJPEG

Tue Nov 26, 2013 3:27 pm

Yes of course, if I have some free time I'm going to add this feature and maybe even the possibility to change the camera-settings while streaming

poing
Posts: 1131
Joined: Thu Mar 08, 2012 3:32 pm

Re: RaspiMJPEG

Tue Nov 26, 2013 4:31 pm

Great! That would be *really* useful (for me ;-) )

User avatar
jbeale
Posts: 3469
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RaspiMJPEG

Tue Nov 26, 2013 9:40 pm

This looks quite useful, thank you! I second the request for ability to save a high-res still on the signal. I presume the recommended location of the ever-changing JPEG image is ramdisk, if the process runs continuously. If you store it 5x per second to flash, I assume your flash card would wear out more quickly than it would otherwise.

Right now the raspistill command can write time-lapse stills with sequentially-numbered file names, but using the maximum possible rate ('-tl 0') I find it is limited to about 2 fps regardless of the image resolution. It seems your application can work faster, so this is a big leap forward. Could RaspiMJPEG also write jpeg file sequences? (something like img%02d.jpg to deliver img01.jpg, img02,jpg, im03.jpg ...)

I also have other suggestions, if it wouldn't be too difficult: the option to set the H.264 bitrate, and also set the H.264 video resolution to something lower. Some applications don't need full-HD, but do need long term recording with low bitrates. In theory you can also simply turn down the bitrate even at full HD, but at low bitrates, you generally get a better image with a smaller image going into the compressor because the rescaling process averages out some of the noise, which helps compression.

silvanmelchior
Posts: 76
Joined: Mon Nov 25, 2013 5:56 pm

Re: RaspiMJPEG

Tue Nov 26, 2013 10:27 pm

Yes, the idea is to use e.g. /dev/shm, otherwise the sd-card can't handle it together with the recording of the 1080p-stream.

I updated the github with my new version, it is now possible to capture images with "im" as pipe-command at the full resolution, the preview just hangs on while capturing and continues afterwards. The filename for the image is set like in raspistill with "image%d.jpg" Additionally it is now possible to set the MJPEG-quality with the parameter -q.

Yes, you're right, I connected the image encoder via image-resizer to the video-output of the camera, so it is possible to write 30 images per second. If you make the quality and resolution low enough, the filesystem can handle it. I added the option to set the output-file as /path/to/output%04d.jpg, like it is possible with RaspiStill, so you can do now "timelapses" with max. 30fps.

I haven't implemented the option to control the video-resolution and bitrate yet, I'm going to do that together with the camera-control-parameters (contrast, white balance, ...). But first I'm going to write a tutorial on how to use my application to control the camera with a website, because my next project is to improve my Quadcopter with the Pi Cam and for that I needed a way to control the cam with a browser. But if its urgent, feel free to modify my source code!

Silvan

mikerr
Posts: 2770
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: RaspiMJPEG

Tue Nov 26, 2013 11:05 pm

Here's an executable binary for those wanting to save 20 minutes compiling:

Code: Select all

wget https://www.dropbox.com/s/3f47cyh902l4dto/raspimjpeg
chmod a+x raspimjpeg

User avatar
jbeale
Posts: 3469
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RaspiMJPEG

Wed Nov 27, 2013 12:16 am

Wow, it works! Thank you very much silvanmelchior for this very useful code! And mikerr also, for the compiled version.

It seems the images start getting saved about 0.5 seconds after starting the program. Using a resolution 480x352 I get a solid 30 fps of JPEG images (at least, no skipped frames in 6 seconds). My setup is not overclocked (ARM = 700 MHz).

Code: Select all

raspimjpeg -w 480 -h 352 -d 1 -of /run/shm/img%03d.jpg
Using 640x480 and asking for 30 fps, the actual achieved framerate is about 25 fps. 800x600 and 1024 x 576 works ok, at least for a few seconds, until the ramdisk is full. You do get some more skipped frames when attempting 30 fps at these resolutions.

Note: you want to select your JPEG dimensions with the same 16:9 aspect ratio of the HD video frame, otherwise it will be distorted (non-square pixels). So my example dimensions above like 640x480 expecting a 4:3 frame are somewhat distorted.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: RaspiMJPEG

Wed Nov 27, 2013 9:18 am

Using 640x480 and asking for 30 fps, the actual achieved framerate is about 25 fps. 800x600 and 1024 x 576 works ok, at least for a few seconds, until the ramdisk is full. You do get some more skipped frames when attempting 30 fps at these resolutions.
What is the limiting factor? CPU 100% ? when doing 640x480 with 25 fps? Oveclocking would then help a lot.
I need to try this. Seems like finally a solution to get mjpeg streaming from the camera.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23060
Joined: Sat Jul 30, 2011 7:41 pm

Re: RaspiMJPEG

Wed Nov 27, 2013 10:08 am

I've been taking a quick look at the MJPEG codec that runs on the GPU - its pretty old code, and needs some tweaks, and as yet isn't working, but if I do get it working it should allow the encoder MMAL component to output MPJEG directly.

No idea if that would help here!

Can anyone explain what the use case for MJPEG is - it's an old format that isn't nearly as good as H264 in quality or size so I am interested to know why people want it.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

silvanmelchior
Posts: 76
Joined: Mon Nov 25, 2013 5:56 pm

Re: RaspiMJPEG

Wed Nov 27, 2013 12:22 pm

MJPEG allows to stream the camera to nearly every browser, including them on mobilephones. H264-livestreams in HTML5 work with m3u-playlists and only the newest browsers support it and there is a big delay (more than a second), where as mjpeg has a very small delay because it always loads the last captured frame, as you can see on my video.

The JPEG-encoder is the same as raspistill uses and is programmed for still images I guess, so I "missuse" it because I feed it with a videostream and write all output pictures to the same file. But it works pretty well so I don't need the mjpeg-encoder on the gpu :).

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: RaspiMJPEG

Wed Nov 27, 2013 12:43 pm

In addition to what silvanmelchior said:
I am using zoneminder with my cameras and it can not handle H264, but it needs MJPEG.
And I have seen questions of how to use zoneminder with Rpi camera, but no answers...so I am not the only one using it.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: RaspiMJPEG

Wed Nov 27, 2013 12:52 pm

jamesh wrote:I've been taking a quick look at the MJPEG codec that runs on the GPU - its pretty old code, and needs some tweaks, and as yet isn't working, but if I do get it working it should allow the encoder MMAL component to output MPJEG directly.
But omxplayer can play mjpeg streams GPU accelerated, so I guess MJPEG code works ?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23060
Joined: Sat Jul 30, 2011 7:41 pm

Re: RaspiMJPEG

Wed Nov 27, 2013 2:55 pm

shuckle wrote:
jamesh wrote:I've been taking a quick look at the MJPEG codec that runs on the GPU - its pretty old code, and needs some tweaks, and as yet isn't working, but if I do get it working it should allow the encoder MMAL component to output MPJEG directly.
But omxplayer can play mjpeg streams GPU accelerated, so I guess MJPEG code works ?
The decoder works fine - the encoder doesn't. I modified the mmal layers and included the encoder in the firmware build (after altering so it's compiles to the new codec API). But it doesn't work...I'll need to talk to the codec guys to see what I missed.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
jbeale
Posts: 3469
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RaspiMJPEG

Wed Nov 27, 2013 5:08 pm

shuckle wrote:
Using 640x480 and asking for 30 fps, the actual achieved framerate is about 25 fps. 800x600 and 1024 x 576 works ok, at least for a few seconds, until the ramdisk is full. You do get some more skipped frames when attempting 30 fps at these resolutions.
What is the limiting factor? CPU 100% ? when doing 640x480 with 25 fps? Oveclocking would then help a lot.
I need to try this. Seems like finally a solution to get mjpeg streaming from the camera.

Code: Select all

raspimjpeg -w 640 -h 480 -d 1 -of /run/shm/image.jpg
shows only 24% CPU utilization, so I imagine it is a timing-synchronization thing where the next camera frame sometimes comes slightly too soon for the file I/O process so it has to wait and do nothing until the next frame after that. (?)

Nu2RPi
Posts: 3
Joined: Tue Dec 10, 2013 10:28 pm

Re: RaspiMJPEG

Tue Dec 10, 2013 10:50 pm

Firstly, hi to all, and secondly I am a RPi Noob, and its been long time since I last used a BSD/linux style system.
RaspiMJPEG looks awesome, however, I am struggling to get it working.
At present i'm getting the following error when running:

Code: Select all

raspimjpeg -w 640 -h 480 -d 1 -of /run/shm/image.jpg
Error:
mmal: mmal_vc_port_parameter_set: failed to set port parameter 40:2:ENOSYS
Error: Could not set inline flag
Now im sure this is something I've missed, for example, I saw in the readme that I needed a 'ARM cross-compiler' installed, I had already installed 'cmake' and compiled 'RaspiMJPEG' (I thought successfully) before reading the readme :roll:.
Anyway, I though id try the precompile version someone posted (many thanks to them) and when I run it shows the same error message.

I was gonna start a fresh and install both 'cmake' and 'https://github.com/raspberrypi/tools/tr ... f-raspbian' however I hadn't a clue how to install the later of these so here I am 'Mr. Clueless' asking for help.
Any help would be greatly appreciated; typically I would search far and wide for a solution before posting, however, RaspiMJPEG appears pretty much bleeding edge and has almost no info on the interweb as of now.

Thx in advance.

silvanmelchior
Posts: 76
Joined: Mon Nov 25, 2013 5:56 pm

Re: RaspiMJPEG

Wed Dec 11, 2013 11:40 am

Have you enabled camera-support (http://www.raspberrypi.org/camera)? Are you running the newest version of raspbian? Maybe a soft- or firmwareupdate might help:

Code: Select all

sudo apt-get update
sudo apt-get dist-upgrade
sudo rpi-update

Nu2RPi
Posts: 3
Joined: Tue Dec 10, 2013 10:28 pm

Re: RaspiMJPEG

Wed Dec 11, 2013 12:27 pm

silvanmelchior wrote:Have you enabled camera-support (http://www.raspberrypi.org/camera)? Are you running the newest version of raspbian? Maybe a soft- or firmwareupdate might help:

Code: Select all

sudo apt-get update
sudo apt-get dist-upgrade
sudo rpi-update
Well all I can say it thank you so much, it works! Well it says 'MJPEG streaming' which is certainly an improvement. The distro needed upgrading.

Now i'm sorry to ask another stupid question but :roll: .....My intention is to use the MJPEG stream with Zoneminder, however, I am a little confused :? as to how exactly I pipe the stream, simply how to access it remotely. I have some idea but rather than muddy the water I'd really like to hear how you experts would make the stream accessible. Ideally with some examples that a simpleton (like myself) can understand.

Joe116
Posts: 24
Joined: Thu Apr 04, 2013 9:22 am

Re: RaspiMJPEG

Wed Dec 11, 2013 12:50 pm

Hello,

for using this with zoneminder you need to stream the jpg with MJPG-stream as example,

http://blog.miguelgrinberg.com/post/how ... spberry-pi
Start with #3 skip #7 and change the path in #8 to the file you wrote with RaspiMJPEG.

I start it like:

Code: Select all

#start cam
nohup raspimjpeg -w 1280 -h 720 -d 1 -of /run/shm/img.jpg &
#start mjpeg-streamer
nohup $(LD_LIBRARY_PATH=/usr/local/lib mjpg_streamer -i "input_file.so -f /run/shm -n img.jpg" -o "output_http.so -w /usr/local/www") &
KR

Joe

Nu2RPi
Posts: 3
Joined: Tue Dec 10, 2013 10:28 pm

Re: RaspiMJPEG

Wed Dec 11, 2013 12:59 pm

Joe116 wrote:Hello,

for using this with zoneminder you need to stream the jpg with MJPG-stream as example,

http://blog.miguelgrinberg.com/post/how ... spberry-pi
Start with #3 skip #7 and change the path in #8 to the file you wrote with RaspiMJPEG.

I start it like:

Code: Select all

#start cam
nohup $(raspimjpeg -w 1280 -h 720 -d 5 -of /run/shm/img.jpg) &
#start mjpeg-streamer
nohup $(LD_LIBRARY_PATH=/usr/local/lib mjpg_streamer -i "input_file.so -f /run/shm -n img.jpg" -o "output_http.so -w /usr/local/www") &
KR

Joe

Many thanks Joe, I'll give that a go hopefully this evening.

Thanks again, Mark.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: RaspiMJPEG

Wed Dec 11, 2013 1:26 pm

Joe116 wrote:Hello,

for using this with zoneminder you need to stream the jpg with MJPG-stream as example,

http://blog.miguelgrinberg.com/post/how ... spberry-pi
Start with #3 skip #7 and change the path in #8 to the file you wrote with RaspiMJPEG.

I start it like:

Code: Select all

#start cam
nohup raspimjpeg -w 1280 -h 720 -d 1 -of /run/shm/img.jpg &
#start mjpeg-streamer
nohup $(LD_LIBRARY_PATH=/usr/local/lib mjpg_streamer -i "input_file.so -f /run/shm -n img.jpg" -o "output_http.so -w /usr/local/www") &
KR

Joe
What kind of speed (fps) you get with 1280x720 resolution?

Joe116
Posts: 24
Joined: Thu Apr 04, 2013 9:22 am

Re: RaspiMJPEG

Wed Dec 11, 2013 3:12 pm

Hello,

the framerate is something between 2 and 5 pictures per second, but the cpu load is below 0.70!

To start the two programs I enter now at the end of /etc/inittab

Code: Select all

T1:234:respawn:raspimjpeg -w 1280 -h 720 -d 5 -of /run/shm/img.jpg
and at the end of /etc/rc.local

Code: Select all

nohup /usr/local/bin/mjpg_streamer -i "/usr/local/lib/input_file.so -f /run/shm -n img.jpg" -o "/usr/local/lib/output_http.so -w /usr/local/www" &
With this the streaming works but I have to test it with zoneminder because my smartphone apps doesn't like the path to the stream

Code: Select all

http://<ip>:8080/?action=stream
KR

Joe

silvanmelchior
Posts: 76
Joined: Mon Nov 25, 2013 5:56 pm

Re: RaspiMJPEG

Wed Dec 11, 2013 3:34 pm

Just a little information: I've written an installer that installs raspimjpeg and appache with a little website on Raspbian, so all you have to do is to run the installer (shell-script) and you'll be able to see the camera preview on a website (640x360 px, 30 fps). I'm going to write a tutorial and will add a link here as soon as I have some free time.

User avatar
jbeale
Posts: 3469
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RaspiMJPEG

Wed Dec 11, 2013 5:55 pm

Thanks again for your work on the great program! To speed downloads for those getting the github source, you might consider removing at least /userland/host_applications/linux/apps/hello_pi/hello_video/test.h264 which is a 31 MB video file (part of "Big Buck Bunny") and is not relevant to the RaspiMJPEG code. That would shrink the download nearly in half.

silvanmelchior
Posts: 76
Joined: Mon Nov 25, 2013 5:56 pm

Re: RaspiMJPEG

Wed Dec 11, 2013 8:37 pm

jbeale wrote:Thanks again for your work on the great program! To speed downloads for those getting the github source, you might consider removing at least /userland/host_applications/linux/apps/hello_pi/hello_video/test.h264 which is a 31 MB video file (part of "Big Buck Bunny") and is not relevant to the RaspiMJPEG code. That would shrink the download nearly in half.
You're welcome :). And you're right, I deleted the movie, so nobody needs to waste space...

Return to “Camera board”