6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Wed Nov 28, 2018 8:19 pm

It's the output from git diff, and can be fed into patch to apply to to your tree.

Code: Select all

diff --git a/host_applications/linux/apps/raspicam/RaspiVid.c b/host_applications/linux/apps/raspicam/RaspiVid.c
Patching file host_applications/linux/apps/raspicam/RaspiVid.c. Git adds the a/ or b/ to differentiate the two trees

Code: Select all

index 21fdc2e..b968c9e 100755
The git hash before (21fdc2e) and after (b968c9e. The file has permissions 100755.

Code: Select all

--- a/host_applications/linux/apps/raspicam/RaspiVid.c
+++ b/host_applications/linux/apps/raspicam/RaspiVid.c
Diffs for host_applications/linux/apps/raspicam/RaspiVid.c

Code: Select all

@@ -1781,7 +1782,7 @@ static MMAL_STATUS_T create_camera_component(RASPIVID_STATE *state)
Start at line 7 lines, starting at line 1781 in one file, 1782 in the other. The function being patched is create_camera_component()

Code: Select all

         mmal_port_parameter_set(video_port, &fps_range.hdr);
    }
 
-   format->encoding = MMAL_ENCODING_OPAQUE;
+   format->encoding = MMAL_ENCODING_I420;
    format->es->video.width = VCOS_ALIGN_UP(state->width, 32);
    format->es->video.height = VCOS_ALIGN_UP(state->height, 16);
    format->es->video.crop.x = 0;
Remove the line "format->encoding = MMAL_ENCODING_OPAQUE;" (hence the minus at the start of the line).
Add the line "format->encoding = MMAL_ENCODING_I420;"

Alternatively if you save it as a file then you should be able to apply it with "patch -p 1 -i patch.txt"
mmal: Unable to set immutable input flag
Not an issue - https://github.com/raspberrypi/userland ... id.c#L2221

I suspect you haven't changed the encoding format (exactly the patch lines I've used as the example above).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Wed Nov 28, 2018 8:25 pm

God! I have not got a clue what all this means lol.

You say there are 2 files to edit?

This is what it looks like are line 1781.

// Set the encode format on the video port

format = video_port->format;
format->encoding_variant = MMAL_ENCODING_I420;

if(state->camera_parameters.shutter_speed > 6000000)
{
MMAL_PARAMETER_FPS_RANGE_T fps_range = {{MMAL_PARAMETER_FPS_RANGE, sizeof(fps_range)},
{ 50, 1000 }, {166, 1000}};
mmal_port_parameter_set(video_port, &fps_range.hdr);
}
else if(state->camera_parameters.shutter_speed > 1000000)
{
MMAL_PARAMETER_FPS_RANGE_T fps_range = {{MMAL_PARAMETER_FPS_RANGE, sizeof(fps_range)},
{ 167, 1000 }, {999, 1000}};
mmal_port_parameter_set(video_port, &fps_range.hdr);
}

At line 7 there are just header comments?

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 7:50 am

After a good nights sleep..

so:

Should the codes changes I need to focus are these:

Start at line 7 lines, starting at line 1781 in one file, 1782 in the other. The function being patched is create_camera_component()
Code: Select all

mmal_port_parameter_set(video_port, &fps_range.hdr);
}

- format->encoding = MMAL_ENCODING_OPAQUE;
+ format->encoding = MMAL_ENCODING_I420;
format->es->video.width = VCOS_ALIGN_UP(state->width, 32);
format->es->video.height = VCOS_ALIGN_UP(state->height, 16);
format->es->video.crop.x = 0;
Remove the line "format->encoding = MMAL_ENCODING_OPAQUE;" (hence the minus at the start of the line).
Add the line "format->encoding = MMAL_ENCODING_I420;"


?

I understand about the git changes. Though confused that you said I need to change 2 files?

I just tired running this:

"patch -p 1 -i patch.txt"

it said it succeeded but it also said before that message: patch unexpectedly ends in middle of line? Is that normal?

Thanks

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 8:16 am

Thanks I got it all to work.

Sorry for all the questions just like to know things.

thanks for your help - very much appreciated!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 10:04 am

For reference I've pushed the commit to a branch at https://github.com/6by9/userland/tree/r ... age_encode.

Sorry, if you want to work with Linux then git (the source control system) and patch are something you'll need to learn at least the basics about fairly quickly.

Now I'll go back to trying to sort the MJPEG codec....
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 10:07 am

Hi,

yes only been only working with it for a few weeks now. Throwing off the shackles of the Windows approach. Thanks again so much for your patience with me. I will definitly endeavour to give something back to this forum as my experience grows.

thanks

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 11:56 am

Hi, I downloaded from that link you gave me and installed it.

I get the error:

mmal_vc_port_info_set: failed to set port info (3:0): EINVAL

I have run a firmware update and did it all again.

The camera is enabled.. the split memory is showing at 256.

so..

rebooted.

did firmware update
copied over that original PR u gave me yesterday
run build.

**DID NOT APPLY PATCH**************

tried to test with;

raspivid -o testvideo.h264 -t 5000

which worked.

Applied the patch which returned OK.

Try to run my app but it exited with the Aborted Message

Tried to run my via command and I got the error from above

Thought u would like to know.

The only thing I can think of is to start from a fresh OS image...
:)

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 1:04 pm

Hi,

I am using a new image

same error as before.

is it possible that this change has damaged the camera? it is OK if that is the case - a risk i am prepared to take. I just dont want to waste my time trying to solve the error if all I need is a new camera which I will need to order so cannot test if that is the case or not.

also running raspistill on the command line now gives an error which it did not do so b4 :(

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 1:15 pm

Using the buildme script installs the updated versions of the app, replacing the standard ones.

Take a fresh image. Don't update anything. Run raspivid or raspistill. Does it work?
Or quicker is to "sudo apt install --reinstall libraspberrypi-bin" which should reinstall the userland packages.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Thu Nov 29, 2018 1:59 pm

"Take a fresh image. Don't update anything. Run raspivid or raspistill. Does it work?"

yes they both work.

Shall I install from that link u gave me this morning?

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Sun Dec 02, 2018 6:48 pm

Just to let you know everything is working.

Many thanks.

a separate question :)

If I sell a commercial product and it is using the Raspberry Pi am I allowed to just have the 'powered by Raspberry Pi" on the packaging only?

thanks

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Sun Dec 02, 2018 6:59 pm

Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.


lagurus
Posts: 46
Joined: Wed Aug 07, 2013 8:02 am

Re: improve quality of jpeg captured from mjpeg stream

Sun Dec 02, 2018 10:22 pm

6by9 wrote:
Wed Nov 28, 2018 5:51 pm
I think I've found the bug in the MJPEG encoder that is causing it to lock up. I'm sort of surprised it worked as well seeing as it is quite such a shocker!
I'll see what I can do about fixing it up.
Just note: I just debug strange behaviour with MJPEG codec which lock camera module and only restart of whole PI repair it. Can it be the same
problem?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 03, 2018 10:13 am

lagurus wrote:
Sun Dec 02, 2018 10:22 pm
6by9 wrote:
Wed Nov 28, 2018 5:51 pm
I think I've found the bug in the MJPEG encoder that is causing it to lock up. I'm sort of surprised it worked as well seeing as it is quite such a shocker!
I'll see what I can do about fixing it up.
Just note: I just debug strange behaviour with MJPEG codec which lock camera module and only restart of whole PI repair it. Can it be the same
problem?
Quite possibly, but as you've given no details of how/when/why you have issues I really can't say.
The issue I found was that if the encoded frame ended up being bigger than the output buffer (default in raspivid of 256kB) then it stalls and won't recover. The codec was incorrectly implementing the API.

I just missed the last rpi-update release with my fix, so it will be in the next one. Linux 4.14.85 and 4.19.6 were released on Saturday, so I suspect Dom will be doing another release in the next couple of days. Watch https://github.com/Hexxeh/rpi-firmware/commits/master for updates, and then use "sudo rpi-update" to update. Normal warnings apply that this is the testing kernel/firmware branch, therefore regressions do sometimes creep in.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Sun Dec 09, 2018 1:56 pm

Out of interest what is the highest resolution i can get out of this? Will it support HD?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 10, 2018 11:15 am

The MJPEG codec will cope with almost any resolution. It is certainly possible to feed the full 8MPix through it.

The hopeful fix for the MJPEG codec is in the latest rpi-update release.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 10, 2018 11:17 am

That's cool :)
I assume I need to apply that patch again? Which is also cool..

thanks

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 10, 2018 12:27 pm

informed wrote:
Mon Dec 10, 2018 11:17 am
That's cool :)
I assume I need to apply that patch again? Which is also cool..
Fixing the MJPEG codec should mean you can go back to the original where you were extracting frames from the MJPEG stream. The patches were all for running image_encode instead of video_encode.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 10, 2018 12:31 pm

wee bit confused. Are you saying I can use the original and still extract jpegs form the stream as I am doing now?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 10, 2018 12:45 pm

informed wrote:
Mon Dec 10, 2018 12:31 pm
wee bit confused. Are you saying I can use the original and still extract jpegs form the stream as I am doing now?
You were reporting that the bitrate/quality factor was limited during MJPEG encode - that is lifted to now allow higher Q-factors.
In investigating that I found a lockup should the encoded frame become bigger than the buffer size - that is also now fixed.

You can still use image_encode, or you can extract frames from the MJPEG stream. Your choice.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 10, 2018 12:47 pm

Ah! Which one is faster or is there absolutely no difference?

Thank you

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6624
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: improve quality of jpeg captured from mjpeg stream

Mon Dec 10, 2018 2:54 pm

informed wrote:
Mon Dec 10, 2018 12:47 pm
Ah! Which one is faster or is there absolutely no difference?
Pass
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Sat Jan 05, 2019 1:42 pm

Hi,
Hope you had a good xmas and New year...

As I know you work for Rpi I thought I would ask you directly.

I have this app that uses the Raspbian stretch OS. I am using your Raspivid.
I wish to sell my product with case, camera and battery with my app installed on it.
I am not charging for your OS - just the hardware and my software.
Do I have to provide a link to your OS like on my web site at all?
Sorry to ask but I have spent weeks trying to get an answer, I do not like to break the rules and if you do not know could you give me a point of contact so I could re-ask them please?

In Hope... :)

informed
Posts: 121
Joined: Mon Oct 26, 2015 6:24 pm

Re: improve quality of jpeg captured from mjpeg stream

Mon Jan 14, 2019 8:46 am

Thanks for following this through.

Got a response which answered everything - thanks...

Final question :)

It occurred to me I may be better off following the main branch rather than the one you kindly created for me to use.

I did a fresh install of raspbian and did all the updates. however, after using the original command line arguments I illustrated right at the beginning of this post it came back with codec not recognised.

Could you confirm the args i need to stream jpegs please?

Thanks

Return to “Camera board”