Muhammad
Posts: 7
Joined: Thu Aug 16, 2012 10:34 pm

Decoding a JPEG into a user-mode buffer

Thu Aug 16, 2012 10:47 pm

I found my RasPi in my pigeon hole at work yesterday and have already started tinkering :) I thought I'd take a look at JPEG decoding, having recently done a lot of work with libjpeg and libjpeg-turbo on other SOCs, and was pleased to find that it's possible to decode a JPEG in hardware, using the OMX API.

I hacked "hello_video" to decode a JPEG to screen (using the "video_render" sink) but haven't managed to get it to decode to an arbitrary user-mode buffer. I had a look at the available Broadcom OMX components, and whilst I can see file writers, I can't see one for "output to this buffer *here*".

The "source" component seems to be what I want (please correct me if I'm wrong), allowing the creation of raw bitmaps in various colour formats. However, I can't seem to set it up correctly.

Is what I'm trying to achieve actually possible with the current firmware? Can it be done using another component, e.g., "eglrender"?

I also presume that there might be a penalty involved if there's a kernel -> user mode copy, unless we're able to use shared memory, or some other no-copy technique.

Any help much appreciated - thanks in advance!

Muhammad
Posts: 7
Joined: Thu Aug 16, 2012 10:34 pm

Re: Decoding a JPEG into a user-mode buffer

Fri Aug 17, 2012 12:58 pm

Update: I ditched the ilclient stuff and decided to have a go decoding to a buffer using the OMX_* APIs.

It seems to be working! It currently decodes a JPEG contained in a buffer to an output buffer, then writes the buffer to a raw data file. Successfully loaded the image in GIMP!

Will post the code once I tidy it up a little :D

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5349
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Decoding a JPEG into a user-mode buffer

Fri Aug 17, 2012 1:04 pm

Muhammad wrote:Update: I ditched the ilclient stuff and decided to have a go decoding to a buffer using the OMX_* APIs.

It seems to be working! It currently decodes a JPEG contained in a buffer to an output buffer, then writes the buffer to a raw data file. Successfully loaded the image in GIMP!

Will post the code once I tidy it up a little :D
Great. Sounds like a library function useful to many people.

Muhammad
Posts: 7
Joined: Thu Aug 16, 2012 10:34 pm

Re: Decoding a JPEG into a user-mode buffer

Fri Aug 17, 2012 2:52 pm

Thanks.

I'm now removing unnecessary memcpy()s and getting the decoder to use pre-allocated buffers as input & output in an attempt to decode directly onto an existing canvas/frame buffer with a different stride to the source image. Going well so far; managed to decode to an arbitrary point in a 32 bpp buffer albeit with a little corruption at the moment.

Once that's done, I'll write a benchmark to see how it performs against libjpeg 6 and post up the code & results.

jumble
Posts: 8
Joined: Wed Aug 22, 2012 2:45 pm

Re: Decoding a JPEG into a user-mode buffer

Wed Aug 22, 2012 2:52 pm

Muhamed

Nice piece of work, i think with openmax and opengl we can do some interesting work on the pi.
I've been trying to use your code to decode image frames from a mjpeg stream sent from my web cam. I've not been able to decode the images, although I've saved them to disk and gimp is happy with them, and there do not appear to be any error messages from the decider. I've also struggled to decode some images from gimp as well. Are you aware of any jpeg elements that are definitely required for the decoder to work

Cheers

Joppe
Posts: 1
Joined: Sun Aug 26, 2012 11:18 am

Re: Decoding a JPEG into a user-mode buffer

Sun Aug 26, 2012 11:27 am

Muhammad wrote:Update: I ditched the ilclient stuff and decided to have a go decoding to a buffer using the OMX_* APIs.

It seems to be working! It currently decodes a JPEG contained in a buffer to an output buffer, then writes the buffer to a raw data file. Successfully loaded the image in GIMP!

Will post the code once I tidy it up a little :D
Nice work! Would it be possible to see the source code? I tried to do the same thing with ilclient as an experiment, but gave up, since I could not figure out how to do it. Your code might lead me in the right direction.

If you don't want to go public, could you please PM me?

Muhammad
Posts: 7
Joined: Thu Aug 16, 2012 10:34 pm

Re: Decoding a JPEG into a user-mode buffer

Fri Aug 31, 2012 12:36 pm

Sorry, been busy with work, I will post it up as soon as I get a moment. Don't have any issues with sharing it :)

User avatar
AndyD
Posts: 2331
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia
Contact: Website

Re: Decoding a JPEG into a user-mode buffer

Tue Nov 27, 2012 6:58 am

There is now a hello_jpeg example in /opt/vc/src/hello_pi. It hasn't appeared on Debian Wheezy as yet, but it can be found here https://github.com/raspberrypi/firmware ... hello_jpeg.

User avatar
AndyD
Posts: 2331
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia
Contact: Website

Re: Decoding a JPEG into a user-mode buffer

Tue Nov 27, 2012 11:52 am

OK, now I understand. The code actually came from this thread http://www.raspberrypi.org/phpBB3/viewt ... 33&t=15463

Muhammad
Posts: 7
Joined: Thu Aug 16, 2012 10:34 pm

Re: Decoding a JPEG into a user-mode buffer

Fri Apr 12, 2013 10:04 am

Hello everyone!

It's been a while, been busy with work and release schedules. The good news is that hopefully I'll be getting back to this JPEG work soon. Just need to find where I put the code, hopefully in Perforce somewhere ;)

Muhammad

Zinahe
Posts: 16
Joined: Sun Jan 29, 2012 11:56 pm

Re: Decoding a JPEG into a user-mode buffer

Tue Jul 23, 2013 5:21 pm

Hi Muhammad,

I think there is still interest in your work. Would like to check the code should you decide to share it.

Cheers,

Return to “OpenMAX”