bbond007
Posts: 73
Joined: Sun Nov 04, 2012 6:10 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Sat Dec 01, 2012 7:03 pm

Beach wrote:Thanks for the makefile

It now compiles. One step at a time, because I am getting a runtime error now:

error while loading shared libraries: LibGLesv2.so: cannot open shared object file: No such file or directory
my bad. I took that Makefile from my project and I don't think you even need that lib. Although I would think you should have that...

try removing " -lGLESv2 -lEGL" from the line LDFLAGS

bbond007
Posts: 73
Joined: Sun Nov 04, 2012 6:10 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Sat Dec 01, 2012 7:13 pm

also do "make clean" trying to make it again...

Beach
Posts: 60
Joined: Tue Jul 24, 2012 11:07 am
Location: Noordwijk The Netherlands

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Dec 02, 2012 12:11 pm

---------------> try removing " -lGLESv2 -lEGL" from the line LDFLAGS

It compiles, but now I am getting the error with libopenmaxil.so

Am I the first to try and compile this? (except from you)

bbond007
Posts: 73
Joined: Sun Nov 04, 2012 6:10 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Dec 02, 2012 3:50 pm

Beach wrote:---------------> try removing " -lGLESv2 -lEGL" from the line LDFLAGS

It compiles, but now I am getting the error with libopenmaxil.so

Am I the first to try and compile this? (except from you)
Well, there was at least the original guy (MattOwnby) then the v2 guy (jumble) :)

Anyway, you got it to compile. This issue is with execution. Your EXE would probably run fine on my system. You seem to be missing some shared libraries as "libopenmax" sounds important. The original BIN in the hello_jpeg.zip would probably do the same thing.

This code does work in my app and has probably decoded a million jpegs by now...

User avatar
AndyD
Posts: 2331
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia
Contact: Website

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Dec 02, 2012 8:42 pm

@Beach,
If you can compile the code, an link it into an executable, then all the libraries that you need are on your system. In UNIX/Linux the .so files are Shared Objects, they are the same as DLLs on Windows systems. These Shared Objects are loaded when your program runs. The loader searches for any required Shared objects in a path called the LD_LIBRARY_PATH (amongst others). You can use the LD_LIBRARY_PATH environment variable to tell the system where to look for Shared Objects.

The file libopenmaxil.so is in /opt/vc/lib, to get the loader to search in /opt/vc/lib you need to type the following before you run the program.

Code: Select all

LD_LIBRARY_PATH=/opt/vc/lib
export LD_LIBRARY_PATH
This set the environment variable LD_LIBRARY_PATH, so that the loader uses this as part of the path that it searches.

Hope this helps.

Have a look at the man page for ld.so

Code: Select all

man ld.so

luc4
Posts: 46
Joined: Mon Nov 12, 2012 12:28 am

Re: OpenMAX: How to decode a JPEG (example source code)

Fri Dec 07, 2012 12:23 am

In case someone is still interested, I started from the excellent code of Matt and implemented an example code for decoding images directly to an OpenGL texture: http://thebugfreeblog.blogspot.it/2012/ ... essed.html. I was really satisfied with the result.
Thanks Matt for your code. I learned a lot from it!

Beach
Posts: 60
Joined: Tue Jul 24, 2012 11:07 am
Location: Noordwijk The Netherlands

Re: OpenMAX: How to decode a JPEG (example source code)

Sat Dec 08, 2012 4:39 pm

Well I tried setting the vaiable LD_LIBRARY_PATH but that did not help.
The path is set corectly, I checked with echo $LD_LIBRARY_PATH

I also recompiled the program again, but no dice
Finally ran ldconfig. But alas That did not work either. The correct path is in one of the .conf files in etc/ld.conf.d/

Why would it not find this library?

Might it be a permissions issue? All my /opt/vc/.... is owned by pi (not root)

User avatar
AndyD
Posts: 2331
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia
Contact: Website

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Dec 09, 2012 2:58 am

@Beach, I would be surprised if it is a permissions problem. If can can link against the shared object, then you should be able to load the shared object at run time. Could you post back the output of the following:-

Code: Select all

ldd hello_jpeg.bin
and

Code: Select all

ls -l /opt/vc/lib/

Beach
Posts: 60
Joined: Tue Jul 24, 2012 11:07 am
Location: Noordwijk The Netherlands

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Dec 09, 2012 11:49 am

Hi AndyD
Thanks for helping me out.

ldd hello_jpeg.bin

/usr/lib/arm-linux-gnueabihf/libcofi_rpi.so (0x40265000)
libopenmaxil.so => not found
libbcm_host.so => not found
libvcos.so => not found
libvchiq_arm.so => not found
libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0x4026e000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x4007d000)
libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x40160000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x4033b000)
libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x4013b000)
librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0x40122000)
/lib/ld-linux-armhf.so.3 (0x400fb000)

ls -l /opt/vc/lib

total 1924
-rwxrwxrwx 1 pi pi 73408 Nov 27 00:36 libbcm_host.so
-rwxrwxrwx 1 pi pi 14430 Nov 27 00:36 libdebug_sym.so
-rwxrwxrwx 1 pi pi 205639 Nov 27 00:36 libEGL.so
-rwxrwxrwx 1 pi pi 267716 Nov 27 00:36 libEGL_static.a
-rwxrwxrwx 1 pi pi 105491 Nov 27 00:36 libGLESv1_CM.so
-rwxrwxrwx 1 pi pi 105491 Nov 27 00:36 libGLESv2.so
-rwxrwxrwx 1 pi pi 102644 Nov 27 00:36 libGLESv2_static.a
-rwxrwxrwx 1 pi pi 12580 Nov 27 00:36 libkhrn_client.a
-rwxrwxrwx 1 pi pi 439628 Nov 27 00:36 libkhrn_static.a
-rwxrwxrwx 1 pi pi 50326 Nov 27 00:36 libmmal.so
-rwxrwxrwx 1 pi pi 38773 Nov 27 00:36 libmmal_vc_client.so
-rwxrwxrwx 1 pi pi 30349 Nov 27 00:36 libopenmaxil.so
-rwxrwxrwx 1 pi pi 103510 Nov 27 00:36 libOpenVG.so
-rwxrwxrwx 1 pi pi 2968 Nov 27 00:36 libvcfiled_check.a
-rwxrwxrwx 1 pi pi 25303 Nov 27 00:36 libvchiq_arm.so
-rwxrwxrwx 1 pi pi 90866 Nov 27 00:36 libvchostif.a
-rwxrwxrwx 1 pi pi 171521 Nov 27 00:36 libvcos.so
-rwxrwxrwx 1 pi pi 31426 Nov 27 00:36 libvmcs_rpc_client.a
-rwxrwxrwx 1 pi pi 56090 Nov 27 00:36 libWFC.so

The path is obviously not found. In /etc the file ld.so.conf includes all the *.conf files in ld.so.conf.d (directory) In this directory there is a file that points to /opt/vc/lib

A fortnight ago I moved the original /opt/vc to /opt/vcold and then copied the newest /opt/vc from github. All the other programs like hello_video.bin suffer the same errors now while in the past these worked too.

I am reluctant to do a total installation from scratch because I tweeked it to run mij TV USB dongle and that finally works. The hello_jpeg example is just to get me closer to a hardware accelerated transcoding of the TV images (preferably real time)

Beach
Posts: 60
Joined: Tue Jul 24, 2012 11:07 am
Location: Noordwijk The Netherlands

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Dec 09, 2012 11:55 am

The omxplayer does not run either. Same problem.

User avatar
AndyD
Posts: 2331
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia
Contact: Website

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Dec 09, 2012 9:21 pm

Beach wrote: A fortnight ago I moved the original /opt/vc to /opt/vcold and then copied the newest /opt/vc from github. All the other programs like hello_video.bin suffer the same errors now while in the past these worked too.
My guess would be in moving /opt/vc around you have broken the ld.so.cache for /opt/vc. You can recreate the cache using the ldconfig command. This needs to be run as root.

Code: Select all

sudo ldconfig -v
You may find this interesting
http://tldp.org/HOWTO/Program-Library-H ... html#AEN70

Beach
Posts: 60
Joined: Tue Jul 24, 2012 11:07 am
Location: Noordwijk The Netherlands

Re: OpenMAX: How to decode a JPEG (example source code)

Mon Dec 10, 2012 11:44 am

Thanks Andy for all your input. However, I tried ldconfig several times. And the libraries from /vc/opt/lib are listed in the cache (found it with ldconfig -p). It is a puzzle.
I finally moved the original /opt/vc directory back (made a backup). And then everything is back to normal again. I do not understand why the new /opt/vc libraries could not be found, but I am not going to pursue that path. For now I'm glad the hardware acceleration is working again.

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: OpenMAX: How to decode a JPEG (example source code)

Thu Feb 28, 2013 2:16 am

Hi,
I have downloaded your code, and have been following your instruction, but when I place it at the right directory structure (as you suggested), and issue the make command, I am getting the following error;

MXComponent.o: In function `OMXComponent::GetParameter(OMX_INDEXTYPE, void*)':
/opt/vc/src/hello_pi/hello_jpeg_v2/OMXComponent.cpp:23: undefined reference to `__cxa_allocate_exception'
/opt/vc/src/hello_pi/hello_jpeg_v2/OMXComponent.cpp:23: undefined reference to `std::allocator<char>::allocator()'
/opt/vc/src/hello_pi/hello_jpeg_v2/OMXComponent.cpp:23: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)'

could someone please tell me what could be wrong with this?
By the way, I believe that everything is uptodated on my Pi.
I would truly appreciate some help.
Thanks, --Rudy

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: OpenMAX: How to decode a JPEG (example source code)

Thu Feb 28, 2013 3:33 am

update:
Inside the Makefile, I changed "gcc" to "g++", and it compiled fine.
Now I notice that this is a 32 bit RGBA color space.
When I changed the color format to 24bitRGB888, I get the following error:

Got execption:
OMX_SetParameter falied

Any idea, what is going on? Has any one tried to change to color space to 24bit RGB, and got a similar error?

Thanks in advnace,
--Rudy

MattOwnby
Posts: 58
Joined: Thu Aug 16, 2012 7:22 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Thu Feb 28, 2013 7:19 pm

luc4 wrote:In case someone is still interested, I started from the excellent code of Matt and implemented an example code for decoding images directly to an OpenGL texture: http://thebugfreeblog.blogspot.it/2012/ ... essed.html. I was really satisfied with the result.
Thanks Matt for your code. I learned a lot from it!
Excellent! I actually would be interested in learning how to do this because I currently am decoding to a buffer on the ARM and then sending it to a texture for OpenGL rendering and I would love to skip this overhead :)

thanks for your work! I am glad I released this code! hehe

MattOwnby
Posts: 58
Joined: Thu Aug 16, 2012 7:22 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Thu Feb 28, 2013 7:21 pm

rudy01 wrote: Now I notice that this is a 32 bit RGBA color space.
When I changed the color format to 24bitRGB888, I get the following error:

Got execption:
OMX_SetParameter falied

Any idea, what is going on? Has any one tried to change to color space to 24bit RGB, and got a similar error?
Looks like the resizer doesn't support RGB. It only supports YUV420, RGB565, and RGBA. See the documentation here: https://github.com/raspberrypi/firmware ... esize.html

I will update my earlier post (which implies that you can use the resizer to do all sorts of colorspace conversions) to reflect this.

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: OpenMAX: How to decode a JPEG (example source code)

Thu Feb 28, 2013 8:35 pm

Matt,
Thanks a lot for your reply.
Yes, I just read the document, and unfortunately, it seems it is the case.
I am still looking into a way to bypass the ARM.

By the way, well, since it is really helping me to learn, if it is okay with you, could you make part of the code available, or some hints, samples or instruction about how did you send to a texture for OpenGL rendering?

I was also just curious if OpenGL can offer hardware decoding, or hardware decoding has to be done in OpenMAX?

Thanks,
--Rudy

MattOwnby
Posts: 58
Joined: Thu Aug 16, 2012 7:22 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Thu Feb 28, 2013 8:57 pm

rudy01 wrote:Matt,
Thanks a lot for your reply.
Yes, I just read the document, and unfortunately, it seems it is the case.
I am still looking into a way to bypass the ARM.

By the way, well, since it is really helping me to learn, if it is okay with you, could you make part of the code available, or some hints, samples or instruction about how did you send to a texture for OpenGL rendering?

I was also just curious if OpenGL can offer hardware decoding, or hardware decoding has to be done in OpenMAX?

Thanks,
--Rudy
I'd love to provide some tips on this since I was very pleased to learn it myself. Doing anything in GLES2 is pretty advanced stuff (which is unfortunate) and you kinda have to know 3D math concepts (although I've avoided it as much as I possibly can hehehehe).

I won't make any promises, but some day (hopefully this year) I would like to release an update to hello_jpeg that renders the image to the screen using GLES2. Sounds like some other people have already done this though.

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: OpenMAX: How to decode a JPEG (example source code)

Fri Mar 22, 2013 7:14 am

Hi,
I have been using this code, and it works great and is very fast. However, I believe there might be some bug in the code.
All I did, inside the hello_jpeg.cpp, I am looping around "DoIt" function for a very long time. I put a for loop around this function, and looping it for 100000 times. But, everytime, after few minutes of running, after looping around for only couple thousand loops, the prgram exits by Segmentation fault !!!

I even tried to look at it in "ddd" compiler, to see what is going on, but it existed with this command:

Program received signal SIGSEGV, Segmentation Fault.
(Switching to Thread 0xb6667440 (LWP 1677))
0xb6f05d28 in lics_callback( from /opt/vc/lib/libopenmaxil.so

Any idea why it breaks after about 3000-4000 loop?
Does it suggest that there is some memroy leak in the code? Any way to fix this?

Thanks,
--Rudy

cmarty
Posts: 38
Joined: Thu Jul 19, 2012 7:23 am
Location: Czech Republic

Re: OpenMAX: How to decode a JPEG (example source code)

Fri Mar 22, 2013 10:14 am

I have the same problem as rudy01 reports.
But I'm using the source code originally written by Matt Ownby (no use of ilclient) and adapted to render to texture.
It decodes jpeg files ok, but sometimes (around 50 - 150 decoded pictures) it SIGSEGV with this :
[Switching to Thread 0xb698c460 (LWP 3111)]
0xb6f66d28 in ilcs_callback () from /opt/vc/lib/libopenmaxil.so

I'm not sure if this sigsegv depends on number of decoded images . It seems to me that it is some race condition problem (but i don't have any proof).

Thanks for any help.
Marty

bbond007
Posts: 73
Joined: Sun Nov 04, 2012 6:10 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Sun Mar 24, 2013 11:20 pm

rudy01 wrote: All I did, inside the hello_jpeg.cpp, I am looping around "DoIt" function for a very long time. I put a for loop around this function, and looping it for 100000 times. But, everytime, after few minutes of running, after looping around for only couple thousand loops, the prgram exits by Segmentation fault !!!

--Rudy
I am having time same type of random segfaults with my program "raspytube" (an implementation of the hello_jpegv2). It typically decodes 50 to several hundred images before the crash.

https://github.com/bbond007/raspytube

I have no idea how to fix it, so now by default the program uses libjpeg and my crappy resizer. It seems like a update of the RPi firmware at one point caused this to break, because did work fine for me when I started the project.

I think some people may actually use raspytube, so it would be nice to figure out the bug because the "OMXjpeg" mode brings much nicer images not to mention performance.

Thanks!
bb7

Danara
Posts: 13
Joined: Mon Dec 31, 2012 8:12 pm
Location: Ottawa Canada

Re: OpenMAX: How to decode a JPEG (example source code)

Thu Mar 28, 2013 5:51 pm

bbond007 wrote:
rudy01 wrote: All I did, inside the hello_jpeg.cpp, I am looping around "DoIt" function for a very long time. I put a for loop around this function, and looping it for 100000 times. But, everytime, after few minutes of running, after looping around for only couple thousand loops, the prgram exits by Segmentation fault !!!

--Rudy
I am having time same type of random segfaults with my program "raspytube" (an implementation of the hello_jpegv2). It typically decodes 50 to several hundred images before the crash.

https://github.com/bbond007/raspytube

I have no idea how to fix it, so now by default the program uses libjpeg and my crappy resizer. It seems like a update of the RPi firmware at one point caused this to break, because did work fine for me when I started the project.

I think some people may actually use raspytube, so it would be nice to figure out the bug because the "OMXjpeg" mode brings much nicer images not to mention performance.

Thanks!
bb7
Why not use the ilclient version (the one in regular c, by Matt Ownby and Anthong Sale? This one seems to have no issues with reliability. I'm getting 30+ fps on 640x425 images over hundreds of thousands of iterations. I did have to tweak a few things in this code. Mostly adjusting the timeouts, and getting rid of a call to sleep(1), which was causing about 30 frames to get dropped. I'm now using TIMEOUT_MS 50 instead of 2000. The sleep(1) around line 522 was changed to usleep(30*1000) (30ms). If this is set too short, the output buffer doesn't get filled. I'd like to get rid of the delay altogether and poll to see when it is done. (TODO item).

--Dan

bbond007
Posts: 73
Joined: Sun Nov 04, 2012 6:10 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Sat Mar 30, 2013 2:27 am

Danara wrote: Why not use the ilclient version (the one in regular c, by Matt Ownby and Anthong Sale? This one seems to have no issues with reliability. I'm getting 30+ fps on 640x425 images over hundreds of thousands of iterations. I did have to tweak a few things in this code. Mostly adjusting the timeouts, and getting rid of a call to sleep(1), which was causing about 30 frames to get dropped. I'm now using TIMEOUT_MS 50 instead of 2000. The sleep(1) around line 522 was changed to usleep(30*1000) (30ms). If this is set too short, the output buffer doesn't get filled. I'd like to get rid of the delay altogether and poll to see when it is done. (TODO item).

--Dan
I took your suggestion and tried the regular c version and I tried changing the timeouts to the values you suggested. I assume the code pasted into this thread was the most recent version?

It works(really slowly) while getting the error "no EOS event from image resizer -1".

It also just seems to stall indefinitely after decoding a random number of images.

I am trying to re-size the output in "portSettingsChanged". That is the only real change I made to the code.
https://github.com/bbond007/raspytube/b ... max.c#L107

This is how I'm calling the code. I'm not sure about I'm how trying to reverse the stride and pick my starting pointer(for vgImageSubData). Maybe I'll try to let the images just be upside-down and see if that's the problem. https://github.com/bbond007/raspytube/b ... lib.c#L204

Thanks, 007
//------------------------------------------------------------------------------
VGImage OpenMAXCreateImageFromBuf(unsigned char * buf, size_t bufSize, size_t outputWidth, size_t outputHeight)
{
static OPENMAX_JPEG_DECODER * OMXDecoder = NULL;
VGImage vgImage;
size_t stStride;
VGImageFormat rgbaFormat = VG_sABGR_8888;
unsigned int remainder = outputWidth % 16;
if (remainder == 0)
stStride = (outputWidth * 4);
else
stStride = (outputWidth + (16 - remainder )) * 4;

// if(OMXDecoder == NULL)
OpenMaxJPEG_setupDecoder(&OMXDecoder);
OpenMaxJPEG_decodeImage(OMXDecoder, (char *) buf, bufSize, outputWidth, outputHeight);
vgImage = vgCreateImage(rgbaFormat, outputWidth, outputHeight, VG_IMAGE_QUALITY_BETTER);
vgImageSubData(vgImage, &OMXDecoder->pOutputBufferHeader->pBuffer[stStride * (outputHeight-1)], stStride * -1,
rgbaFormat, 0, 0, outputWidth, outputHeight);
OpenMaxJPEG_cleanup(OMXDecoder);
return vgImage;
}

Danara
Posts: 13
Joined: Mon Dec 31, 2012 8:12 pm
Location: Ottawa Canada

Re: OpenMAX: How to decode a JPEG (example source code)

Sat Mar 30, 2013 4:30 am

Initially, your results seem quite different than what I was seeing, but I was able to reproduce your error message using an image which is much smaller than what I normally use.

When I use an image of 1280x720 (123kb), the error doesn't appear, and the output is good. When I resize that same image to 640x360 using GIMP, I see the No EOS event error on both the decoder and the resizer. Maybe that will help somebody figure out where the bug is lurking.

Cheers
Dan

bbond007
Posts: 73
Joined: Sun Nov 04, 2012 6:10 pm

Re: OpenMAX: How to decode a JPEG (example source code)

Sat Mar 30, 2013 5:08 am

Danara wrote:Initially, your results seem quite different than what I was seeing, but I was able to reproduce your error message using an image which is much smaller than what I normally use.
Thanks Dan!

I should have mentioned that my images are very small to begin with (thumbnails from YouTube) and then I'm further downsizing them even smaller. I think around 88x75 how I have my settings now.

Seems like, in my program, the libjpeg version (with fast integer re-sizer) is fastest, and reliable.

The the cpp version is still pretty fast with occasional timeouts and segfaults.

The c (ilclient) version is the slowest and tends to stall. I have experimented with the various sleep intervals. Maybe the constant resizer EOS error is also causing the slowdown.

I'm not so sure that using the hardware jpeg decoding is best option for me.

Perhaps, maybe, its a little like driving to the airport, parking, going through TSA, getting on plane, loosing luggage, etc... when my destination is only 50 miles away.

Return to “C/C++”