pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

OMX_AllocateBuffer fails for audio decoder component

Fri Jan 04, 2013 7:53 pm

Hi,

I'm working on a program to play livetv from a mythtv backend. I got the communication with the backend working and can start a stream. In this case mpegps. I use ffmpeg to demux the stream an have the video working using example code from the hello_video and omxplayer.

Now I am trying to get audio working. Goal for now is to play through hdmi with hwdecoding (AC3) Creation of the different components is working but when I try to Allocate the buffers for the decoder input it fails after creation of about 10 buffers and returns OMX_ErrorIncorrectStateOperation (0x80001018). Before it return an OMX_EventError with nData1=0x80001019 and nData2=0x00000001 is generated. (0x80001019 = OMX_ErrorUnsupportedSetting).

Im using the same process as in the AllocInputBuffers in omxplayer.
- Put component into Idle state if it is not.
- Enable input port
- OMX_AllocateBuffer for nBufferCountActual buffers.
- WaitForFinished enable port command.

This process is also used for the video codec and this one is successful. It does not make any difference if I first allocate buffers for the audio decoder or the video decoder only the audio decoder fails.

Any help is appreciated.
Michel.

Following is a small part of my debug log for the audio codec:

Code: Select all

demuxerInitOMXAudio:563 This audio stream has codec_id=86019, sample_rate=48000, bits_per_coded_sample=0 and channels=6.
omxSetAudioCompressionFormatAndBuffer:868 OMX_AUDIO_CodingDDP
omxSetAudioCompressionFormatAndBuffer:904 chunkLen=6144, bufferLen=576000.
omxSetStateForComponent:522 before OMX_SendCommand.
omxEventHandlerCallback:284 OMX.broadcom.audio_decode Event: OMX_EventCmdComplete, nData1=0x00000000, nData2=0x00000002.
omxEventHandlerCallback:296 Before omxAddEvent.
omxAddEvent:65 empty eventlist going to create new list.
omxAddEvent:67 After createSimpleListItem.(OMX.broadcom.audio_decode)
omxEventHandlerCallback:298 Added event to queue.
omxSetStateForComponent:524 after OMX_SendCommand.
omxSetStateForComponent:536 omx_err == OMX_ErrorNone.
omxWaitForCommandComplete:140 Going to wait for CommandComplete event command=0x00000000, nData2=0x00000002 with 1000 microseconds of timeout.
omxGetEvent:112 Retreived event from queue nData1=0x00000000, nData2=0x00000002.(OMX.broadcom.audio_decode)
omxWaitForCommandComplete:153 Received event from queue.
omxWaitForCommandComplete:155 Received valid event from queue.
omxWaitForCommandComplete:158 Event OMX_EventCmdComplete.
omxWaitForCommandComplete:160 Found event we were waiting for.
omxSetStateForComponent:538 after omxWaitForCommandComplete.
omxSetStateForComponent:546 end of omxSetStateForComponent.
omxEnablePort:558 Port 120 on component OMX.broadcom.audio_decode.
omxAllocInputBuffers:1065 Component(OMX.broadcom.audio_decode) - port(120), nBufferCountMin(4), nBufferCountActual(93), nBufferSize(6144), nBufferAlignmen(16)
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 0.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 1.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 2.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 3.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 4.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 5.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 6.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 7.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 8.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 9.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 10.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 11.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 12.
omxAllocInputBuffers:1070 Component(OMX.broadcom.audio_decode) - creating buffer 13.
omxEventHandlerCallback:284 OMX.broadcom.audio_decode Event: OMX_EventError, nData1=0x80001019, nData2=0x00000001.
omxEventHandlerCallback:296 Before omxAddEvent.
omxAddEvent:65 empty eventlist going to create new list.
omxAddEvent:67 After createSimpleListItem.(OMX.broadcom.audio_decode)
omxEventHandlerCallback:298 Added event to queue.
omxAllocInputBuffers:1086 Component(OMX.broadcom.audio_decode) - OMX_UseBuffer/OMX_AllocateBuffer failed with omxErr(0x80001018)
demuxerInitOMXAudio:584 audio decoder omxAllocInputBuffers. (Error=0x80001018).
Details from ffmpeg demuxer probing:

Code: Select all

Input #0, mpegts, from '':
  Duration: N/A, start: 120988.296811, bitrate: 384 kb/s
  Program 1 
    Stream #0:0[0x205]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x5c](dut): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:2[0x24](dut): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
    Stream #0:3[0x32]: Unknown: none ([5][0][0][0] / 0x0005)
Debug log for creating the video decoder buffer:

Code: Select all

omxSetStateForComponent:524 after OMX_SendCommand.
omxSetStateForComponent:536 omx_err == OMX_ErrorNone.
omxWaitForCommandComplete:140 Going to wait for CommandComplete event command=0x00000000, nData2=0x00000002 with 1000 microseconds of timeout.
omxGetEvent:112 Retreived event from queue nData1=0x00000000, nData2=0x00000002.(OMX.broadcom.video_decode)
omxWaitForCommandComplete:153 Received event from queue.
omxWaitForCommandComplete:155 Received valid event from queue.
omxWaitForCommandComplete:158 Event OMX_EventCmdComplete.
omxWaitForCommandComplete:160 Found event we were waiting for.
omxSetStateForComponent:538 after omxWaitForCommandComplete.
omxSetStateForComponent:546 end of omxSetStateForComponent.
demuxerInitOMXVideo:447 This video stream has a framerate of 25.000000 fps.
demuxerInitOMXVideo:455 This video stream has a framesize of 1920x1080.
omxEnablePort:558 Port 130 on component OMX.broadcom.video_decode.
omxAllocInputBuffers:1065 Component(OMX.broadcom.video_decode) - port(130), nBufferCountMin(1), nBufferCountActual(60), nBufferSize(81920), nBufferAlignmen(16)
omxAllocInputBuffers:1070 Component(OMX.broadcom.video_decode) - creating buffer 0.
.....
omxAllocInputBuffers:1070 Component(OMX.broadcom.video_decode) - creating buffer 59.
omxEventHandlerCallback:284 OMX.broadcom.video_decode Event: OMX_EventCmdComplete, nData1=0x00000003, nData2=0x00000082.
omxEventHandlerCallback:296 Before omxAddEvent.
omxAddEvent:65 empty eventlist going to create new list.
omxAddEvent:67 After createSimpleListItem.(OMX.broadcom.video_decode)
omxEventHandlerCallback:298 Added event to queue.
omxWaitForCommandComplete:140 Going to wait for CommandComplete event command=0x00000003, nData2=0x00000082 with 10000 microseconds of timeout.
omxGetEvent:112 Retreived event from queue nData1=0x00000003, nData2=0x00000082.(OMX.broadcom.video_decode)
omxWaitForCommandComplete:153 Received event from queue.
omxWaitForCommandComplete:155 Received valid event from queue.
omxWaitForCommandComplete:158 Event OMX_EventCmdComplete.
omxWaitForCommandComplete:160 Found event we were waiting for.

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Sat Jan 05, 2013 9:00 am

Update.

Browsed to the latest omxplayer code in XBMC and it always is using OMX_AUDIO_CodingPCM as audio encoding instead of also allowing OMX_AUDIO_CodingDDP for AC3.

I think the audio_decoder is still only allowing audio passthrough or PCM audio. And XBMC is using ffmpeg to convert audio stream to PCM or uses passthrough.

I am changing my code to do passthrough as my TV can do AC3. Will let you know the result.

saintdev
Posts: 39
Joined: Mon Jun 18, 2012 10:56 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Sat Jan 05, 2013 9:32 pm

pimiv wrote:Hi,

I'm working on a program to play livetv from a mythtv backend. I got the communication with the backend working and can start a stream. In this case mpegps. I use ffmpeg to demux the stream an have the video working using example code from the hello_video and omxplayer.
I think you mean MPEG-TS, MPEG-PS is what is used in DVDs ;)

I would be interested in helping out on this, do you have your code anywhere public yet? I was planning on starting something similar, but haven't gotten around to it yet. It sounds like you might have similar goals to me. A lighter weight mythfrontend for use on the Pi. Initially I planned to only support LiveTV/ATSC, with recordings and possibly other standards handled later (for countries like NZ that use h.264 and AAC).
Now I am trying to get audio working. Goal for now is to play through hdmi with hwdecoding (AC3) Creation of the different components is working but when I try to Allocate the buffers for the decoder input it fails after creation of about 10 buffers and returns OMX_ErrorIncorrectStateOperation (0x80001018). Before it return an OMX_EventError with nData1=0x80001019 and nData2=0x00000001 is generated. (0x80001019 = OMX_ErrorUnsupportedSetting).
Are you trying to actually do hardware AC3 decode on the VideoCore, or just pass through the AC3? Currently there is no license for the VideoCore AC3 decoder, you have to either decode it in software, or do HDMI pass through and have the device at the other end of the HDMI cable decode it itself.

Edit: Just noticed your reply, you figured that out already. Let me know how it goes!
SELECT `signature` FROM `users` WHERE `username`='saintdev';
Empty set (0.00 sec)

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Sun Jan 06, 2013 3:36 pm

Hi. Thank you for your reply.

I just uploaded the code to github: https://github.com/1stsetup/piMythclient
It currently works with myth v0.26

saintdev
Posts: 39
Joined: Mon Jun 18, 2012 10:56 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Sun Jan 06, 2013 8:55 pm

pimiv wrote:Hi. Thank you for your reply.

I just uploaded the code to github: https://github.com/1stsetup/piMythclient
Awesome, I'll see what I can do to help out.
It currently works with myth v0.26
Heh, I'm still stuck on 0.25-fixes, the Gentoo maintainers are still getting 0.26 ready.
SELECT `signature` FROM `users` WHERE `username`='saintdev';
Empty set (0.00 sec)

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Mon Jan 07, 2013 6:33 am

This can be easily fixed to get it compiled also for v0.25
When I have time tonight I will create a fix or a modified makefile which can compile for v0.25

Please be patient.

saintdev
Posts: 39
Joined: Mon Jun 18, 2012 10:56 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Mon Jan 07, 2013 6:39 am

I don't think the protocol has changed too much between the two versions. Though I haven't yet looked at the 0.26 protocol changes yet. No need for you to worry about getting it to work with 0.25, I can take care of it ;)
SELECT `signature` FROM `users` WHERE `username`='saintdev';
Empty set (0.00 sec)

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 10, 2013 2:48 pm

Hi,

I just wanted to say that I'm working on a similar project - a Pi frontend for tvheadend. I've been developing this on github here:

https://github.com/linuxstb/pidvbip

It's still in the early stages, and a lot of the code is still first attempts just to get something running, rather than polished code that I can be proud of, but it may contain things that are of interest to you. The video playback code is just a hack at the moment (I don't use the OMX clocks, it's just the hello_video playback and a hack to keep the audio vaguely in sync) and sorting that out i's the next thing I need to work on.

AFAIK, there are no hardware audio decoders available on the Pi, you need to decode the audio on the ARM and pass PCM to the GPU. I don't use ffmpeg in my project - I am using libmpg123, liba52 and libfaad directly for audio decoding. tvheadend's streaming format (htsp) provides easy-to-use video and audio packets, so I don't need any demuxers.

pidvbip is also written in C - I'm happy to see you're using the same (too much C++ around!).

Regards.

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 10, 2013 4:07 pm

Thank you for the info.

I have the video and audio working now in sync.
For audio no hwdecoding. I use ffmpeg for decoding and downsampling to stereo. I did not find the right setting yet to use the hw audio mixer to do this downsampling. ffmpeg decodes the AC3 audio stream to PCM float planar and accoring to the ilcomponent manual for the audio mixer is only accepts linear input.

Still with all my debugging on and a h264/AC3 1080p 50 hz high profile livetv stream it only uses 35%cpu. Cool.

Only Video difference I see between the pi and my NVIDIA vdpau htpc is when there is some rapid movement in the picture (like a waving arm). The pi's output has more blurring in it than the vdpau machine.

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Fri Jan 11, 2013 9:02 am

Hi pimiv,

That's great. Looking at your code on github, I can't see any (C) messages or license information. What license are you releasing this code under?

Also, have you pushed all your changes to make A/V sync work?

Thanks.

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Fri Jan 11, 2013 2:06 pm

Hi,

The license info still needs to be entered. It will be GPL. I put it online because someone else also wanted to have a look at it.

The audio and video is in sync. This is done by the hw clock which is hooked up to the video_render and audio_render.

The data I feed to the video_decoder has an omx_buffer->nTimeStamp. This is the PTS for the video frame which is returned by the ffmpeg demuxer and is converted to microseconds. The data I feed to the audio_render also has an omx_buffer->nTimeStamp. This is the PTS for audio frame which is returned by the ffmpeg demuxer and is converted to microseconds.

The broadcom hw arranges the sync.

jannewmarch
Posts: 35
Joined: Thu Jan 17, 2013 12:45 am

Re: OMX_AllocateBuffer fails for audio decoder component

Wed Jan 23, 2013 8:09 am

Has anyone managed to get AllocateBuffers working with audio_decode? I'm getting the same OMX_ErrorIncorrectStateOperation on the first buffer I allocate, even though I have already called OMX_SendCommand(handle, OMX_CommandStateSet, OMX_StateIdle, NULL). Or is this a broken component?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23059
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Wed Jan 23, 2013 9:15 am

jannewmarch wrote:Has anyone managed to get AllocateBuffers working with audio_decode? I'm getting the same OMX_ErrorIncorrectStateOperation on the first buffer I allocate, even though I have already called OMX_SendCommand(handle, OMX_CommandStateSet, OMX_StateIdle, NULL). Or is this a broken component?
Al the components are pretty thoroughly tested and are used in production equipment, so unlikely to be a broken component.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Wed Jan 23, 2013 11:27 am

I got this error when I set the encoding for the audio_decoder to DTS. After reading the manual currently only PCM is supported as for the other types a license is required which is not available last time I checked.

jannewmarch
Posts: 35
Joined: Thu Jan 17, 2013 12:45 am

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 24, 2013 7:15 am

Solved! The default value for eEncoding is MP3. This is a patent-protected encoding and is not supported by Broadcom (presumably they would have to pay money to the Fraunhofer Institute). If you leave the encoding to the default, the component signals this on a call to OMX_AllocateBuffer by returning OMX_ErrorIncorrectStateOperation ("A command or method was attempted that is not allowed during the present state"). Well, that's a pretty useless error message and not in the slightest bit accurate or informative!

The only supported audio encoding seems to be PCM. So the fix is to set the encoding to PCM _BEFORE_ any call to AllocateBuffer:

sPortDef.format.audio.eEncoding = OMX_AUDIO_CodingPCM;
sPortDef.nBufferCountActual = sPortDef.nBufferCountMin;
OMX_SetParameter(handle, OMX_IndexParamPortDefinition, &sPortDef);
...
OMX_AllocateBuffer(handle, pBuffer, portNumber, NULL, nBufferSize);

So what's the point of an audio_decode component that can only decode from PCM to PCM? I don't know... Pity it can't at least support Ogg Vorbis which is a free and open encoding. I'll put this fix into my free online book http://jan.newmarch.name/LinuxSound/

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23059
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 24, 2013 10:20 am

jannewmarch wrote:Solved! The default value for eEncoding is MP3. This is a patent-protected encoding and is not supported by Broadcom (presumably they would have to pay money to the Fraunhofer Institute). If you leave the encoding to the default, the component signals this on a call to OMX_AllocateBuffer by returning OMX_ErrorIncorrectStateOperation ("A command or method was attempted that is not allowed during the present state"). Well, that's a pretty useless error message and not in the slightest bit accurate or informative!

The only supported audio encoding seems to be PCM. So the fix is to set the encoding to PCM _BEFORE_ any call to AllocateBuffer:

sPortDef.format.audio.eEncoding = OMX_AUDIO_CodingPCM;
sPortDef.nBufferCountActual = sPortDef.nBufferCountMin;
OMX_SetParameter(handle, OMX_IndexParamPortDefinition, &sPortDef);
...
OMX_AllocateBuffer(handle, pBuffer, portNumber, NULL, nBufferSize);

So what's the point of an audio_decode component that can only decode from PCM to PCM? I don't know... Pity it can't at least support Ogg Vorbis which is a free and open encoding. I'll put this fix into my free online book http://jan.newmarch.name/LinuxSound/
Welcome to the not so wonderful world of OpenMAX, where error codes are purely for entertainment.

The audio decode component can decode from anything to anything - depending on the available codecs. Since only one GPU codec is provided (not necessarily the case on other platforms), you only have PCM to PCM. It's probable we don't have an OGG decoder on the GPU, but I'll try to find out why its not provided if we do.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 24, 2013 12:11 pm

I have not tested the settings for OGG/vorbis or any of the other open formats like g711. Broadcom is the only one who can tell what type of codecs are in the firmware/hardware

Doe anyone know how to see if a license for a codec is installed? Is there an API call for? And I do not mean try to read the config.xtxt file in /boot. Or do I need to open a new thread for this?

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5282
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 24, 2013 1:16 pm

pimiv wrote:I have not tested the settings for OGG/vorbis or any of the other open formats like g711. Broadcom is the only one who can tell what type of codecs are in the firmware/hardware

Doe anyone know how to see if a license for a codec is installed? Is there an API call for? And I do not mean try to read the config.xtxt file in /boot. Or do I need to open a new thread for this?
Only PCM is supported by audio_decode currently. I'm am working on getting OGG Vorbis supported.
You can check for supported codecs with vcgencmd codec_enabled
http://www.elinux.org/RPI_vcgencmd_usage

(I expect it is possible through openMAX too)

User avatar
vk2
Posts: 3
Joined: Sun Jan 13, 2013 4:03 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Fri Jan 25, 2013 7:02 am

Only PCM is supported by audio_decode currently. I'm am working on getting OGG Vorbis supported.
Thanks for you reply! Is there any chance to get hardware AAC/MP3/WMA decoding? May be with paid license?

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Fri Jan 25, 2013 9:49 am

dom wrote:
pimiv wrote:I have not tested the settings for OGG/vorbis or any of the other open formats like g711. Broadcom is the only one who can tell what type of codecs are in the firmware/hardware

Doe anyone know how to see if a license for a codec is installed? Is there an API call for? And I do not mean try to read the config.xtxt file in /boot. Or do I need to open a new thread for this?
Only PCM is supported by audio_decode currently. I'm am working on getting OGG Vorbis supported.
You can check for supported codecs with vcgencmd codec_enabled
http://www.elinux.org/RPI_vcgencmd_usage

(I expect it is possible through openMAX too)
Thank you for the answer. Following is an c example to get the results using the API:
Compile with: cc -I/opt/vc/include -I/opt/vc/include/interface/vcos/pthreads -L/opt/vc/lib -lbcm_host -o vc_gencmd_example vc_gencmd_example.c

Code: Select all

#include <stdio.h>
#include <interface/vmcs_host/vcgencmd.h>

void showLicenseStatus()
{
   int gencmdHandle = vc_gencmd_init();

   if (gencmdHandle != 0) {
      printf("Error vc_gencmd_init. Error=%d.\n", gencmdHandle);
      return;
   }

   char *response = malloc(1024);
   memset(response, 0, 1024);

   int vc_error = vc_gencmd(response, 1023, "codec_enabled H264");
   if (vc_error != 0) {
      printf("Error vc_gencmd. Error=%d.\n", vc_error);
      free(response);
      return;
   }

   printf("vc_gencmd response: %s.\n", response);
   free(response);
}

int main(int argc, char *argv[])
{
   bcm_host_init();
   showLicenseStatus();
   return 0;
}

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5282
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: OMX_AllocateBuffer fails for audio decoder component

Fri Jan 25, 2013 11:54 am

vk2 wrote:Thanks for you reply! Is there any chance to get hardware AAC/MP3/WMA decoding? May be with paid license?
Yes, that is possible. In fact GPU accelerated DTS and AC3 have been supported for months, but licensing hasn't been negotiated, so we can't provide keys.

jannewmarch
Posts: 35
Joined: Thu Jan 17, 2013 12:45 am

Re: OMX_AllocateBuffer fails for audio decoder component

Sat Jan 26, 2013 2:14 am

dom said "I expect it [getting suported formats] is possible through openMAX too"

Here is the OpenMAX code to get supported formats. It works okay for Broadcom, but the Bellagio library returns CodingUnused for all possible indices, which seems incorrect to me - the spec says that at least one format must be supported. Sorry about the lost indentation

void getSupportedAudioFormats(int portNumber) {
OMX_AUDIO_PARAM_PORTFORMATTYPE sAudioPortFormat;

setHeader(&sAudioPortFormat, sizeof(OMX_AUDIO_PARAM_PORTFORMATTYPE));
sAudioPortFormat.nIndex = 0;
sAudioPortFormat.nPortIndex = portNumber;

printf("Supported audio formats are:\n");
for(;;) {
err = OMX_GetParameter(handle, OMX_IndexParamAudioPortFormat, &sAudioPortFormat);
if (err == OMX_ErrorNoMore) {
printf("No more formats supported\n");
break;
}
if (sAudioPortFormat.eEncoding == OMX_AUDIO_CodingUnused) {
printf("No coding format returned\n");
return;
}

switch (sAudioPortFormat.eEncoding) {
case OMX_AUDIO_CodingPCM:
printf("Supported encoding is PCM\n");
break;
case OMX_AUDIO_CodingVORBIS:
printf("Supported encoding is Ogg Vorbis\n");
break;
case OMX_AUDIO_CodingMP3:
printf("Supported encoding is MP3\n");
break;
default:
printf("Supported encoding is not PCM or MP3 or Vorbis, is %d\n",
sAudioPortFormat.eEncoding);
}
sAudioPortFormat.nIndex++;
}
}

jannewmarch
Posts: 35
Joined: Thu Jan 17, 2013 12:45 am

Re: OMX_AllocateBuffer fails for audio decoder component

Tue Jan 29, 2013 7:12 am

I used my previous program to find the supported formats that the audio_decode component supports. According to the OpenMAX IL spec, "OMX_AUDIO_PARAM_PORTFORMATTYPE is the structure for the port format parameter. This structure enumerates the various data formats that the port supports." Running the program attached results in

Code: Select all

    Port 120 requires 4 buffers
    Port 120 is an audio port
    Port mimetype (null)
    Port encoding is MP3
      Supported audio formats are:
      Supported encoding is MP3
          MP3 default sampling rate 0
          MP3 default bits per sample 0
          MP3 default number of channels 0
      Supported encoding is PCM
          PCM default sampling rate 0
          PCM default bits per sample 0
          PCM default number of channels 0
      Supported encoding is AAC
      Supported encoding is WMA
      Supported encoding is Ogg Vorbis
      Supported encoding is RA
      Supported encoding is AMR
      Supported encoding is EVRC
      Supported encoding is G726
      Supported encoding is FLAC
      Supported encoding is DDP
      Supported encoding is DTS
      Supported encoding is WMAPRO
      Supported encoding is ATRAC3
      Supported encoding is ATRACX
      Supported encoding is ATRACAAL
      Supported encoding is MIDI
      No more formats supported
if you set the encoding to, say, OMX_AUDIO_CodingPC by OMX_SetParameter(handle, OMX_IndexParamPortDefinition, &sPortDef) then you can get default info about the format. The above shows the defaults for MP3 and PCM.

So: this doesn't match what happens in practice, where only PCM is supported (at present) while the component seems to report that it supports 17 different formats. Any clues about this discrepancy?

Code: Select all

/**
   Based on code
   Copyright (C) 2007-2009 STMicroelectronics
   Copyright (C) 2007-2009 Nokia Corporation and/or its subsidiary(-ies).
   under the LGPL
*/

#define RASPBERRY_PI

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <string.h>
#include <pthread.h>
#include <unistd.h>
#include <sys/stat.h>

#include <OMX_Core.h>
#include <OMX_Component.h>
#include <OMX_Types.h>
#include <OMX_Audio.h>

#ifdef RASPBERRY_PI
#include <bcm_host.h>
#endif

OMX_ERRORTYPE err;
OMX_HANDLETYPE handle;
OMX_VERSIONTYPE specVersion, compVersion;

OMX_CALLBACKTYPE callbacks;

#define indent {int n = 0; while (n++ < indentLevel*2) putchar(' ');}

static void setHeader(OMX_PTR header, OMX_U32 size) {
    /* header->nVersion */
    OMX_VERSIONTYPE* ver = (OMX_VERSIONTYPE*)(header + sizeof(OMX_U32));
    /* header->nSize */
    *((OMX_U32*)header) = size;

    /* for 1.2
       ver->s.nVersionMajor = OMX_VERSION_MAJOR;
       ver->s.nVersionMinor = OMX_VERSION_MINOR;
       ver->s.nRevision = OMX_VERSION_REVISION;
       ver->s.nStep = OMX_VERSION_STEP;
    */
    ver->s.nVersionMajor = specVersion.s.nVersionMajor;
    ver->s.nVersionMinor = specVersion.s.nVersionMinor;
    ver->s.nRevision = specVersion.s.nRevision;
    ver->s.nStep = specVersion.s.nStep;
}

void printState() {
    OMX_STATETYPE state;
    err = OMX_GetState(handle, &state);
    if (err != OMX_ErrorNone) {
	fprintf(stderr, "Error on getting state\n");
	exit(1);
    }
    switch (state) {
    case OMX_StateLoaded: fprintf(stderr, "StateLoaded\n"); break;
    case OMX_StateIdle: fprintf(stderr, "StateIdle\n"); break;
    case OMX_StateExecuting: fprintf(stderr, "StateExecuting\n"); break;
    case OMX_StatePause: fprintf(stderr, "StatePause\n"); break;
    case OMX_StateWaitForResources: fprintf(stderr, "StateWiat\n"); break;
    default:  fprintf(stderr, "State unknown\n"); break;
    }
}

OMX_ERRORTYPE setEncoding(int portNumber, OMX_AUDIO_CODINGTYPE encoding) {
    OMX_PARAM_PORTDEFINITIONTYPE sPortDef;

    setHeader(&sPortDef, sizeof(OMX_PARAM_PORTDEFINITIONTYPE));
    sPortDef.nPortIndex = portNumber;
    sPortDef.nPortIndex = portNumber;
    err = OMX_GetParameter(handle, OMX_IndexParamPortDefinition, &sPortDef);
    if(err != OMX_ErrorNone){
        fprintf(stderr, "Error in getting OMX_PORT_DEFINITION_TYPE parameter\n",
 0);
        exit(1);
    }

    sPortDef.format.audio.eEncoding = encoding;
    sPortDef.nBufferCountActual = sPortDef.nBufferCountMin;

    err = OMX_SetParameter(handle, OMX_IndexParamPortDefinition, &sPortDef);
    return err;
}

void getPCMInformation(int indentLevel, int portNumber) {
    /* assert: PCM is a supported mode */
    OMX_AUDIO_PARAM_PCMMODETYPE sPCMMode;

    /* set it into PCM format before asking for PCM info */
    if (setEncoding(portNumber, OMX_AUDIO_CodingPCM) != OMX_ErrorNone) {
	fprintf(stderr, "Error in setting coding to PCM\n");
	return;
    }
       
    setHeader(&sPCMMode, sizeof(OMX_AUDIO_PARAM_PCMMODETYPE));
    sPCMMode.nPortIndex = portNumber;
    err = OMX_GetParameter(handle, OMX_IndexParamAudioPcm, &sPCMMode);
    if(err != OMX_ErrorNone){
	indent printf("PCM mode unsupported\n");
    } else {
	indent printf("  PCM default sampling rate %d\n", sPCMMode.nSamplingRate);
	indent printf("  PCM default bits per sample %d\n", sPCMMode.nBitPerSample);
	indent printf("  PCM default number of channels %d\n", sPCMMode.nChannels);
  }      
}
void getMP3Information(int indentLevel, int portNumber) {
    /* assert: MP3 is a supported mode */
    OMX_AUDIO_PARAM_MP3TYPE sMP3Mode;

    /* set it into MP3 format before asking for MP3 info */
    if (setEncoding(portNumber, OMX_AUDIO_CodingMP3) != OMX_ErrorNone) {
	fprintf(stderr, "Error in setting coding to MP3\n");
	return;
    }
    
    setHeader(&sMP3Mode, sizeof(OMX_AUDIO_PARAM_MP3TYPE));
    sMP3Mode.nPortIndex = portNumber;
    err = OMX_GetParameter(handle, OMX_IndexParamAudioMp3, &sMP3Mode);
    if(err != OMX_ErrorNone){
	indent printf("MP3 mode unsupported\n");
    } else {
	indent printf("  MP3 default sampling rate %d\n", sMP3Mode.nSampleRate);
	indent printf("  MP3 default bits per sample %d\n", sMP3Mode.nBitRate);
	indent printf("  MP3 default number of channels %d\n", sMP3Mode.nChannels);
    }   
}


void getSupportedAudioFormats(int indentLevel, int portNumber) {
    OMX_AUDIO_PARAM_PORTFORMATTYPE sAudioPortFormat;

    setHeader(&sAudioPortFormat, sizeof(OMX_AUDIO_PARAM_PORTFORMATTYPE));
    sAudioPortFormat.nIndex = 0;
    sAudioPortFormat.nPortIndex = portNumber;

    indent printf("Supported audio formats are:\n");
    for(;;) {
        err = OMX_GetParameter(handle, OMX_IndexParamAudioPortFormat, &sAudioPortFormat);
        if (err == OMX_ErrorNoMore) {
	    indent printf("No more formats supported\n");
	    return;
        }

	/* This shouldn't occur, but does with Broadcom library */
	if (sAudioPortFormat.eEncoding == OMX_AUDIO_CodingUnused) {
	     indent printf("No coding format returned\n");
	     return;
	}

	switch (sAudioPortFormat.eEncoding) {
	case OMX_AUDIO_CodingPCM:
	    indent printf("Supported encoding is PCM\n");
	    getPCMInformation(indentLevel+1, portNumber);
	    break; 
	case OMX_AUDIO_CodingVORBIS:
	    indent printf("Supported encoding is Ogg Vorbis\n");
	    // getVorbisInformation(indentLevel+1, portNumber);
	    break; 
	case OMX_AUDIO_CodingMP3:
	    indent printf("Supported encoding is MP3\n");
	    getMP3Information(indentLevel+1, portNumber);
	    break;
#ifdef RASPBERRY_PI
	case OMX_AUDIO_CodingFLAC:
	    indent printf("Supported encoding is FLAC\n");
	    break; 
	case OMX_AUDIO_CodingDDP:
	    indent printf("Supported encoding is DDP\n");
	    break; 
	case OMX_AUDIO_CodingDTS:
	    indent printf("Supported encoding is DTS\n");
	    break; 
	case OMX_AUDIO_CodingWMAPRO:
	    indent printf("Supported encoding is WMAPRO\n");
	    break; 
#endif
	    
	case OMX_AUDIO_CodingAAC:
	    indent printf("Supported encoding is AAC\n");
	    break; 
	case OMX_AUDIO_CodingWMA:
	    indent printf("Supported encoding is WMA\n");
	    break;
	case OMX_AUDIO_CodingRA:
	    indent printf("Supported encoding is RA\n");
	    break; 
	case OMX_AUDIO_CodingAMR:
	    indent printf("Supported encoding is AMR\n");
	    break; 
	case OMX_AUDIO_CodingEVRC:
	    indent printf("Supported encoding is EVRC\n");
	    break;
	case OMX_AUDIO_CodingG726:
	    indent printf("Supported encoding is G726\n");
	    break;
	case OMX_AUDIO_CodingMIDI:
	    indent printf("Supported encoding is MIDI\n");
	    break;
	case OMX_AUDIO_CodingATRAC3:
	    indent printf("Supported encoding is ATRAC3\n");
	    break;
	case OMX_AUDIO_CodingATRACX:
	    indent printf("Supported encoding is ATRACX\n");
	    break;
	case OMX_AUDIO_CodingATRACAAL:
	    indent printf("Supported encoding is ATRACAAL\n");
	    break;
	    /*
	case OMX_AUDIO_Coding:
	    indent printf("Supported encoding is \n");
	    break;
	    */
	default:
	    indent printf("Supported encoding is not PCM or MP3 or Vorbis, is %d\n",
		  sAudioPortFormat.eEncoding);
	}
        sAudioPortFormat.nIndex++;
    }
}



void getAudioPortInformation(int indentLevel, int nPort, OMX_PARAM_PORTDEFINITIONTYPE sPortDef) {
    indent printf("Port %d requires %d buffers\n", nPort, sPortDef.nBufferCountMin); 
    
    if (sPortDef.eDir == OMX_DirInput) {
	indent printf("Port %d is an input port\n", nPort);
    } else {
	indent printf("Port %d is an output port\n",  nPort);
    }
    switch (sPortDef.eDomain) {
    case OMX_PortDomainAudio:
	indent printf("Port %d is an audio port\n", nPort);
	indent printf("Port mimetype %s\n",
	       sPortDef.format.audio.cMIMEType);

	switch (sPortDef.format.audio.eEncoding) {
	case OMX_AUDIO_CodingPCM:
	    indent printf("Port encoding is PCM\n");
	    break; 
	case OMX_AUDIO_CodingVORBIS:
	    indent printf("Port encoding is Ogg Vorbis\n");
	    break; 
	case OMX_AUDIO_CodingMP3:
	    indent printf("Port encoding is MP3\n");
	    break; 
	default:
	    indent printf("Port encoding is not PCM or MP3 or Vorbis, is %d\n",
		   sPortDef.format.audio.eEncoding);
	}
	getSupportedAudioFormats(indentLevel+1, nPort);

	break;
	/* could put other port types here */
    default:
	indent printf("Port %d is not an audio port\n",  nPort);
    }
      

}

void getAllAudioPortsInformation(int indentLevel) {
    OMX_PORT_PARAM_TYPE param;
    OMX_PARAM_PORTDEFINITIONTYPE sPortDef;

    int startPortNumber;
    int nPorts;
    int n;

    setHeader(&param, sizeof(OMX_PORT_PARAM_TYPE));
    //param.nVersion.nVersion = OMX_VERSION;
    err = OMX_GetParameter(handle, OMX_IndexParamAudioInit, &param);
    if(err != OMX_ErrorNone){
	fprintf(stderr, "Error in getting OMX_PORT_PARAM_TYPE parameter\n", 0);
	exit(1);
    }
    indent printf("Audio ports:\n");
    indentLevel++;
    //getAudioPortInformation((OMX_PORT_PARAM_TYPE)param);

    startPortNumber = param.nStartPortNumber;
    nPorts = param.nPorts;
    if (nPorts == 0) {
	indent printf("No ports of this type\n");
	return;
    }

    indent printf("Ports start on %d\n", startPortNumber);
    indent printf("There are %d open ports\n", nPorts);

    for (n = 0; n < nPorts; n++) {
	setHeader(&sPortDef, sizeof(OMX_PARAM_PORTDEFINITIONTYPE));
	sPortDef.nPortIndex = startPortNumber + n;
	err = OMX_GetParameter(handle, OMX_IndexParamPortDefinition, &sPortDef);
	if(err != OMX_ErrorNone){
	    fprintf(stderr, "Error in getting OMX_PORT_DEFINITION_TYPE parameter\n", 0);
	    exit(1);
	}
	getAudioPortInformation(indentLevel+1, startPortNumber + n, sPortDef);
    }
}

void getAllVideoPortsInformation(int indentLevel) {
    OMX_PORT_PARAM_TYPE param;
    int startPortNumber;
    int nPorts;
    int n;

    setHeader(&param, sizeof(OMX_PORT_PARAM_TYPE));
    //param.nVersion.nVersion = OMX_VERSION;
    err = OMX_GetParameter(handle, OMX_IndexParamVideoInit, &param);
    if(err != OMX_ErrorNone){
	fprintf(stderr, "Error in getting OMX_PORT_PARAM_TYPE parameter\n", 0);
	exit(1);
    }
    printf("Video ports:\n");
    indentLevel++;

    startPortNumber = param.nStartPortNumber;
    nPorts = param.nPorts;
    if (nPorts == 0) {
	indent printf("No ports of this type\n");
	return;
    }

    indent printf("Ports start on %d\n", startPortNumber);
    indent printf("There are %d open ports\n", nPorts);
}

void getAllImagePortsInformation(int indentLevel) {
    OMX_PORT_PARAM_TYPE param;
    int startPortNumber;
    int nPorts;
    int n;

    setHeader(&param, sizeof(OMX_PORT_PARAM_TYPE));
    //param.nVersion.nVersion = OMX_VERSION;
    err = OMX_GetParameter(handle, OMX_IndexParamVideoInit, &param);
    if(err != OMX_ErrorNone){
	fprintf(stderr, "Error in getting OMX_PORT_PARAM_TYPE parameter\n", 0);
	exit(1);
    }
    printf("Image ports:\n");
    //getPortInformation((OMX_PORT_PARAM_TYPE)param);
    indentLevel++;

    startPortNumber = param.nStartPortNumber;
    nPorts = param.nPorts;
    if (nPorts == 0) {
	indent printf("No ports of this type\n");
	return;
    }

    indent printf("Ports start on %d\n", startPortNumber);
    indent printf("There are %d open ports\n", nPorts);
}

void getAllOtherPortsInformation(int indentLevel) {
    OMX_PORT_PARAM_TYPE param;
    int startPortNumber;
    int nPorts;
    int n;

    setHeader(&param, sizeof(OMX_PORT_PARAM_TYPE));
    //param.nVersion.nVersion = OMX_VERSION;
    err = OMX_GetParameter(handle, OMX_IndexParamVideoInit, &param);
    if(err != OMX_ErrorNone){
	fprintf(stderr, "Error in getting OMX_PORT_PARAM_TYPE parameter\n", 0);
	exit(1);
    }
    printf("Other ports:\n");
    //getPortInformation((OMX_PORT_PARAM_TYPE)param);
    indentLevel++;

    startPortNumber = param.nStartPortNumber;
    nPorts = param.nPorts;
    if (nPorts == 0) {
	indent printf("No ports of this type\n");
	return;
    }

    indent printf("Ports start on %d\n", startPortNumber);
    indent printf("There are %d open ports\n", nPorts);
}

int main(int argc, char** argv) {

    OMX_PORT_PARAM_TYPE param;
    OMX_PARAM_PORTDEFINITIONTYPE sPortDef;
    OMX_AUDIO_PORTDEFINITIONTYPE sAudioPortDef;
    OMX_AUDIO_PARAM_PORTFORMATTYPE sAudioPortFormat;
    OMX_AUDIO_PARAM_PCMMODETYPE sPCMMode;

#ifdef RASPBERRY_PI
    char *componentName = "OMX.broadcom.audio_mixer";;
#else
    char *componentName = "OMX.st.volume.component";
#endif
    unsigned char name[128]; /* spec says 128 is max name length */
    OMX_UUIDTYPE uid;
    int startPortNumber;
    int nPorts;
    int n;

    /* ovveride component name by command line argument */
    if (argc == 2) {
	componentName = argv[1];
    }

# ifdef RASPBERRY_PI
    bcm_host_init();
# endif

    err = OMX_Init();
    if(err != OMX_ErrorNone) {
	fprintf(stderr, "OMX_Init() failed\n", 0);
	exit(1);
    }
    /** Ask the core for a handle to the volume control component
     */
    err = OMX_GetHandle(&handle, componentName, NULL /*app private data */, &callbacks);
    if(err != OMX_ErrorNone) {
	fprintf(stderr, "OMX_GetHandle failed\n", 0);
	exit(1);
    }
    err = OMX_GetComponentVersion(handle, name, &compVersion, &specVersion, &uid);
    if(err != OMX_ErrorNone) {
	fprintf(stderr, "OMX_GetComponentVersion failed\n", 0);
	exit(1);
    }
    printf("Component name: %s version %d.%d, Spec version %d.%d\n",
	   name, compVersion.s.nVersionMajor,
	   compVersion.s.nVersionMinor,
	   specVersion.s.nVersionMajor,
	   specVersion.s.nVersionMinor);

    /** Get  ports information */
    getAllAudioPortsInformation(0);
}

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23059
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Tue Jan 29, 2013 7:51 am

Interesting! I'll check with the OpenMAX team.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23059
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Tue Jan 29, 2013 10:20 am

Hi,

Checked with the team. The way it works is that the component passes back success for all the codecs it can potentially support. (i.e. all the codecs we've ever had going). That is then constrained by what codecs are actually installed. It would be better to run time detect which codecs are present, but that code has never been written since its never been required.
It's also unlikely ever to be done as Broadcom no longer support audio codecs in this way - they have moved off the Videocore to the host CPU since they are now powerful enough to handle any audio decoding task. I'm not sure how you would do this on a Raspi using OpenMAX (i.e. run audio decode on the Arm), out of my knowledge domain!

James
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Return to “OpenMAX”