jannewmarch
Posts: 35
Joined: Thu Jan 17, 2013 12:45 am

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 31, 2013 7:33 am

Thanks James

That is a very interesting answer! So the current situation looks like:
  • audio_render and presumably audio_capture are probably just thin wrappers around the
    ALSA driver
  • audio_decode and presumably audio_encode and all the other audio_XXX components
    only support PCM
  • These audio components will not be upgraded to support other encodings as processing
    will be left to the CPU (using e.g. ffmpeg)
My question now is: if I am doing audio processing only (no video), then is there any value
in using OpenMAX IL at all? Instead, I could just use ffmpeg and ALSA and avoid any OpenMAX
overheads. Is the situation any different if I want to do video processing too (maybe timing and synchronisation?)?

pimiv
Posts: 10
Joined: Fri Jan 04, 2013 7:33 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 31, 2013 8:45 am

jamesh wrote:Hi,

Checked with the team. The way it works is that the component passes back success for all the codecs it can potentially support. (i.e. all the codecs we've ever had going). That is then constrained by what codecs are actually installed. It would be better to run time detect which codecs are present, but that code has never been written since its never been required.
It's also unlikely ever to be done as Broadcom no longer support audio codecs in this way - they have moved off the Videocore to the host CPU since they are now powerful enough to handle any audio decoding task. I'm not sure how you would do this on a Raspi using OpenMAX (i.e. run audio decode on the Arm), out of my knowledge domain!

James
James,

Thank you for your reply. I hope the history of this thread helps others to identity the cause of the problem they possibly run into as I did.

I still will use the audio_render component for synchronizing audio and video in a simple way by connecting them both to the same clock and use ffmpeg to do the decoding. I hope that ffmpeg will get better support for the Pi to do faster decoding by using the ARM cpu (in a smarter way) or that broadcom will add new functionality to the existing firmware so OpenMAX audio_codec can be used. Or is this last part out of the question?

Cheers Michel.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23059
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Thu Jan 31, 2013 9:40 am

jannewmarch wrote:Thanks James

That is a very interesting answer! So the current situation looks like:
  • audio_render and presumably audio_capture are probably just thin wrappers around the
    ALSA driver
  • audio_decode and presumably audio_encode and all the other audio_XXX components
    only support PCM
  • These audio components will not be upgraded to support other encodings as processing
    will be left to the CPU (using e.g. ffmpeg)
My question now is: if I am doing audio processing only (no video), then is there any value
in using OpenMAX IL at all? Instead, I could just use ffmpeg and ALSA and avoid any OpenMAX
overheads. Is the situation any different if I want to do video processing too (maybe timing and synchronisation?)?
1) I think they are wrappers around the codecs rather than alsa in particular, but I'm not expert.
2) They support which codecs are installed AND enabled. (ie licenced). At the moment, PCM is the only codec that fits those requirements.
3) I believe so. Broadcom policy is now to move audio processing off the Videocore on to the host CPU.

Your question. No idea!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

jannewmarch
Posts: 35
Joined: Thu Jan 17, 2013 12:45 am

Re: OMX_AllocateBuffer fails for audio decoder component

Sat Feb 02, 2013 10:13 am

Let's try a different approach. There is a Bellagio implementation of OpenMAX IL at https://sourceforge.net/projects/omxil/?source=dlp. This is open source and runs under Linux (compiles ok on theRPi), but its major advantage is that there is a 14 page document explaining how to write your own components using the Bellagio core at http://en.sourceforge.jp/projects/sfnet ... e_0.3.pdf/. In addition, it has (incomplete) source code examples with components decoding Ogg Vorbis, MP3 etc using libavformat and ffmpeg.

I guess we are not going to see decoders and encoders real soon from Broadcom that support encodings other than PCM. But there are a million RPi's sold already, and in this forum we see linuxstb, pimiv and dom working on encoding/decoding, plus a big forum on "omxtx - open source transcoding example". If there was a document explaining how to write components for the Broadcom core, then there is a good chance that we would see people filling in the gaps so that there could be a complete end-to-end OpenMAX IL solution. Yes, these components would be run on the CPU and yes, they would use libavformat/ffmpeg libraries but that's okay.

Does anyone know if there is, or will be, a public document explaining how to write components for the Broadcom core?

saintdev
Posts: 39
Joined: Mon Jun 18, 2012 10:56 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Sat Feb 02, 2013 8:30 pm

jannewmarch wrote:I guess we are not going to see decoders and encoders real soon from Broadcom that support encodings other than PCM.
Dom has already added support for Vorbis to the latest testing firmware (along with a few other royalty free video codecs) see this post. These are not real hardware accelerated codecs, but do utilize the VideoCore SIMD engine to improve their performance.
Does anyone know if there is, or will be, a public document explaining how to write components for the Broadcom core?
In short, no. The OpenMax userspace library is already open source https://github.com/raspberrypi/userland. However, it is just a wrapper around calls to the VideoCore firmware. It is very very very unlikely that Broadcom will release documentation on programming the VideoCore GPU.
SELECT `signature` FROM `users` WHERE `username`='saintdev';
Empty set (0.00 sec)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23059
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_AllocateBuffer fails for audio decoder component

Sat Feb 02, 2013 8:53 pm

jannewmarch wrote:Let's try a different approach. There is a Bellagio implementation of OpenMAX IL at https://sourceforge.net/projects/omxil/?source=dlp. This is open source and runs under Linux (compiles ok on theRPi), but its major advantage is that there is a 14 page document explaining how to write your own components using the Bellagio core at http://en.sourceforge.jp/projects/sfnet ... e_0.3.pdf/. In addition, it has (incomplete) source code examples with components decoding Ogg Vorbis, MP3 etc using libavformat and ffmpeg.

I guess we are not going to see decoders and encoders real soon from Broadcom that support encodings other than PCM. But there are a million RPi's sold already, and in this forum we see linuxstb, pimiv and dom working on encoding/decoding, plus a big forum on "omxtx - open source transcoding example". If there was a document explaining how to write components for the Broadcom core, then there is a good chance that we would see people filling in the gaps so that there could be a complete end-to-end OpenMAX IL solution. Yes, these components would be run on the CPU and yes, they would use libavformat/ffmpeg libraries but that's okay.

Does anyone know if there is, or will be, a public document explaining how to write components for the Broadcom core?
The codec issue isn't really an OpenMAX one - the AudioDecode component is there and working, it's just that it can only decode when there is an appropriate codec installed. I'll need to chat to the OMX team, but I do wonder we have access to Arm side decode components that use the Arm codecs to decode. I'm not sure how we do that at the moment, since we already decode on the Arm for things like DTS, so it must do something along those lines. Dom would know....
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Return to “OpenMAX”