cjweiland
Posts: 2
Joined: Fri May 22, 2015 4:35 pm

audio_decode/audio_render capabilities/questions

Fri May 22, 2015 5:01 pm

I'm trying to play back audio (duh), and I have a few questions that I just can't find proper answers to. Hopefully someone here can answer them...

First, what is the point of OMX.broadcom.audio_decode? Does it serve any purpose whatsoever? I hear that it can only decode PCM to PCM, so does it do anything at all? Can it at least do resampling/upmixing/downmixing in hardware, or can it literally only push out what you push into it? I know it *could* decode other types of audio in the future, but I've also heard that there are no plans to add that kind of support. So...is there any reason why I should feed my audio data through it, or should I just skip it and go straight to the renderer?

Second, I'm having trouble with rendering more than 2 channels with OMX.broadcom.audio_render. When I try to output 6 channels (i.e. 5.1), the pitch is fine, but it sounds like a really fast studder. If I downmix it to stereo, it sounds fine. And if I play back normal stereo source (i.e. no downmixing), that sounds fine too. It says here - http://home.nouwen.name/RaspberryPi/doc ... ender.html - that it will only accept a power of 2 number of channels, but I can't confirm that information from any other source. But if I use ffmpeg to upmix to 7.1 or even if I manually pad each sample with 4 bytes of zeros to fill out 2 more channels, it sounds very low pitch and all around terrible. So it seems to be "ok" with 6 channels other than the quick studdering. Do I need to do something else, like specify the channel layout to OMX? Should I still be able to go with 6 channels or do I really need to give it 8 channels?

From what I can tell, omxplayer doesn't actually upmix to 8 channels unless I'm missing something. I see a few places where it takes into account the power of 2 requirement when it's fiddling with the OMX components, but I don't actually see it fiddling with the channel count of the audio data itself before it gets sent to OMX.

Thanks a million to anyone with any answers!

cjweiland
Posts: 2
Joined: Fri May 22, 2015 4:35 pm

Re: audio_decode/audio_render capabilities/questions

Fri May 22, 2015 6:49 pm

I should maybe clarify that...
-I'm using a Raspberry pi 2/Raspbian
-I'm not using audio_decode since I don't see any point to it at the moment - I'm feeding the audio straight to audio_render
-omxplayer plays the audio that I'm testing with just fine

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5213
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: audio_decode/audio_render capabilities/questions

Wed May 27, 2015 11:48 am

Do you just want audio, or are you using OMX for video too? For just audio ALSA may be a simpler interface.
For openmax, the hello_audio demo app may be informative.

omxplayer uses audio_decode as that is required for passthrough formats (DTS/AC3), Audio decode does parse the compressed bitstream to extract info and packetise it correctly.
Using audio_decode for PCM data makes handling the passthough/pcm formats more consistent.
It is possible to drive audio without an audio_decode component. See the Pi Sink in Kodi for example code.

If you want up/down mixing, then you will need the audio_mixer component. You can provide an 8x8 mixing matrix which allows arbitrary up/down mixing from up to 8 channels to up to 8 channels.
See omxplayer for example usage.

Officially the audio components in openmax only support powers of two numbers of channels. However as 6 channel audio is common and padding from 6->8 channels wastes cycles,
I've added enough support for non-power of two channels to keep omxplayer and kodi happy.

jannewmarch
Posts: 35
Joined: Thu Jan 17, 2013 12:45 am

Re: audio_decode/audio_render capabilities/questions

Sun Jun 07, 2015 10:49 am

The audio_decode component seems happier with WAV data than with PCM data. In omxplayer the output from the ffmpeg decoder is PCM, and a WAV header is added before sending it to the audio_decode component. After months of breaking my head over the omxplayer code I have added a chapter to my online RPi/GPU book at http://jan.newmarch.name/RPi which talks its way through some of the issues such as this.

kevleyski
Posts: 5
Joined: Tue Feb 14, 2012 7:36 pm

Re: audio_decode/audio_render capabilities/questions

Tue Jul 19, 2016 11:59 pm

I'm trying to decode MP3 and AAC using OpenMAX IL. The Broadcom documentation doesn't seem to say it wouldn't work (so where I see the "PCM to PCM only" is new info to me, where is this documented?)

I notice in various examples FFmpeg has been used to get at the LPCM samples, so my question is, is FFmpeg strictly necessary or can the hardware actually handle the audio decode? (struggling to find a working example ;-)

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5213
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: audio_decode/audio_render capabilities/questions

Wed Jul 20, 2016 11:19 am

kevleyski wrote:I'm trying to decode MP3 and AAC using OpenMAX IL. The Broadcom documentation doesn't seem to say it wouldn't work (so where I see the "PCM to PCM only" is new info to me, where is this documented?)

I notice in various examples FFmpeg has been used to get at the LPCM samples, so my question is, is FFmpeg strictly necessary or can the hardware actually handle the audio decode? (struggling to find a working example ;-)
There is no audio decode support possible due to licensing. You must decode to PCM on the arm, or you can use passthough of AC3/DTS if your TV/receiver supports that.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6422
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: audio_decode/audio_render capabilities/questions

Wed Jul 20, 2016 11:33 am

dom wrote:There is no audio decode support possible due to licensing. You must decode to PCM on the arm, or you can use passthough of AC3/DTS if your TV/receiver supports that.
Do I see the vorbis codec lurking in the build variant? Not that common, and I couldn't say when it was last tested, but it seems to be there.
Decode on the ARM is the easiest way to go though.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

kevleyski
Posts: 5
Joined: Tue Feb 14, 2012 7:36 pm

Re: audio_decode/audio_render capabilities/questions

Wed Jul 20, 2016 2:40 pm

6by9 wrote:
dom wrote:There is no audio decode support possible due to licensing. You must decode to PCM on the arm, or you can use passthough of AC3/DTS if your TV/receiver supports that.
Do I see the vorbis codec lurking in the build variant? Not that common, and I couldn't say when it was last tested, but it seems to be there.
Decode on the ARM is the easiest way to go though.
Thanks. No major drama with soft decode.
I'd certainly be interested to know if this changes in the future if a per unit license comes available similar to VC-1/MPEG2

Return to “OpenMAX”