timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

OMX_EmptyThisBuffer segfault/no PortSettingsChanged [SOLVED]

Tue Apr 09, 2013 11:00 am

Hi all,

I am modifying the video_decode example to get a simple (h264 encoded) streamer working.
The modifications are slight for now as I have simply replaced the file read by a server call (which goes through ok!).

My problem is that the OMX_EmptyThisBuffer call inside the while loop segfaults.
Being new to RPi and OpenMAX/ilclient, I have not yet determined why this is happening.
Do you have any ideas?

I have called bcm_host_init, so that is not the problem.

Thanks for your time.
Last edited by timmehboy on Wed May 01, 2013 9:45 am, edited 2 times in total.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23624
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_EmptyThisBuffer segfault

Tue Apr 09, 2013 11:59 am

Without seeing the specific code, I'm guessing a buffer which should exist doesn't. Check out your changes and specifically what is happening with buffer allocation.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Tue Apr 09, 2013 12:06 pm

Hi jamesh,

I am writing the encoded VNC payload to "buf", which is the OMX_BUFFERHEADERTYPE pointer in the example code -- perhaps I should be writing to "buf->pBuffer" instead?

Also, once "buf" has been returned from ilclient_get_input_buffer(), it has been allocated right?

Thanks for your reply!

EDIT: I am indeed writing to buf->pBuffer...
It is weird that simply replacing a call to fread() would break it so badly! :/

I have noticed that hello_video reads in chunks of 16384 bytes -- I am getting some chunks at least 4 times that size. I will investigate dividing them up.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23624
Joined: Sat Jul 30, 2011 7:41 pm

Re: OMX_EmptyThisBuffer segfault

Tue Apr 09, 2013 2:38 pm

I think you would need to provide some sort of buffer allocation. I don't think OpenMAX does it for you.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 10:57 am

It doesn't.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 12:52 pm

Ok, I have fixed the segmentation fault, it was caused by me writing the whole packet in one go to the (smaller) pBuffer.

My problem now is that the OMX_EventPortSettingsChanged event does not seem to be firing/caught properly; my code is getting stuck on:

Code: Select all

         // ...
         if(port_settings_changed == 0 &&
            ((data_len > 0 && ilclient_remove_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1) == 0) ||
             (data_len == 0 && ilclient_wait_for_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1,
                                                       ILCLIENT_EVENT_ERROR | ILCLIENT_PARAMETER_CHANGED, 10000) == 0)))
once my intial buffer has been emptied (I have verified this). The result is that the renderer and the tunnel are not getting enabled!

Again, my code is almost identical to https://github.com/raspberrypi/firmware ... eo/video.c, save for reading and breaking the packet up into 16K chunks.

What could be causing this to happen?
Last edited by timmehboy on Wed May 01, 2013 9:45 am, edited 1 time in total.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 12:58 pm

Are you reading whole frames, and setting the buffer flags appropriately when you're feeding it partials?

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 1:31 pm

dickon wrote:Are you reading whole frames, and setting the buffer flags appropriately when you're feeding it partials?
I am reading in whole frames, and dividing them up into 16K blocks for the decoder's buffer, as per hello_video.
What do you mean by setting the flags for the partials?
I have tested dumping the stream to a file, and hello_video happily displays it, so the stream contents should not be the issue.

Apologies in advance if I am missing the obvious!
Last edited by timmehboy on Wed May 01, 2013 9:46 am, edited 1 time in total.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 1:56 pm

You're splitting a frame into chunks. The last chunk needs to have OMX_BUFFERFLAG_ENDOFFRAME set, or the decoder won't know it's got a full frame to decode.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 2:05 pm

dickon wrote:You're splitting a frame into chunks. The last chunk needs to have OMX_BUFFERFLAG_ENDOFFRAME set, or the decoder won't know it's got a full frame to decode.
Fair enough. I have modified my code in the following way:

Code: Select all

         buf->nFilledLen = data_len;
         data_len = 0;

         buf->nOffset = 0;
         if(first_packet)
         {
            buf->nFlags = OMX_BUFFERFLAG_STARTTIME;
            first_packet = 0;
         }
         else
            buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN;

         // check for end of frame
         if(data_len == 0)
            buf->nFlags |= OMX_BUFFERFLAG_ENDOFFRAME;
        //etc.
But unfortunately I am still stuck with the same problem, the EventPortSettingsChanged does not seem to be coming through.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 2:35 pm

You'll have to stick your code up somewhere.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 3:20 pm

Sure. I have modified it to buffer more data in before decoding, which according to the trace works:

Code: Select all

snip
The function HandleH264 is called every time a RFB update message is received.
Last edited by timmehboy on Wed May 01, 2013 9:47 am, edited 1 time in total.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 3:35 pm

You're only calling VNCReadData() once at the top of HandleH264, which doesn't look right. The decoder may need more than one frame of data before it'll change state.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 3:40 pm

dickon wrote:You're only calling VNCReadData() once at the top of HandleH264, which doesn't look right. The decoder may need more than one frame of data before it'll change state.
Notice how the HandleH264 only proceeds once BUFFER_MIN bytes have been read, so multiple (at least half a dozen) VNCReadData() calls actually go through in practice before we proceed with the decoding (verified through printf's ;) ). i have tried with values of 200,000 and 500,000 for BUFFER_MIN, with the same result.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 4:27 pm

Oh, yes. I see.

You're actually reading an arbitrary amount of data -- arbitrary number of H.264 frames -- into a large buffer, then passing that to the decoder in 16K chunks, setting the 'end of frame' marker on the last bit of the buffer you send. This isn't right.

You need to pass -- ideally -- one frame per buffer to the decoder. If a frame spans more than one buffer, you split it, and set the end-of-frame marker on the last buffer. That's quite different to what you're doing.

Why are you choosing 16K, OOI? buf->nBufferLen would be more sensible.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 4:42 pm

dickon wrote:Oh, yes. I see.

You're actually reading an arbitrary amount of data -- arbitrary number of H.264 frames -- into a large buffer, then passing that to the decoder in 16K chunks, setting the 'end of frame' marker on the last bit of the buffer you send. This isn't right.

You need to pass -- ideally -- one frame per buffer to the decoder. If a frame spans more than one buffer, you split it, and set the end-of-frame marker on the last buffer. That's quite different to what you're doing.

Why are you choosing 16K, OOI? buf->nBufferLen would be more sensible.
I chose 16K simply to stay as similar to "hello_video" as possible in case things went wrong (as they have done) -- you are right about nAllocLen, I will try that.

About splitting on frames; how exactly would I go about that? I am rather new to video decoding/OpenMAX (I guess that is apparent by now...).

Thanks for your time.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 5:01 pm

This is where it gets complex...

H.264 streams are often (but not always) split into NAL units. These are what the decoder expects to see; if it's not, you'll need to repacketise it using ffmpeg's libav* h264_mp4toannexb() bitstream filter (as you've already said if you write the bitstream to disc hello_video will play it, I imagine this won't be an issue. Worth pointing out, though). To spot NALs, you need to look for the bytes 0x00, 0x00, 0x00, 0x01 -- in that order -- anywhere within the stream, and that's the start of your NAL. The end is where the next one starts.

Once you've got a NAL -- there are some very small setup ones at the beginning of most streams, but others may be tens of KB -- copy it into OMX_Buffer(s), and pass it to EmptyThisBuffer as you'd expect.

Good luck.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 5:13 pm

dickon wrote:This is where it gets complex...

H.264 streams are often (but not always) split into NAL units. These are what the decoder expects to see; if it's not, you'll need to repacketise it using ffmpeg's libav* h264_mp4toannexb() bitstream filter (as you've already said if you write the bitstream to disc hello_video will play it, I imagine this won't be an issue. Worth pointing out, though). To spot NALs, you need to look for the bytes 0x00, 0x00, 0x00, 0x01 -- in that order -- anywhere within the stream, and that's the start of your NAL. The end is where the next one starts.

Once you've got a NAL -- there are some very small setup ones at the beginning of most streams, but others may be tens of KB -- copy it into OMX_Buffer(s), and pass it to EmptyThisBuffer as you'd expect.

Good luck.
Thanks for the info dickon.
Good news is, each packet starts with 0x00, 0x00, 0x00, 0x01; hence, each one is a NAL.
So should I try to fit a whole NAL into the buffer? I am getting a buf->nAllocLen of 81920, but my first NAL is 91653 bytes.
Last edited by timmehboy on Wed May 01, 2013 9:47 am, edited 1 time in total.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Wed Apr 10, 2013 11:44 pm

Good. Stick the first 81920 bytes into a buffer and feed it to the decoder, then stick the remaining bytes in a second buffer flagged as OMX_BUFFERFLAG_ENDOFFRAME, and feed that in. Repeat as needed, and you should see your state change.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Thu Apr 11, 2013 8:25 am

dickon wrote:Good. Stick the first 81920 bytes into a buffer and feed it to the decoder, then stick the remaining bytes in a second buffer flagged as OMX_BUFFERFLAG_ENDOFFRAME, and feed that in. Repeat as needed, and you should see your state change.
Ok, will try that, thank you.

Could you explain why buffering in the whole session, copying it into the decoder buffer in the same size increments as hello_video, does not work, whereas hello_video happily plays the same data dumped to a file?
Last edited by timmehboy on Wed May 01, 2013 9:47 am, edited 1 time in total.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault

Thu Apr 11, 2013 9:17 am

Hmm. No, I can't.

timmehboy
Posts: 22
Joined: Tue Apr 09, 2013 10:47 am

Re: OMX_EmptyThisBuffer segfault

Thu Apr 11, 2013 9:23 am

Update: I have finally got it working: I had to buffer 2MB of data, and then stream it out into the buffer (my previous buffering had a bug).
It seems OpenMAX requires more data upfront than some software decoders, for stream information. Next step from here is minimising the buffering and streamlining the operation from here.

Thanks a lot dickon for the help! :)
Last edited by timmehboy on Wed May 01, 2013 9:48 am, edited 1 time in total.

dickon
Posts: 468
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OMX_EmptyThisBuffer segfault/no PortSettingsChanged [SOL

Thu Apr 11, 2013 9:48 pm

Excellent. I'm happy it works.

Return to “OpenMAX”