gumbi
Posts: 1
Joined: Sat Dec 01, 2018 8:06 pm

Using H.264 Intra Refresh Modes

Wed Dec 05, 2018 11:54 pm

I've been trying to get the intra refresh options to work, without much success. Was wondering if anyone else was able to get them to work. Here's the command I'm using:

Code: Select all

raspivid -l -o tcp://0.0.0.0:3333 -n -t 0 -md 4 -fps 10 -if cyclic 
and on the receiving PC to render the macroblock types:

Code: Select all

ffplay -debug vis_mb_type tcp://<RPi IP address>:3333
(Note that FFmpeg older than 4.0 is required, because in 4.0 they removed the vis_mb_type option)

Firmware version:

Code: Select all

pi@raspberrypi:~ $ vcgencmd version
Nov  4 2018 16:35:17 
Copyright (c) 2012 Broadcom
version ed5baf9520a3c4ca82ba38594b898f0c0446da66 (clean) (release)
The P frame looks like this:
Image

While the I frame looks like this:
Image

(Legend for the colors: https://trac.ffmpeg.org/wiki/Debug/Macr ... blockTypes)

So I guess if cyclic was working, I would expect to see a row or column of red macroblocks sweeping across the frame as the video is playing.

I'm hoping that I could get it to work, since the connection I'm using is lossy and packets could be lost, and unless there's motion in the video, that part of the frame won't be "refreshed" so to speak

Thanks in advance

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Using H.264 Intra Refresh Modes

Thu Dec 06, 2018 12:13 pm

Cyclic refresh mode requires more configuration, and raspivid isn't doing that (there's a commented out line related).

In the MMAL_PARAMETER_VIDEO_INTRA_REFRESH_T structure is cir_mbs. It maps directly to the OMX IL structure OMX_VIDEO_PARAM_INTRAREFRESHTYPE nCirMBs. From the spec: "nCirMBs is the number of consecutive macroblocks to be coded as intra when
cyclic intra-refresh (CIR) is enabled.".
Default I suspect is 0 which will do next to nothing.

The requirement within the codec is ((nCirMBS % mbwidth) == 0) && (nCirMBs > mbwidth). So the commented out line in raspivid.c

Code: Select all

         //if (state->intra_refresh_type == MMAL_VIDEO_INTRA_REFRESH_CYCLIC_MROWS)
         //   param.cir_mbs = 10;
probably wants to be

Code: Select all

         if (state->intra_refresh_type == MMAL_VIDEO_INTRA_REFRESH_CYCLIC_MROWS || state->intra_refresh_type == MMAL_VIDEO_INTRA_REFRESH_CYCLIC)
            param.cir_mbs = VCOS_ALIGN_UP(state->width, 16) >> 4;
to make it a macroblock row.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Using H.264 Intra Refresh Modes

Thu Dec 06, 2018 12:25 pm

Trying that and it works, although not quite in the way I expected. I do see a horizontal bar scrolling down the frame. Drop the frame rate and it becomes more obvious.
What I wasn't expecting was that it still encodes a full I-frame every intra-period. You can set the intra-period to 0 so that it only send a full I-frame at the beginning of the stream and never again. The cyclic refresh will provide an update to all pixels, however no subsequent frame will ever be an IDR frame, so there is no option for seeking in the stream.

Reverting my changes it still seems to work correctly, so perhaps it does take some sensible defaults, but you need to disable the full I-frame.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”