OMX_Video_ControlRateVariable

[mdoe]

Hi rpi forum,

As a bit of background I've been playing with the hardware video encoder in conjunction with an rtmp service, for the most part this has gone well, but I've been unable to constrain the bitrate of the video output to any real degree of success. I am not a developer of any sort, so please excuse any glaring errors in my reasoning going forward.

At first I thought this was down to a command line error on my part, using ffmpeg, but on closer inspection to the source code I suspect the use of OMX_Video_ControlRateVariable is the cause of the wide variation of its resulting output. I've looked at the OpenMAX specification document and found that the following rate control types are defined;

OMX_Video_ControlRateDisable,
OMX_Video_ControlRateVariable,
OMX_Video_ControlRateConstant,
OMX_Video_ControlRateVariableSkipFrames,
OMX_Video_ControlRateConstantSkipFrames,
OMX_Video_ControlRateMax

At this point I went back to first principles and changed the hello_pi/hello_encode source to use OMX_Video_ControlRateConstant. Once compiled it fails to run, hanging and gives no error. I've tried all rate control types and the only ones that work are OMX_Video_ControlRateVariable and OMX_Video_ControlRateVariableSkipFrames.

At this point I thought I'd ask if I'm barking up the wrong tree entirely!

Thanks.

M

[6by9]

Constant bitrate does not work with H264 high or main profiles, but works fine on base. It's enabling CABAC that is incompatible with constant bitrate on Pi.

It looks like a couple of the various discussions were
https://github.com/raspberrypi/firmware/issues/254
https://github.com/raspberrypi/firmware/issues/250

[mdoe]

6by9, thank you for taking the time to find and post those links, my Google-fu failed me this time. That explains why the output was always H264 High 4.0 and I couldn't work out how to change it.

So essentially, the code needs to be changed to instruct the encoder to use a baseline profile, that way CABAC isn't used, allowing CBR to be achieved, until them, CBR won't work.

That coding is beyond my skill, but I hope this post points someone in that direction at a later date.

Best I find a new project for my little Pi!

M

[6by9]

If you're playing with hello_encode, then add the block

Code: Select all

   OMX_VIDEO_PARAM_AVCTYPE avcParams;
   // set current bitrate to 1Mbit
   memset(&avcParams, 0, sizeof(OMX_VIDEO_PARAM_AVCTYPE));
   avcParams.nSize = sizeof(OMX_VIDEO_PARAM_AVCTYPE);
   avcParams.nVersion.nVersion = OMX_VERSION;
   r = OMX_GetParameter(ILC_GET_HANDLE(video_encode),
                       OMX_IndexParamVideoAvc, &avcParams);
   avcParams.eProfile = OMX_VIDEO_AVCProfileBase;
   r = OMX_SetParameter(ILC_GET_HANDLE(video_encode),
                       OMX_IndexParamVideoAvc, &avcParams);
   if (r != OMX_ErrorNone) {
      printf
        ("%s:%d: OMX_SetParameter() for level/profile for video_encode port 201 failed with %x!\n",
         __FUNCTION__, __LINE__, r);
      exit(1);
  }

to around the same place as the rate control mode is set.
(code not tested, but should give you the right shape).

[mdoe]

6by9, very kind of you to offer some code, that's a great help, thank you.

I've been tinkering this morning/afternoon and have the following code working in the hello_encode

Code: Select all

   OMX_VIDEO_PARAM_AVCTYPE avcParams;
   // set current bitrate to 1Mbit
   memset(&avcParams, 0, sizeof(OMX_VIDEO_PARAM_AVCTYPE));
   avcParams.nSize = sizeof(OMX_VIDEO_PARAM_AVCTYPE);
   avcParams.nVersion.nVersion = OMX_VERSION;
   avcParams.nPortIndex = 201;
   r = OMX_GetParameter(ILC_GET_HANDLE(video_encode),
                       OMX_IndexParamVideoAvc, &avcParams);
   avcParams.eProfile = OMX_VIDEO_AVCProfileBaseline;
   avcParams.eLevel = OMX_VIDEO_AVCLevel32;
   r = OMX_SetParameter(ILC_GET_HANDLE(video_encode),
                       OMX_IndexParamVideoAvc, &avcParams);
   if (r != OMX_ErrorNone) {
      printf
        ("%s:%d: OMX_SetParameter() for level/profile for video_encode port 201 failed with %x!\n",
         __FUNCTION__, __LINE__, r);
      exit(1);
  }

Setting bEntropyCodingCABAC = OMX_FALSE; has zero effect, as suggested in the above links, you can't turn it off in any other way than encoding in Baseline.

I did find reference to "OMX_IndexConfigBrcmVideoH264DisableCABAC" here I take it that's completely unrelated?

M

[6by9]

Looking at the firmware source, High profile and bEntropyEncodingCABAC=false will fail, otherwise the setting is ignored. Reading it back will return false for baseline, true for others.

OMX_IndexConfigBrcmVideoH264DisableCABAC is passed to the codec to supposedly disable CABAC. Why it is a separate parameter I have no idea. It is passed to the codec so has a chance of working, but I couldn't say for certain.

[jamesh]

Looking at the firmware source, High profile and bEntropyEncodingCABAC=false will fail, otherwise the setting is ignored. Reading it back will return false for baseline, true for others.

OMX_IndexConfigBrcmVideoH264DisableCABAC is passed to the codec to supposedly disable CABAC. Why it is a separate parameter I have no idea. It is passed to the codec so has a chance of working, but I couldn't say for certain.

I'm stretching my memory here, but I think I tried that disable CABAC using that option and I could not get it to work - had to go to a lower profile.

[mdoe]

I've just noticed that the OMX_Video_ControlRateConstant is working even when I set the profile to main in the hello_encode, it's not hanging like before, odd.

CABAC is probably nothing for me worry about, the ability to switch profiles was the most important thing, if that's working with CBR, even better

Thanks again for your input.

[mdoe]

Hello again

Thanks to your help I've spent the last few days playing around with the parameters in OMX_VIDEO_PARAM_AVCTYPE, I've come to the conclusion constant rate control doesn't appear to work on any profile/level, you can enable it without the encoding process stalling, but it has no effect on the encoded output. There also seems to be very little difference between variable rate control and no rate control. My target bitrate is 3200Kbps, but I see regular spikes of 6000+. Only a static image sits at the desired target rate, which I guess is to be expected.

This did lead me on to try some other AVC parameters, to try and look at what the I, P, and B-frames might be doing, it seems I can't configure Bframes , when I do, i get the following error

OMX_SetParameter() for level/profile for video_encode port 201 failed with 80001005

Code: Select all

   OMX_VIDEO_PARAM_AVCTYPE avcParams;
   memset(&avcParams, 0, sizeof(OMX_VIDEO_PARAM_AVCTYPE));
   avcParams.nSize = sizeof(OMX_VIDEO_PARAM_AVCTYPE);
   avcParams.nVersion.nVersion = OMX_VERSION;
   avcParams.nPortIndex = 201;
//   avcParams.bEntropyCodingCABAC = OMX_FALSE;
   r = OMX_GetParameter(ILC_GET_HANDLE(video_encode),
                       OMX_IndexParamVideoAvc, &avcParams);
   avcParams.eProfile = OMX_VIDEO_AVCProfileHigh;
   avcParams.eLevel = OMX_VIDEO_AVCLevel42;
   avcParams.nPFrames = 3;
   avcParams.nBFrames = 1;
   r = OMX_SetParameter(ILC_GET_HANDLE(video_encode),
                       OMX_IndexParamVideoAvc, &avcParams);
   if (r != OMX_ErrorNone) {
      printf
        ("%s:%d: OMX_SetParameter() for level/profile for video_encode port 201 failed with %x!\n",
         __FUNCTION__, __LINE__, r);
      exit(1);
  }

The encoder will only run if I set the BFrames to 0, then the PFrames can be set to 1 or more.

Any ideas where I might be going wrong here?

Thanks.

[6by9]

B frames are not supported. That's an easy one.

With rate control generally please remember that this is a real time, single pass, encode. The quantisation factors all have to be chosen before the frame is encoded based on previous frames and the target bitrate. If you get a major scene change then the predicted frame size will be off, and it has to compensate over the next few frames, in extreme circumstances by dropping them totally.
Constant bitrate should be constant over a small number of frames. That's the best that can be expected on a single pass encode.

[mdoe]

That is an easy one I take it they disabled in the hardware? I was hoping it would afford me some more efficiency, but no matter. Is there a list of all the options that are disabled?

I've discovered that the ffmpeg implementation adjusted the nPFrames by -1 relative to the defined GOP size, which is slightly odd as you'd end up with the key frame drifting out of what you've set, I removed the -1 and it seems to be steady over 30 mins of testing.

I think this is as close to the required specification as I can get now, taking into account the single pass encode, it's fairly close.

Thank you for the responses, it continues to be an enjoyable learning experience.

M