Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

How tunnel of openmax work?

Sat Nov 03, 2018 11:24 am

Hello guys.

My English is not good so I am sorry for this. I am new on openmax. I work with hello_video code on Rpi3. I couldn't understand some points.
1- Why openmax use tunnel ? What is that mean?
2- I saw set port(format.nPortIndex=130). Is it can be change? I tried random number for this but gave error.
3- Openmax just work h.264 format on Rpi(I think). I tried different format but I did not accomplish. I think Mpeg4 doesn't supported on Rpi I saw this viewtopic.php?f=70&t=218237 and I can not online converter website for convert from mp4 to h264. Please help me this title. Thanks :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How tunnel of openmax work?

Sun Nov 04, 2018 8:50 pm

Faruk wrote:
Sat Nov 03, 2018 11:24 am
Hello guys.

My English is not good so I am sorry for this. I am new on openmax. I work with hello_video code on Rpi3. I couldn't understand some points.
1- Why openmax use tunnel ? What is that mean?
Best to read the spec - https://www.khronos.org/registry/omxil/ ... cation.pdf
You can either run the components with the data always going to and from the client, or you can tunnel components together so that the pipeline runs autonomously.
Faruk wrote:2- I saw set port(format.nPortIndex=130). Is it can be change? I tried random number for this but gave error.
Read the spec.
Every component has ports. You can query what ports and the numbers associated via OMX_IndexParamAudioInit, OMX_IndexParamVideoInit, OMX_IndexParamImageInit, and OMX_IndexParamOtherInit.
Or just refer to the documentation provided at https://github.com/raspberrypi/firmware ... components, with an HTML version available via http://www.jvcref.com/files/PI/document ... omponents/
Faruk wrote:3- Openmax just work h.264 format on Rpi(I think). I tried different format but I did not accomplish. I think Mpeg4 doesn't supported on Rpi I saw this viewtopic.php?f=70&t=218237 and I can not online converter website for convert from mp4 to h264. Please help me this title. Thanks :)
video_decode supports MJPEG, H264, MPEG4, H263, and you can buy licences for MPEG-2 and VC1.
video_encode only support MJPEG and H264.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Mon Nov 05, 2018 4:57 am

Thanks for your reply :)

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Tue Nov 06, 2018 12:25 pm

Additionality I have two question. I am sorry. I couldn't understand 'status' on hello_video example. It is return be result of 'video_decode_test' functionally. But I couldn't understand.
Other question this: How can I change screen size. I don't want fullscreen. I tried 'OMX_FRAMESIZETYPE' function but I couldn't do. Can I use 500x500 px screen.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How tunnel of openmax work?

Tue Nov 06, 2018 2:28 pm

Faruk wrote:
Tue Nov 06, 2018 12:25 pm
Additionality I have two question. I am sorry. I couldn't understand 'status' on hello_video example. It is return be result of 'video_decode_test' functionally. But I couldn't understand.
status is the value returned to the calling process so that it can do error checking.
It is assigned a few apparently random values, I suspect mainly so that the caller can tell where it failed.
Faruk wrote:Other question this: How can I change screen size. I don't want fullscreen. I tried 'OMX_FRAMESIZETYPE' function but I couldn't do. Can I use 500x500 px screen.
OMX_IndexConfigDisplayRegion
See the docs http://www.jvcref.com/files/PI/document ... playRegion
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Wed Nov 07, 2018 10:11 am

Hello again I tried 'OMX_CONFIG_DISPLAYREGIONTYPE' but its didn't work. I remarked that my added code. My code is:

Code: Select all

// Video deocode demo using OpenMAX IL though the ilcient helper library

/*typedef unsigned long OMX_U32*/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include "bcm_host.h"
#include "ilclient.h"
#include "OMX_Video.h"
#include "OMX_Other.h"



static int video_decode_test(char *filename)
{
   OMX_VIDEO_PARAM_PORTFORMATTYPE format; 
   OMX_TIME_CONFIG_CLOCKSTATETYPE cstate;
   OMX_CONFIG_DISPLAYREGIONTYPE region;      
   COMPONENT_T *video_decode = NULL, *video_scheduler = NULL, *video_render = NULL, *clock = NULL;  
   COMPONENT_T *list[5];
   TUNNEL_T tunnel[4];
   ILCLIENT_T *client;       
   FILE *in;          
   int status = 0;
   unsigned int data_len = 0;


   memset(list, 0, sizeof(list));     
   memset(tunnel, 0, sizeof(tunnel));    


   if((in = fopen(filename, "rb")) == NULL)
      return -2;

   if((client = ilclient_init()) == NULL)      
   {
      fclose(in);
      return -3;
   }

   if(OMX_Init() != OMX_ErrorNone)      
   {
      ilclient_destroy(client);      
      fclose(in);
      return -4;
   }
   /*******I added this section*******/
   memset(&region, 0, sizeof(region));
   region.nSize = sizeof(OMX_CONFIG_DISPLAYREGIONTYPE);
   region.nVersion.nVersion = OMX_VERSION;
   region.nPortIndex = 90;
   region.src_rect.x_offset =0;
   region.src_rect.y_offset = 0;
   region.src_rect.width = 550;
   region.src_rect.height = 550;
   region.src_rect.x_offset =0;
   region.src_rect.y_offset = 0;
   region.src_rect.width = 550;
   region.src_rect.height = 550;
   region.fullscreen =OMX_FALSE;
   region.noaspect = OMX_TRUE;



    region.set = (OMX_DISPLAYSETTYPE)(OMX_DISPLAY_SET_SRC_RECT | OMX_DISPLAY_SET_DEST_RECT | OMX_DISPLAY_SET_FULLSCREEN | OMX_DISPLAY_SET_NOASPECT);
if(OMX_SetParameter(ILC_GET_HANDLE(video_render), OMX_IndexConfigDisplayRegion, &region) != OMX_ErrorNone) {
  status = -14;
}

/*****************************************/


   // create video_decode
   if(ilclient_create_component(client, &video_decode, "video_decode", ILCLIENT_DISABLE_ALL_PORTS | ILCLIENT_ENABLE_INPUT_BUFFERS) != 0)
      status = -14;
   list[0] = video_decode;

   // create video_render
   if(status == 0 && ilclient_create_component(client, &video_render, "video_render", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[1] = video_render;

   // create clock
   if(status == 0 && ilclient_create_component(client, &clock, "clock", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[2] = clock;



   memset(&cstate, 0, sizeof(cstate));
   cstate.nSize = sizeof(cstate);        
   cstate.nVersion.nVersion = OMX_VERSION;        
   cstate.eState = OMX_TIME_ClockStateWaitingForStartTime;    
   cstate.nWaitMask = 1;            //
   if(clock != NULL && OMX_SetParameter(ILC_GET_HANDLE(clock), OMX_IndexConfigTimeClockState, &cstate) != OMX_ErrorNone)  
      status = -13;      
   // create video_scheduler
   if(status == 0 && ilclient_create_component(client, &video_scheduler, "video_scheduler", ILCLIENT_DISABLE_ALL_PORTS) != 0)   
      status = -14;
   list[3] = video_scheduler;

   set_tunnel(tunnel, video_decode, 131, video_scheduler, 10);      
   set_tunnel(tunnel+1, video_scheduler, 11, video_render, 90);   
   set_tunnel(tunnel+2, clock, 80, video_scheduler, 12);        

   // setup clock tunnel first
   if(status == 0 && ilclient_setup_tunnel(tunnel+2, 0, 0) != 0)        
      status = -15;
   else
      ilclient_change_component_state(clock, OMX_StateExecuting);   

   if(status == 0)
      ilclient_change_component_state(video_decode, OMX_StateIdle);  
   memset(&format, 0, sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE));
   format.nSize = sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE);       
   format.nVersion.nVersion = OMX_VERSION;                     
   format.nPortIndex = 130;                                     
   format.eCompressionFormat = OMX_VIDEO_CodingAVC;             

   if(status == 0 &&
      OMX_SetParameter(ILC_GET_HANDLE(video_decode), OMX_IndexParamVideoPortFormat, &format) == OMX_ErrorNone &&         
      ilclient_enable_port_buffers(video_decode, 130, NULL, NULL, NULL) == 0)        
   {
      OMX_BUFFERHEADERTYPE *buf;     
      int port_settings_changed = 0;
      int first_packet = 1;

      ilclient_change_component_state(video_decode, OMX_StateExecuting);       
      while((buf = ilclient_get_input_buffer(video_decode, 130, 1)) != NULL)      
      {
         // feed data and wait until we get port settings changed
         unsigned char *dest = buf->pBuffer;  

         data_len += fread(dest, 1, buf->nAllocLen-data_len, in);   
        if(port_settings_changed == 0 &&
            ((data_len > 0 && ilclient_remove_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1) == 0) ||    
             (data_len == 0 && ilclient_wait_for_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1,
                                                       ILCLIENT_EVENT_ERROR | ILCLIENT_PARAMETER_CHANGED, 10000) == 0)))
         {
            port_settings_changed = 1;      

            if(ilclient_setup_tunnel(tunnel, 0, 0) != 0)
            {
               status = -7;
               break;
            }

            ilclient_change_component_state(video_scheduler, OMX_StateExecuting);        
            // now setup tunnel to video_render
            if(ilclient_setup_tunnel(tunnel+1, 0, 1000) != 0)
            {
               status = -12;
               break;
            }

            ilclient_change_component_state(video_render, OMX_StateExecuting);       
         }
         if(!data_len)
            break;

         buf->nFilledLen = data_len;
         data_len = 0;

         buf->nOffset = 0;
         if(first_packet)
         {
            buf->nFlags = OMX_BUFFERFLAG_STARTTIME;
            first_packet = 0;
         }
         else
            buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN;

         if(OMX_EmptyThisBuffer(ILC_GET_HANDLE(video_decode), buf) != OMX_ErrorNone)
         {
            status = -6;
            break;
         }
      }

      buf->nFilledLen = 0;
      buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN | OMX_BUFFERFLAG_EOS;

      if(OMX_EmptyThisBuffer(ILC_GET_HANDLE(video_decode), buf) != OMX_ErrorNone) 
         status = -20;

      // wait for EOS from render
      ilclient_wait_for_event(video_render, OMX_EventBufferFlag, 90, 0, OMX_BUFFERFLAG_EOS, 0,
                              ILCLIENT_BUFFER_FLAG_EOS, -1);

      // need to flush the renderer to allow video_decode to disable its input port
      ilclient_flush_tunnels(tunnel, 0);

   }

   fclose(in);

   ilclient_disable_tunnel(tunnel);
   ilclient_disable_tunnel(tunnel+1);
   ilclient_disable_tunnel(tunnel+2);
   ilclient_disable_port_buffers(video_decode, 130, NULL, NULL, NULL);
   ilclient_teardown_tunnels(tunnel);

   ilclient_state_transition(list, OMX_StateIdle);
   ilclient_state_transition(list, OMX_StateLoaded);

   ilclient_cleanup_components(list);

   OMX_Deinit();

   ilclient_destroy(client);
   return status;
}

int main (int argc, char **argv)
{
   if (argc < 2) {
      printf("Usage: %s <filename>\n", argv[0]);
      exit(1);
   }
   bcm_host_init();
   return video_decode_test(argv[1]);
}


I searched anyway for this bu I couldn't. My video screen size doesn't change. How can I fix this?
In addition I saw 'OMX_FRAMESIZETYPE' on OpenmAX IL spefication. Is it benefit for change screen size ?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How tunnel of openmax work?

Wed Nov 07, 2018 10:23 am

Ahem

Code: Select all

   region.src_rect.x_offset =0;
   region.src_rect.y_offset = 0;
   region.src_rect.width = 550;
   region.src_rect.height = 550;
   region.src_rect.x_offset =0;
   region.src_rect.y_offset = 0;
   region.src_rect.width = 550;
   region.src_rect.height = 550;
   region.fullscreen =OMX_FALSE;
   region.noaspect = OMX_TRUE;
You don't set up dest_rect, so it's left at 0,0,0,0 from the memset. That will be treated as invalid, and IIRC mapped to fullscreen.

src_rect crops the incoming video buffer, which probably isn't what you want to do.
dest_rect is where on the screen you wish to put the image.

The same call is used in omxplayer - https://github.com/popcornmix/omxplayer ... o.cpp#L835
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Wed Nov 07, 2018 11:15 am

OK. I success this. I think I wrote function but I add it wrong place and I forgot to add config settings in my function. I share my code which maybe it is true. This:

Code: Select all

  // create video_render
   if(status == 0 && ilclient_create_component(client, &video_render, "video_render", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;


      OMX_CONFIG_DISPLAYREGIONTYPE configDisplay;
memset(&configDisplay, 0, sizeof configDisplay);
configDisplay.nSize = sizeof configDisplay;
configDisplay.nVersion.nVersion = OMX_VERSION;
configDisplay.nPortIndex = 90;
configDisplay.fullscreen = OMX_FALSE;
configDisplay.noaspect = OMX_TRUE;
configDisplay.set = (OMX_DISPLAYSETTYPE)(OMX_DISPLAY_SET_TRANSFORM | OMX_DISPLAY_SET_LAYER | OMX_DISPLAY_SET_NUM | OMX_DISPLAY_SET_DEST_RECT | OMX_DISPLAY_SET_SRC_RECT | OMX_DISPLAY_SET_FULLSCREEN | OMX_DISPLAY_SET_NOASPECT) ;  // set edilen ayarlarin buraya girilmesi gerekiyor.
configDisplay.num = 1;
configDisplay.layer = 1;//(1<<15)-1;
configDisplay.transform = 270;
configDisplay.dest_rect.x_offset  = 50;
   configDisplay.dest_rect.y_offset  = 50;
   configDisplay.dest_rect.width     = 1280;
   configDisplay.dest_rect.height    = 720;
   configDisplay.src_rect.x_offset   = 0;
   configDisplay.src_rect.y_offset   = 0;
   configDisplay.src_rect.width      = 1280;
   configDisplay.src_rect.height     = 720;

if (OMX_SetConfig(ILC_GET_HANDLE(video_render), OMX_IndexConfigDisplayRegion, &configDisplay) != OMX_ErrorNone)
status = -15;

   list[1] = video_render;

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Wed Nov 07, 2018 3:54 pm

Hello again. I so much ask today. Sorry. I want run mpeg-4 video format. I tried change this section on my code:

Code: Select all

 format.eCompressionFormat = OMX_VIDEO_CodingAVC;     
but openmax did not decode mpeg-4. My argument name is 'A.mp4' its format look like be mpeg-4 on its properties. I set argument and run but I couldn't screen. What's wrong?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How tunnel of openmax work?

Wed Nov 07, 2018 4:03 pm

Faruk wrote:
Wed Nov 07, 2018 3:54 pm
Hello again. I so much ask today. Sorry. I want run mpeg-4 video format. I tried change this section on my code:

Code: Select all

 format.eCompressionFormat = OMX_VIDEO_CodingAVC;     
but openmax did not decode mpeg-4. My argument name is 'A.mp4' its format look like be mpeg-4 on its properties. I set argument and run but I couldn't screen.
Don't get confused between the MPEG-4 part 2 video codec and the MPEG-4 part 14 container format (see also MPEG 4 parts 12 and 15).
For further confusion, H264 is also known as MPEG-4 part 10

video_decode will only ever take an elementary stream. To process a container format you want to use something like libavcodec (either from libav or ffmpeg) to extract the elementary stream first.

Should the video codec be MPEG-4 part 2, then you want to set

Code: Select all

format.eCompressionFormat = OMX_VIDEO_CodingMPEG4;
OMX_VIDEO_CodingAVC is correct for H264. OMX_VIDEO_CodingH263 is correct for H263, and you can buy the licences for OMX_VIDEO_CodingMPEG2 and OMX_VIDEO_CodingWMV.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Thu Nov 08, 2018 11:47 am

So how can I create elementary stream? I saw ffmpeg but I couldn't it. But I achieve againts. I convert test.h264 to .mp4 file with raspivid. Also can I decode video directly?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How tunnel of openmax work?

Thu Nov 08, 2018 11:57 am

From a command line,
ffmpeg -i input.mp4 -c:v copy -bsf h264_mp4toannexb output.h264

There is no container parsing on the GPU - your application has to do it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Thu Nov 08, 2018 12:34 pm

Thanks for your help :). It working so great. I tried like this but video seemed distorted. Actually some file still broke but I can see different video now :)

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Fri Nov 09, 2018 11:40 am

I achieved decode video but just I can see display, I couldn't audio. I did ffmpeg process for audio and I converted from .mp4 file to .aac file. Openmax can decoding to .aac file with 'OMX_AUDIO_CodingAAC'. I try doing decode audio but I don't know what will result. I don't sure both video and audio will run same time. Is it possible? Is run same time both audio and display? And that time how can I give two arguments(video.h264 and audio.aac) to codeblocks? Thanks your time.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6336
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How tunnel of openmax work?

Fri Nov 09, 2018 5:19 pm

Faruk wrote:
Fri Nov 09, 2018 11:40 am
I achieved decode video but just I can see display, I couldn't audio. I did ffmpeg process for audio and I converted from .mp4 file to .aac file. Openmax can decoding to .aac file with 'OMX_AUDIO_CodingAAC'. I try doing decode audio but I don't know what will result. I don't sure both video and audio will run same time. Is it possible? Is run same time both audio and display? And that time how can I give two arguments(video.h264 and audio.aac) to codeblocks? Thanks your time.
You would be describing omxplayer there.
If trying to do audio and video simultaneously then you MUST add container parsing to your application. Timestamp information is only present in the container and not the elementary streams. OMX has various techniques for providing A/V sync (eg clock components) - read the spec.

OpenMax IL is a swine of an API - have you looked at using MMAL instead? It was written to try and solve some of the nastiness of IL. There is a demo app for a/v playback at https://github.com/raspberrypi/userland ... s/mmalplay, although I must confess to not having run it recently.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Sat Nov 10, 2018 5:15 am

My work looking so difficult. I will search OMX and MMAL. Thank you :)

Faruk
Posts: 54
Joined: Sat Sep 08, 2018 7:34 am

Re: How tunnel of openmax work?

Mon Nov 12, 2018 8:42 pm

Hello again. I wonder one things. Can I use gstreamer for this job? I saw some comment gstreamer but I don't sure how can it run? I want decode video and audio like omx. Because I searched MMAL and container parse but it is look difficult and I can't find documentation according to me. I am pretty this job so sometimes I can't understand it.

Return to “OpenMAX”