mccp
Posts: 26
Joined: Thu Dec 01, 2011 8:53 am
Contact: Website

Re: Hardware-assisted H.264 video encoding

Thu Feb 09, 2012 1:41 pm

First of all I have no desire to distract anyone from the task of releasing the boards that are about to be made so please ignore the following if it is distracting .

We report sales of MPEG-2 and H.264 encoders/decoders to MPEG-LA (who may or may not be evil, but they are certainly persistent). I believe that we are obliged to pay $2.00 for an MPEG-2 encoder and the same for a decoder (i.e. $4.00 for a transcoder).

I can't find the agreements at the moment, but I believe that we only report the quantities of H.264 encoder/decoders as we are below the annual sales threshold that triggers royalty payments.

I have just looked at the sample agreement on the MPEG-LA site and it seems that they consider that a unit is: "(a decoder, encoder, or product consisting of one decoder and one encoder = “unit”)".

License fees are 0-100,000 units per annum $0.00, 100,000 to 5,000,000 per annum $0.20 and >5m per annum is $0.10 per unit.

So it may be that MPEG-LA patent portfolio license is not the culprit for the lack of H.264 support.

We found that the fees for AAC were much more painful than H.264 - this is managed by Via Licensing. They charge $0.98 for 1-500,000 units for hardware encoders/decoders or $0.48 for SW decoders & $0.98 for SW encoders. Plus there's a $15K up front payment required.

I would very much like to see an encoding variant of the Raspberry Pi as I can see many useful applications, but I can easily understand that this is not going to be at the forefront of the organisation's plans if it adds significant costs or administration. I hope at least that it will be added to a wishlist for future releases to be considered when there is time.

I also hope that the bleating about evil licensors and charity status, etc. will stop as it must be clear that the Raspberry Pi organisation is selling products to end-users that are required to be licensed by the laws of various countries. Unless you've had the pleasure of dealing with the lawyers employed by organisations such as MPEG-LA/VIALicensing/Fraunhofer your opinion on the subject may not be terribly relevant .

9+9 (personal best!)

waveform
Posts: 28
Joined: Wed Nov 30, 2011 6:10 pm

Re: Hardware-assisted H.264 video encoding

Sat Feb 11, 2012 12:10 am

Hey mccp!

Thanks a lot for sharing your post and giving some insight.

It would still be great though if there was some discounted rate offered for the educational Pi units.

pyke369
Posts: 15
Joined: Sat Feb 04, 2012 8:54 pm

Re: Hardware-assisted H.264 video encoding

Sat Feb 11, 2012 11:32 am

mccp said:

I have just looked at the sample agreement on the MPEG-LA site and it seems that they consider that a unit is: "(a decoder, encoder, or product consisting of one decoder and one encoder = “unit”)".
License fees are 0-100,000 units per annum $0.00, 100,000 to 5,000,000 per annum $0.20 and >5m per annum is $0.10 per unit.


According to this statement, it seems the licence paid for the decoder also covers the encoding portion of the "unit". So why wouldn't Broadcom unlock the feature as part of the initial release for the R.Pi? Or am I reading this wrong?

Cheers,

Pierre-Yves

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23629
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware-assisted H.264 video encoding

Sat Feb 11, 2012 11:35 am

No, I checked this - we only pay for a decode licence. Encode will add more cost.

I also agree with the posters assessment of AAC- that is really very expensive for such a crappy little codec - much more than H264. Which is why it is not include on the GPU and users will need to install Arm side AAC decode.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
Gert van Loo
Posts: 2484
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Hardware-assisted H.264 video encoding

Sat Feb 11, 2012 1:49 pm

As to a discount for the Pi: I don't think that would be feasible even if they wanted as the boards do not all go to charity/education. It has been clearly stated that anybody can buy the boards and sell them on. So giving a discount or free licenses would cause false competition.

pyke369
Posts: 15
Joined: Sat Feb 04, 2012 8:54 pm

Re: Hardware-assisted H.264 video encoding

Sat Feb 11, 2012 2:05 pm


JamesH said:


No, I checked this - we only pay for a decode licence. Encode will add more cost.



Bummer

According to this http://www.mpegla.com/main/pro.....ummary.pdf (probably what mccp was referring to), there is nothing to pay under 100k units / year. Above, it's an extra $0.20 but it opens interesting options for video-conferencing and video streaming.

Anyway, when you say "we only pay", who's "we"? The R.Pi foundation? Or Broadcom?

Is the decode licensing price already included by Broadcom when it sells the BCM2835 to the R.Pi foundation?

(sorry for being insistent, I just want to know the gory details )

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23629
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware-assisted H.264 video encoding

Sat Feb 11, 2012 6:24 pm

The foundation pays the licence fees. Broadcom just provides the GPU software to match the licences purchased.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Minthos
Posts: 1
Joined: Tue Apr 24, 2012 7:22 am

Re: Hardware-assisted H.264 video encoding

Tue Apr 24, 2012 8:43 am

So I assume the added cost is for the software from Broadcom, not the licence from MPEG-LA?

Would it be possible to add hardware encoding in a C version of the device? Perhaps with other additional features as well?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23629
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware-assisted H.264 video encoding

Tue Apr 24, 2012 8:58 am

No, there is an addition licence cost that needs to be paid to MPEG-LA. If that cost is paid, Broadcom will provide the appropriate encoder software on the GPU blob.

I'm not sure what you mean by C version.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

mole125
Posts: 228
Joined: Tue Jan 10, 2012 2:01 pm

Re: Hardware-assisted H.264 video encoding

Tue Apr 24, 2012 9:01 am

Minthos said:


So I assume the added cost is for the software from Broadcom, not the licence from MPEG-LA?

Would it be possible to add hardware encoding in a C version of the device? Perhaps with other additional features as well?


No, from what has been said in numerous other posts, the added cost is for the licences from MPEG-LA. If the the foundation can prove to Broadcom they have got the necessary patent license then Broadcom can provide the keys to allow the other hardware decoders/encoders to be used.  The challenge for the foundation is

a) the cost and effort to negotiate and administer the license

b) convincing the MPEG-LA that they have sufficient technical measures to stop them from being abused - ie people not taking the model C bootcode.bin and using it on a model B without paying the license.

pyke369
Posts: 15
Joined: Sat Feb 04, 2012 8:54 pm

Re: Hardware-assisted H.264 video encoding

Tue Apr 24, 2012 7:13 pm

This is so depressing and sad

It would have been a lovely feature on the Pi (I was dreaming of a Pi H.264 encoding cluster, but I'm biased toward video encoding ... ^_^)

User avatar
jbeale
Posts: 3491
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Hardware-assisted H.264 video encoding

Tue Apr 24, 2012 7:26 pm

How about Ogg Theora? http://en.wikipedia.org/wiki/Theora

Three magic words: "no licensing fees" (encode or decode).

Theora has seen limited use due to a number of reasons, probably much due to H.264 being already widely adopted. But if the RasPi supported Theora encode and decode, things might change.

"…the differences in quality, bitrate and file size between a YouTube H.264 video and a transcoded Ogg video file are negligible."
from http://en.wikipedia.org/wiki/Theora which is referencing http://www.freesoftwaremagazin.....la_firefox

Max

Re: Hardware-assisted H.264 video encoding

Tue Apr 24, 2012 10:06 pm

jbeale said:


How about Ogg Theora? http://en.wikipedia.org/wiki/Theora


Or Google's WebM? http://en.wikipedia.org/wiki/Webm

Advantage being that the encoded files could be played back by Firefox, Chrome and Opera without additional software. (See browser list at Youtube's HTML 5 beta page: http://www.youtube.com/html5 )

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23629
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware-assisted H.264 video encoding

Wed Apr 25, 2012 7:21 am

Max said:


jbeale said:


How about Ogg Theora? http://en.wikipedia.org/wiki/Theora


Or Google's WebM? http://en.wikipedia.org/wiki/Webm

Advantage being that the encoded files could be played back by Firefox, Chrome and Opera without additional software. (See browser list at Youtube's HTML 5 beta page: http://www.youtube.com/html5 )


There is an implementation of WebM on the GPU available, but it's software only i.e. just uses the vector core rather than using the HW acceleration blocks which are tuned to H264. So its only up to about 20fps or something. Can't remember the exact figure.

Not sure about Ogg Theora though.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

mr_sunshine
Posts: 1
Joined: Sun Aug 28, 2011 11:09 am

Re: Hardware-assisted H.264 video encoding

Tue May 01, 2012 1:05 am

mole125 said:


a) the cost and effort to negotiate and administer the license

b) convincing the MPEG-LA that they have sufficient technical measures to stop them from being abused - ie people not taking the model C bootcode.bin and using it on a model B without paying the license.


The Pi clearly needs an H.264 encoder since it opens up so many new uses - video conferencing, video editing and so on - and also because the HW is already capable of it.

But not all users would need it - perhaps it would be possible to offer it as an add-on SW component including the license?

mole125
Posts: 228
Joined: Tue Jan 10, 2012 2:01 pm

Re: Hardware-assisted H.264 video encoding

Tue May 01, 2012 8:39 am

mr_sunshine said:


mole125 said:


a) the cost and effort to negotiate and administer the license

b) convincing the MPEG-LA that they have sufficient technical measures to stop them from being abused - ie people not taking the model C bootcode.bin and using it on a model B without paying the license.


The Pi clearly needs an H.264 encoder since it opens up so many new uses - video conferencing, video editing and so on - and also because the HW is already capable of it.

But not all users would need it - perhaps it would be possible to offer it as an add-on SW component including the license?


There's a difference between need and want unfortunately. For the Foundation to meet there primary aims with the RPi they don't need H264, but I agree we all want it and it opens up so many opportunities. The foundation have however said they plan to provide a hi-res camera and that will need an encoder so I think there is a very good chance that the foundation will find a way to solve the challenges above, hopefully that way will also allow none video encoding use (which probably isn't guaranteed depending how much of a pain MPEG LA are).

errikos
Posts: 33
Joined: Thu Feb 09, 2012 12:47 pm

Re: Hardware-assisted H.264 video encoding

Fri Aug 24, 2012 10:38 pm

So, now that the hardware H.264 encoder is feasible, where should we search to make something?! :D :D

kalehrl
Posts: 350
Joined: Tue Jul 24, 2012 10:49 am

Re: Hardware-assisted H.264 video encoding

Sat Aug 25, 2012 7:35 am

Hardware encoding is very bad compared to software.
If you expect to be able to make good rips using your raspi, I believe you'll be disappointed.

ghans
Posts: 7873
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany

Re: Hardware-assisted H.264 video encoding

Sat Aug 25, 2012 9:31 am

We'll see.
First somebody has to wrestle himself through OpenMAX for anything to happen.

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org

virtualas
Posts: 8
Joined: Fri Oct 19, 2012 10:41 am

Re: Hardware-assisted H.264 video encoding

Sun Oct 21, 2012 10:26 am

Maybe some updates on this topic related to video streaming? I am looking for same too! :) And founding just open topics... :/

ghans
Posts: 7873
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany

Re: Hardware-assisted H.264 video encoding

Sun Oct 21, 2012 10:58 am

Nobody seems to be willing to donate his/her time to this ...
The feature is freely available for C programmers now , they don't
even have to unlock it ... but nobody seems to like the OpenMAX API.
VLC says "the implementation is only 99.999 % conforming to the standard , that's not enough" ,
ffmpeg enthusiasts chickened out when they started on OpenMAX (i guess here , because i've not
heard anything since the request for docs ) and gstreamer devs seem very wary of optimization work.

Sad , I wish i already knew how to program properly.

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23629
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware-assisted H.264 video encoding

Sun Oct 21, 2012 11:02 am

OpenMAX is quite unpleasant to work with. Its a great idea, that really hasn't worked out in practice. Probably explains the lack of progress.

What are people actually looking for from the encode ability? Once the camera module is released we plan to have encode working from that, but what else is requried?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5331
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Hardware-assisted H.264 video encoding

Sun Oct 21, 2012 11:20 am

jamesh wrote:OpenMAX is quite unpleasant to work with. Its a great idea, that really hasn't worked out in practice. Probably explains the lack of progress.
It is not *that* hard.
kulve was first to get h264 encode running through openMAX and it's only a couple of pages of C code. See hello_encode demo based off his code:
https://github.com/raspberrypi/firmware ... e/encode.c

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23629
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware-assisted H.264 video encoding

Sun Oct 21, 2012 1:00 pm

dom wrote:
jamesh wrote:OpenMAX is quite unpleasant to work with. Its a great idea, that really hasn't worked out in practice. Probably explains the lack of progress.
It is not *that* hard.
kulve was first to get h264 encode running through openMAX and it's only a couple of pages of C code. See hello_encode demo based off his code:
https://github.com/raspberrypi/firmware ... e/encode.c
Ah, but you are smarter than I am! Main problem I've found is you know what needs to be done, and there doesn't appear to be an order required, but there is...and it doesn't tell you want that order needs to be...and the error messages are 'interesting'!

That said, there is example code as linked! So I don't have to think about it!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Santa77
Posts: 32
Joined: Wed Oct 31, 2012 2:08 pm
Location: Slovakia

Bitrate control for hello_encode

Wed Oct 31, 2012 2:10 pm

If someone is interested in, small patch is here:

https://github.com/raspberrypi/firmware/issues/114

Return to “General discussion”