AlexKordic
Posts: 9
Joined: Sun Jun 02, 2013 12:46 am

Re: Camera Interface Specs

Sat Feb 01, 2014 2:19 am

Will HDMI-in board schematic and software be open-sourced ?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24191
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Sat Feb 01, 2014 10:58 am

AlexKordic wrote:Will HDMI-in board schematic and software be open-sourced ?
Don't know. If the Foundation decides to market it, probably not, as their income source would dry up overnight as soon as China woke up.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
redhawk
Posts: 3465
Joined: Sun Mar 04, 2012 2:13 pm
Location: ::1

Re: Camera Interface Specs

Sat Feb 01, 2014 11:27 am

The Chinese are not stupid they've already cloned the Pi Camera with a rubbishy lens, so even if schematics for HDMI board wasn't available online it would still get cloned too anway.

From what I understand Toshiba make HDMI to CSI bridge ICs which I'm assuming is what this board is based on.
So it would probably consist of a HDMI connector, CSI connector, XTAL timing crystal, SMD capacitors, inductors, resistors and sticky back plaster. :)

Richard S.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24191
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Sat Feb 01, 2014 1:48 pm

redhawk wrote:The Chinese are not stupid they've already cloned the Pi Camera with a rubbishy lens, so even if schematics for HDMI board wasn't available online it would still get cloned too anway.

From what I understand Toshiba make HDMI to CSI bridge ICs which I'm assuming is what this board is based on.
So it would probably consist of a HDMI connector, CSI connector, XTAL timing crystal, SMD capacitors, inductors, resistors and sticky back plaster. :)

Richard S.
There a difference between copying from a PCB, and making from the schematics. Look how long it took for alternative cameras to come out. It's all in the timing.

You basically have defined the required components - feel free to make one!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Sun Mar 02, 2014 6:16 pm

James,

A lot has changed since I last read this thread. So now I have some new questions:

1. Earlier in this thread, you had stated that part of the problem of supporting an HDMI input board was that the video input processor between the CSI input and the H.264 encoder is designed for a CCD camera that needs a long, tedious tuning process. As it is assumed any CCD will require this tuning process, there are no “factory default” settings for already tuned video and it would require the same long, tedious process to find the “already tuned” parameters. I assume this has now been done. Is this “tuned camera profile” available to the general public or is it proprietary to the Foundation?
2. You had stated another issue would be either the necessity of firmware to decode HDMI control information and use it to set parameters like resolution, frame rate, etc. or as I had suggested that for revision 1, the user would have to set these parameters manually. How are you currently doing this?
3. Does the Toshiba chip require any configuration? If yes, is this configuration file available to the general public or is it proprietary to the Foundation?

4. Gordon stated that currently, your HDMI input board will not support 1080 resolution because only two lanes of CSI is brought out to the CSI input connector. So I think it is therefore a reasonable assumption that the Toshiba chip does not have the capability to translate HDMI into two 10 bit CSI lanes. This brings up question 4a or 4b
4a. Does the Foundation have any plans to do a hardware revision that would bring more CSI lanes out to the connector?
4b. If someone were to do an FPGA that would either take HDMI in directly or the CSI output from the Toshiba chip and convert it to 2 10bit CSI lanes to support 1080 HDMI, would the Foundation support this effort with whatever firmware tweaks would be necessary?

Best Regards,
Hardware_man
www.linkedin.com/pub/martin-risso/71/599/a09

tonybarry
Posts: 9
Joined: Thu Jan 02, 2014 4:59 am

Re: Camera Interface Specs

Sun Mar 02, 2014 9:12 pm

Hi James,

As with Martin (hardware man) I am still very interested in the HDMI-in board (as it presently stands).

Regards,
Tony Barry

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Tue Mar 04, 2014 5:30 pm

James and Gordon,

Could you please reply to my questions? If the Foundation is going to manufacture an HDMI input board, then I certainly understand that you want to keep as much information proprietary as you can.

But if the Foundation is not going to manufacture the HDMI input board, then it is in your best interest to support a third party manufacturer as the more add on accessories for the Pi in the marketplace, the more Pi sales you will make.
If you want to do a survey to get an idea of potential HDMI input board sales, then I recommend placing it on your home page, not buried deep down here in this thread.

Hardware_man
www.linkedin.com/pub/martin-risso/71/599/a09

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24191
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Tue Mar 04, 2014 6:17 pm

Hardware_man wrote:James and Gordon,

Could you please reply to my questions? If the Foundation is going to manufacture an HDMI input board, then I certainly understand that you want to keep as much information proprietary as you can.

But if the Foundation is not going to manufacture the HDMI input board, then it is in your best interest to support a third party manufacturer as the more add on accessories for the Pi in the marketplace, the more Pi sales you will make.
If you want to do a survey to get an idea of potential HDMI input board sales, then I recommend placing it on your home page, not buried deep down here in this thread.

Hardware_man
http://www.linkedin.com/pub/martin-risso/71/599/a09
Sorry, hadn't seen the question.

1) Tuning. Looks OK to me. Not sure how much tuning was required actually - looks like it pretty much worked OK out of the box.
2) This code has been done.
3) Not sure. Probably, but again this code has been done.
4) Odd. I thought I had it going in 1080p. Maybe I was wrong, it was a while ago. The two lane CSI may be a limitation of the Raspi camera interface I think - the SoC supports 4, but they may not all be brought out.
4a) No idea.
4b) Doubt it. Although not sure its necessary. 2 lanes should be enough for 1080p. Its enough for the camera for example. There may be some HW implications I am unaware of.

Pretty sure the Foundation are the best judges of what is in their best interest though. I do not believe a decision has been made on manufacture yet. They are all rather busy with educational stuff and the screen which is clearly going to be a much larger market, and some other stuff which will excite a large number of people.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Tue Mar 04, 2014 7:13 pm

James and Gordon,

When you use the HDMI input board, are you using the same camera profile as you use with the Foundation camera or a different camera profile?

It is good to know the code has been done. My question is this code available to any Pi user or is it Foundation proprietary?

Please see Gordon’s post earlier in this thread
January 02, 2014, 7:52 AM
“Actually no its only good for 720P30 (two lanes only)”
January 02, 2014 8:15 AM
“Difference is camera is bayer 10 bit per pixel whereas the capture outputs 24bit RGB”

I’m not trying to tell you how to do business. I have people who want the HDMI input board “yesterday” and want to commission me to design schematic and Gerber files. If the Foundation is going to manufacture and sell the HDMI input board, then my people certainly don’t want to pay me to design and lay out a PCB if they can simply buy it “off the shelf” from you.

But in good conscience, I don’t want to take their money if the necessary firmware will not be available to use the board.

Hardware_man

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24191
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Tue Mar 04, 2014 7:35 pm

Hardware_man wrote:James and Gordon,

When you use the HDMI input board, are you using the same camera profile as you use with the Foundation camera or a different camera profile?

It is good to know the code has been done. My question is this code available to any Pi user or is it Foundation proprietary?

Please see Gordon’s post earlier in this thread
January 02, 2014, 7:52 AM
“Actually no its only good for 720P30 (two lanes only)”
January 02, 2014 8:15 AM
“Difference is camera is bayer 10 bit per pixel whereas the capture outputs 24bit RGB”

I’m not trying to tell you how to do business. I have people who want the HDMI input board “yesterday” and want to commission me to design schematic and Gerber files. If the Foundation is going to manufacture and sell the HDMI input board, then my people certainly don’t want to pay me to design and lay out a PCB if they can simply buy it “off the shelf” from you.

But in good conscience, I don’t want to take their money if the necessary firmware will not be available to use the board.

Hardware_man
Ah, the RGB stuff explains the speed difference - much higher bandwidth required. Just looked at the code base, this device output BGR888, so no tuning file required. It does require a driver of course, but the data coming in is already is a state where it can be passed straight around the ISP.

I don't know if the Foundation is going to sell the board. I don't know if they plan to release schematics. I don't know if they plan to release the software (it's on the GPU so would be a firmware blob). I don't know if someone makes their own whether the Foundation will be willing/able to write the firmware for it. I don't know whether the Raspi is the right platform for your customer.

I do know they are insanely busy and this would be very much a sideline as the market is pretty small.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Tue Mar 04, 2014 7:49 pm

James,

For months I had asked you if you start with video that is already tuned, can you bypass the ISP and go straight into the H.264 encoder and for months you said no, you would have to write a unique camera profile that is "tuned" for tuned video. Today, you tell me RGB888 just bypasses the ISP!

Could you please forward my questions to the right people who can answer them?

Hardware_man

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Tue Mar 04, 2014 8:42 pm

James wrote:

"I don't know if the Rasb Pi is the right platform for my customer"

Here is what I do know. Samsung chose Broadcom's H.264 encoder over their own company's H.264 encoder for their latest and greatest smart phones. This must be an issue of shame for both Samsung and South Korean national pride. So it must be a GREAT H.264 implementation for technology to win over politics to this extent.

So unless you can buy a gazillion Broadcom chips, then the RaspPi is the right platform for anybody who needs a H.264 real time encoder.

Hardware_man

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24191
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Wed Mar 05, 2014 9:20 am

Hardware_man wrote:James,

For months I had asked you if you start with video that is already tuned, can you bypass the ISP and go straight into the H.264 encoder and for months you said no, you would have to write a unique camera profile that is "tuned" for tuned video. Today, you tell me RGB888 just bypasses the ISP!

Could you please forward my questions to the right people who can answer them?

Hardware_man
It's called learning. I learnt that the HDMI-CSI chip outputs RGB rather than RAW. What has to be written is a driver to communicate with the chip as if it were a camera, and pass the incoming data through to the ISP correctly (in effect bypassing lots of the processing stages). Which is the unique part. (this is mostly done, although there are comments in the driver noting things that still need to be done)

I think Gordon/Eben are the only people who can answer your manufacture/distribution questions. I will send them a link to this post. If they do answer be ready to receive an answer you really really don't want to hear. Remember, that HDMI in is a niche market - there are many more important things they need to be dealing with.

I'm bowing out now.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1445
Joined: Sat Sep 10, 2011 11:43 am

Re: Camera Interface Specs

Wed Mar 05, 2014 9:46 am

Hardware_man wrote: 1. Earlier in this thread, you had stated that part of the problem of supporting an HDMI input board was that the video input processor between the CSI input and the H.264 encoder is designed for a CCD camera that needs a long, tedious tuning process. As it is assumed any CCD will require this tuning process, there are no “factory default” settings for already tuned video and it would require the same long, tedious process to find the “already tuned” parameters. I assume this has now been done. Is this “tuned camera profile” available to the general public or is it proprietary to the Foundation?
Actually the ISP is basically turned off with the HDMI input because it's already in RGB and doesn't need all the rest of the ISP

No this is not available to the GP... It requires too much support
Hardware_man wrote: 2. You had stated another issue would be either the necessity of firmware to decode HDMI control information and use it to set parameters like resolution, frame rate, etc. or as I had suggested that for revision 1, the user would have to set these parameters manually. How are you currently doing this?
Software in the firmware blob is doing this
Hardware_man wrote: 3. Does the Toshiba chip require any configuration? If yes, is this configuration file available to the general public or is it proprietary to the Foundation?
Yes it requires quite a bit of configuration and no it's not available to the GP for the same reason (support)
Hardware_man wrote: 4. Gordon stated that currently, your HDMI input board will not support 1080 resolution because only two lanes of CSI is brought out to the CSI input connector. So I think it is therefore a reasonable assumption that the Toshiba chip does not have the capability to translate HDMI into two 10 bit CSI lanes. This brings up question 4a or 4b
Actually mathematics...
1920 x 1080 x 24 x 60 = 3Gbits
MIPI CSI spec maxes out at 1GBit per channel
Hardware_man wrote: 4a. Does the Foundation have any plans to do a hardware revision that would bring more CSI lanes out to the connector?
4b. If someone were to do an FPGA that would either take HDMI in directly or the CSI output from the Toshiba chip and convert it to 2 10bit CSI lanes to support 1080 HDMI, would the Foundation support this effort with whatever firmware tweaks would be necessary?
This is not possible there are only two CSI lanes on the Raspberry Pi you cannot squeeze more data down the two lanes.

If you can prove this is a viable product that has significant funding behind it then we could think about actually working on a product but for now the required resources are not available. I'd also like to make it clear that in future any Raspberry Pi add ons (like the display board) will have a security system to stop chinese copies like the camera board has. At least until someone hacks it!

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

JohnnyGo
Posts: 2
Joined: Thu Mar 13, 2014 9:19 pm

Re: Camera Interface Specs

Fri Mar 14, 2014 12:10 am

Our company has bought a few RPi's to experiment with in a video project we are working on. So far, so good. If everything works out we will be purchasing a few hundred.

However, we would also like to transmit our camera signal through Wi-Fi to our switcher, which has two iVGA (network) inputs, for live shoots. This would eliminate the need for laying cable over long distances.

That being said we would certainly appreciate a way to input a video signal into the RPi through the CSI. 720p@30fps would be great.

You have our vote for production of an HDMI-CSI board ASAP.

P.S. Gordon, if you have any to spare, please send us a couple.

auvidea
Posts: 13
Joined: Sun Nov 24, 2013 6:57 pm

Re: Camera Interface Specs

Fri Mar 14, 2014 4:31 pm

My HDMI input board for the Raspberry Pi is making some very good progress. Yesterday I have started it as a kickstarter project. Please check it out at https://www.kickstarter.com/projects/14 ... spberry-pi, if you are interested.

Jurgen

auvidea
Posts: 13
Joined: Sun Nov 24, 2013 6:57 pm

Re: Camera Interface Specs

Fri Mar 14, 2014 4:34 pm

Hi Gordon,

it looks like, that you are also looking at an HDMI to CSI converter board. What is your status? Would you be interested in working together?

Jurgen

JohnnyGo
Posts: 2
Joined: Thu Mar 13, 2014 9:19 pm

Re: Camera Interface Specs

Fri Mar 14, 2014 9:16 pm

auvidea wrote:My HDMI input board for the Raspberry Pi is making some very good progress. Yesterday I have started it as a kickstarter project. Please check it out at https://www.kickstarter.com/projects/14 ... spberry-pi, if you are interested.

Jurgen
Looks very interesting. I'll run it by my partner tonight.

Question: Will the unit be able to stream HDMI through a Wi-Fi connection?

tonybarry
Posts: 9
Joined: Thu Jan 02, 2014 4:59 am

Re: Camera Interface Specs

Fri Mar 14, 2014 9:50 pm

Hi Gordon / Foundation people,

If I can re-iterate my +1 for this project while you are looking …

Regards,
Tony Barry

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Fri Mar 14, 2014 10:15 pm

Audivea,

How are you handling the issue that the Foundation isn’t releasing the firmware to take CSI input “past” the camera tuning ISP and the information that you would need to do this yourself is Broadcom proprietary?

Hardware_man

tonybarry
Posts: 9
Joined: Thu Jan 02, 2014 4:59 am

Re: Camera Interface Specs

Fri Mar 14, 2014 10:22 pm

Hi Martin,

The Foundation will do as it wishes. The trick is to get the wish.

Regards,
TB

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24191
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Sat Mar 15, 2014 8:48 am

Hardware_man wrote:Audivea,

How are you handling the issue that the Foundation isn’t releasing the firmware to take CSI input “past” the camera tuning ISP and the information that you would need to do this yourself is Broadcom proprietary?

Hardware_man
It's not just that though - I'm having trouble seeing how this could be done without a very specific driver on the CSI-2 as well, which again is not possible without Broadcom/Foundation help.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

PiGraham
Posts: 3671
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Camera Interface Specs

Sat Mar 15, 2014 10:08 am

jamesh wrote:
Hardware_man wrote:Audivea,

How are you handling the issue that the Foundation isn’t releasing the firmware to take CSI input “past” the camera tuning ISP and the information that you would need to do this yourself is Broadcom proprietary?

Hardware_man
It's not just that though - I'm having trouble seeing how this could be done without a very specific driver on the CSI-2 as well, which again is not possible without Broadcom/Foundation help.
It seems they have no idea how to do that either:
The next challenge is the MIPI/CSI interface to the Raspberry Pi. Mainly from a software perspective. It is the first time, we use this CSI converter chip. This chip is well documented. The real challenge is the software interface to the Raspberry Pi. We welcome any help in this area. We are happy to provide an early development board to experienced programmers, who are interested to help us.
I suppose it would be possible to reverse-engineer the I2C controls and CSI timing ofthe PiCam and make the HDMI capture pretend to be a PiCam so that no Pi software is required. If that can be done then raspivid will capture from it.I can imagine there might be a lot of things to reverse engineer to make that happen, including lens aberation.

PiGraham
Posts: 3671
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Camera Interface Specs

Sat Mar 15, 2014 10:18 am

Very interesting. What latency should we expect in a script?
What effect does a script loop have on the rest of the system?


Sorry, wrong topic
Last edited by PiGraham on Sat Mar 15, 2014 10:33 am, edited 1 time in total.

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Camera Interface Specs

Sat Mar 15, 2014 10:29 am

PiGraham wrote: I suppose it would be possible to reverse-engineer the I2C controls and CSI timing ofthe PiCam and make the HDMI capture pretend to be a PiCam so that no Pi software is required. If that can be done then raspivid will capture from it.I can imagine there might be a lot of things to reverse engineer to make that happen, including lens aberation.
I think that might be the approach that is being considered here. See the "Software interface to the Raspberry Pi" link in the FAQ section at the bottom of the Kickstarter page:
"Will a HDMI input source appear to the Raspberry Pi operating system in the same way the camera module currently does? In other words, can raspivid be used for hardware encoding video coming in from the HDMI port?

This is the idea. This module converts the HDMI input into a MIPI/CSI input, so it can be fed into the CSI connector for the camera board. My parallel to MIPI/CSI converter chip is extremely flexible and supports all kinds of modes, so it should be possible to create a CSI signal, which is very similar to the signal of the camera board. In this manner, all software, which works with the camera board should also work with this HDMI In Module. However, I expect that some software and/or drivers on the Raspberry Pi need to be modified. Any help in this area is welcome."

Return to “Camera board”