Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Camera Interface Specs

Tue Dec 04, 2012 7:09 pm

Hi,
I want to interface a different camera to the Pi primarily because the camera that I want to use has a zoom lens. It looks like you have two LVDS data inputs and an LVDS clock input on connector S5. How much information can you give about this connector specification without hurting your Broadcom NDA?

Alternatively, if I tell you what format I have, and what formats I can convert it to, can someone write the appropriate driver?

Hardware_man

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Tue Dec 04, 2012 8:56 pm

I think all the connector specs are public (maybe not camera?), but the camera interface connects directly to the GPU either using CCP2 or CSI-2. Your camera would require a new driver on the GPU, and they are usually pretty complicated. Since yours has a zoom lens you also need a more than usually complicated lens driver and appropriate tuning. I doubt we already have a driver since we have never worked on a a system with a zoom lens. It's a big job I'm afraid, and wouldn't be possible for anyone outside the Foundation or Broadcom to do it. Therefore that would be something you would need to contract the Foundation or Broadcom to do for you. Might be worth it depending on your use case.

Sorry, that's probably not what you wanted to hear.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Tue Dec 04, 2012 9:39 pm

The camera that I want to use is a Sony FCB-EH3150. It is a camera module, more than just the CCD. HD outputs in Analog component or digital LVDS.

I think a lot of the "tuning" is already done in the camera.

I want to use the Pi board to run a high profile H.264 encoder to get data rates down to the write speed of the SD card for a high end DVR implementation.

Does this simplify the process?

Hardware_man

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Wed Dec 05, 2012 4:12 pm

Jamesh,

You mentioned contacting the foundation. How do I do that?

Hardware_man

User avatar
Gert van Loo
Posts: 2486
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Camera Interface Specs

Wed Dec 05, 2012 4:41 pm

Hardware_man wrote:Jamesh,

You mentioned contacting the foundation. How do I do that?

Hardware_man
Easily misread: he said you have to contract the foundation or Broadcom.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Wed Dec 05, 2012 4:59 pm

Hardware_man wrote:The camera that I want to use is a Sony FCB-EH3150. It is a camera module, more than just the CCD. HD outputs in Analog component or digital LVDS.

I think a lot of the "tuning" is already done in the camera.

I want to use the Pi board to run a high profile H.264 encoder to get data rates down to the write speed of the SD card for a high end DVR implementation.

Does this simplify the process?

Hardware_man
Ah, OK, we are talking at cross purposes. You camera is already doing all the hard work, and doesn't use the same camera interface available on the Raspberry Pi. I have no idea how you would connect a camera like that, having looked briefly at the data sheet, I don't think it's possible without extra hardware,
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Wed Dec 05, 2012 7:52 pm

Jamesh,

I am a hardware engineer. I already expected to have to do an "adapter" board. The Sony outputs HD digital on 4 7 bit LVDS channels. I noticed the camera connector only brings two I think LVDS data inputs and an LVDS clock input out. And I expect impossible to tack solder wires onto the Broadcom chip as the BGA pins are probably buried deep under the chip.
So I would do an adapter board to convert the Sony LVDS to parallel and then convert again to two LVDS channels and a clock LVDS channel to drive the camera connector.
But I would need to know stuff like data format, MSB first or LSB first, number of bits for LVDS channel, clock speed etc.
But I don’t know much about writing drivers. I guess this would be a “generic” driver as it would be for a camera that is already “tuned”

Hardware_man

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Wed Dec 05, 2012 10:07 pm

You would need to convert whatever the Sony outputs in LVDS in to MIPI CSI-2 (or CCP2 - the Pi can take use either, but CSI-2 is better), also it looks to be controlled by RS232, our camera drivers usually use I2C, so you may need to convert that as well. It would require a new camera driver - each module always does as they are always a bit different from each other.

It's not going to be easy, and I'm not the best person to comment on it, and I really don't know who would be - there's no experience in our team with a camera like that (as far as I know).
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

ghans
Posts: 7878
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany

Re: Camera Interface Specs

Thu Dec 06, 2012 8:22 am

But this would still need custom debugged GPU code to work ,
and without this it won't work at all, or am i mistaken ? So this
needs definitely involvment of Broadcom employees.


ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Thu Dec 06, 2012 9:34 am

ghans wrote:But this would still need custom debugged GPU code to work ,
and without this it won't work at all, or am i mistaken ? So this
needs definitely involvment of Broadcom employees.

ghans
Yes, it would need custom GPU code (if it's even possible), so either done by Broadcom or the Foundation (who have access to the code).
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Thu Dec 06, 2012 5:04 pm

Jamesh,

Actually, I don’t need the Pi board to control the camera. I already remotely control the camera via the RS-232 for remote control of the zoom lens. I use the factory defaults for all of the video parameters.

From a block diagram point of view, consider if I simply took the standard definition composite video from any camera and plugged it into the composite video in of a VCR. I’m recording the camera but the VCR doesn’t have any control of the camera.

So I’m basically trying to use the Pi board as a physically very small high definition DVR recording to the SD card.

Now, If I used the analog high definition component video from the Sony and put a 3 channel video A/D converter on my adaptor board, this would then become a much more “universal” design as any high definition “tuned” camera that had analog component video out could just “plug and play” without regard to exact data formats of the camera’s digital output.

If I agree to give this adapter board schematic to the foundation, then would the foundation argee to writing the universal driver?

Hardware_man

6677
Posts: 383
Joined: Wed Mar 14, 2012 9:23 pm

Re: Camera Interface Specs

Thu Dec 06, 2012 7:34 pm

The driver cost would run into the tens or even hundreds of thousands of dollars, possibly more. A hardware board diagram would hardly be payment.

The pi has no video input other than that CSI connector

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Thu Dec 06, 2012 8:45 pm

You would need to convert whatever out put the camera has to the CSI-2 or CCP-2 format (I'd go CSI-2 - it's faster). Then a driver could be written to simply grab data from the CSI-2 and encode it. We've done something similar with HDMI in - converted the HDMI to CSi-2. I'm not sure how well that would work if the camera was being controlled from elsewhere though. Still quite a big job, and some thousands of $.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

EdZ
Posts: 11
Joined: Sat Dec 01, 2012 11:36 am

Re: Camera Interface Specs

Thu Dec 06, 2012 9:35 pm

We've done something similar with HDMI in - converted the HDMI to CSi-2.
Is this a "we've done it with some other board, so it's theoretically possible", or a "we have a HDMI-in driver in the works for the 'Pi"?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Fri Dec 07, 2012 3:32 pm

EdZ wrote:
We've done something similar with HDMI in - converted the HDMI to CSi-2.
Is this a "we've done it with some other board, so it's theoretically possible", or a "we have a HDMI-in driver in the works for the 'Pi"?
Done as a project using the VC4 but not involving Raspberry PI. Not sure whether its hit the market yet (or if it ever will!). Not to say it wouldn't be possible on the Pi though, but the main issue is HDMI in needs all sorts of nasty DRM.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Fri Dec 07, 2012 5:43 pm

Jamesh,

This was my thought. I do an analog component video to CSI-2 adapter / converter board. You grab the CSI-2 data and make several H.264 encoding algorithms available (high quality / not much SD card record time to low quality / lots of SD card record time). Put the SD card video file into a container file like AVI or WMP and play the SD card on any computer.

Like I said, then the Pi becomes a "plug and play" DVR for any video in analog component format. I'm certain at that point, somebody will expand it so you could record hours on a USB hard drive.

What do you think?

Hardware_man

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Fri Dec 07, 2012 7:00 pm

Hardware_man wrote:Jamesh,

This was my thought. I do an analog component video to CSI-2 adapter / converter board. You grab the CSI-2 data and make several H.264 encoding algorithms available (high quality / not much SD card record time to low quality / lots of SD card record time). Put the SD card video file into a container file like AVI or WMP and play the SD card on any computer.

Like I said, then the Pi becomes a "plug and play" DVR for any video in analog component format. I'm certain at that point, somebody will expand it so you could record hours on a USB hard drive.

What do you think?

Hardware_man
It does sound feasible. It would require a new driver on the GPU, but the pass through needed (no need for the ISP) is in place. I'll have a chat with the guy who did the HDMI version and see what he thinks - that'll be Monday at the earliest.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Fri Dec 07, 2012 8:10 pm

Jamesh,

Thanks, I’ll see what you say Monday.

BTW, I don’t need or want this, but from what I hear, there may be a real need for an HDMI input DVR. According to some on video forums, some cable TV companies are trying to make it impossible for people to save content to DVD or even VHS. Some on the video forums claim the latest cable boxes will only have HDMI output (no composite or component) and there is a lot of pressure on DVD recorder manufactures to not come out with an HDMI input machine. So the cable boxes will have an internal DVR good for some amount of hours, but there will be no way to ever save a program to DVD. Eventually, when you run out of time on your cable box DVR, you have to write over old stuff. Where I live, I still get standard definition analog cable so my ancient VCR still works. :D

Hardware_man

EdZ
Posts: 11
Joined: Sat Dec 01, 2012 11:36 am

Re: Camera Interface Specs

Sat Dec 08, 2012 12:20 am

Component video input would be quite useful for making a standalone video processor. There are dedicated boxes for doing things like adding scanlines to video (to replicate the look of older consoles on non-CRT displays), scaling to correct aspect ratios, colour correction, etc that could all be done at once on the 'Pi. A tuneable video upscaler would be quite fun to play around with.

Is HDMI input without any HDCP support a viable thing? Or is it also a licensing problem (paying for the license, abiding by rules that conflict with the 'Pi's open nature, etc) on top of a time sink?

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Tue Dec 11, 2012 1:49 pm

Jamesh,

I was thinking as you already have an HDMI driver and no matter what, I have to convert my Sony LVDS into so other format, I might as well convert to HDMI and use your existing driver.

Does your HDMI implementation suppory Y Cb Cr?

Hardware_man

Old Busted Hotness
Posts: 2
Joined: Thu Dec 13, 2012 6:26 pm

Re: Camera Interface Specs

Thu Dec 13, 2012 6:34 pm

Hi guys, first post here. Haven't received my Pi yet but I'm looking forward to it.

An HDMI DVR app would be really sweet. I'd love to upgrade my robotics camera to HD.

maw
Posts: 1
Joined: Wed Dec 19, 2012 7:52 pm

Re: Camera Interface Specs

Thu Dec 20, 2012 12:02 am

Hello,

I'm interested in doing a very similar thing as Hardware_man, i.e. connecting a Sony block camera to Raspberry Pi and use the Pi for encoding the stream using H264. I've found a chip from Toshiba that should be very helpful in achieving the CSI-2 interfacing part of the task: TC358746XBG. On the other side of the bridge the chip uses a much simpler interface than HDMI, so there would be no problems with any DRM. On the other hand most probably some additional logic would be required to interface it further to the camera.

I wonder how much would there be to do from Raspberry Pi's side, but I haven't really dug into the problem yet. Jamesh, you wrote "a driver could be written to simply grab data from the CSI-2 and encode it". How do you think, does one need some data available only from Broadcom NDA to achieve that or would it be possible with e.g. already available GPU drivers and maybe the OMX interface?

Best regards,
maw

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24142
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera Interface Specs

Thu Dec 20, 2012 8:09 am

I've had chat with the guy who got HDMI in working on the Videocore. But unfortunately he did it in a way that won't work on the RaspberryPi.

So, what would need to be done would be to find a HDMI->CSI-2 chip (Toshiba?) and write a driver on the GPU to interface to that chip. This would not support DHCP (too many nasty licence hassles), so any encrypted HDMI streams would not work. There might be some other changes required, since usually its the host that tells the camera what resolution to run at, whereas in this case the input stream is the source of that information - I'm not sure how that would affect the OpenMAX layers. It's quite a bit of work on the GPU code.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Hardware_man
Posts: 94
Joined: Tue Dec 04, 2012 6:28 pm
Contact: Website

Re: Camera Interface Specs

Thu Dec 20, 2012 4:23 pm

Jamesh,

The only reason that I considered using the HDMI is that you thought that you already had a driver. But I would go back to my original idea of using the CSI-2 port. You tell me what works best for you.
I don’t know for a fact if your HDMI port is an input only, output only, or configurable to be either. I know for a fact the CSI-2 port is an input.
To avoid trying to make this all “automatic”, just provide a method where the user has to enter all of the configuration information:
• Resolution
• FPS
• Aspect ratio
• RGB or Y Cb Cr
• Etc
Also, let the user select the encoding algorithm MPEG2 or H.264 (MPEG only works if you buy the key)
Encoding details H.264 profile, level, etc
But what only somebody with access to the non public sections of the Broadcom data sheet can do is route the video from the CSI-2 connector, through the VIP to the encoder and select the encoder parameters.
Then the GPU can generate the correct ”container” file that specifies which encoder, profile, level, etc was used.
Hardware_man

kumarnarain
Posts: 7
Joined: Sat May 19, 2012 1:27 pm

Re: Camera Interface Specs

Fri Dec 21, 2012 3:10 am

grabbing video off the Pi will be a great idea. Waiting to see what happens out of this thread
Kumar

Return to “Camera board”