stotlem
Posts: 3
Joined: Sat May 07, 2016 11:08 pm

4 Camera Modules with Compute Module & IO Board

Sat May 07, 2016 11:13 pm

I'm an EECS student and I have a project which features synchronized image acquisition from multiple camera modules (four to be exact).

Are the camera and display inputs on the IO board the same?
Could I attach four camera modules to the IO board and capture images from each camera module in synch?

Thanks!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7431
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: 4 Camera Modules with Compute Module & IO Board

Sun May 08, 2016 6:10 am

stotlemail wrote:Are the camera and display inputs on the IO board the same?
Could I attach four camera modules to the IO board and capture images from each camera module in synch?
No, the display ports are outputs, the camera ports are inputs.
There's no way of simultaneously running 4 cameras in sync on 1 Pi/cm. You can probably set things up to sync multiple Pis together, but there's no infrastructure in place to do it at the moment.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

stotlem
Posts: 3
Joined: Sat May 07, 2016 11:08 pm

Re: 4 Camera Modules with Compute Module & IO Board

Sun May 08, 2016 7:25 pm

Thanks for the reply! I realized hours ago this was a stupid question after looking at the IO board schematics. Currently, my only options are to design my own IO board, replacing the two display outputs?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7431
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: 4 Camera Modules with Compute Module & IO Board

Sun May 08, 2016 8:50 pm

Replace them with what? There are only two camera receiver interface peripherals on th Broadcom SoC. You physically can't connect more cameras too it.

The display connectors connect to the display peripherals on the SoC, and those pins are dedicated outputs so can't be repurposed.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jrychter
Posts: 20
Joined: Tue Jun 28, 2016 11:03 am
Contact: Website

Re: 4 Camera Modules with Compute Module & IO Board

Thu Jun 30, 2016 5:09 pm

Actually, I've been looking at the same problem and there is a reasonably straightforward way to use more than 2 cameras in hardware, but it requires software support. I haven't looked at the driver code (is all of it actually available?) to see how complex that would be.

Hardware-side, you do it by multiplexing camera connections. There is a mux from Fairchild specifically designed for that: FSA642. Obviously you need to mux the I2C connections, too. But it should work just fine for most apps where you don't need the full bandwidth of the 2 CSI lanes. In my applications I should be just fine for video framerates of ~10fps.

This isn't something I need today, but I will need it within the next 6 months (I already have a draft HW design for a board that includes a CM and 4 cameras), so I'm posting this hoping to attract some friendly attention. I'm worried about whether I will be able to change the driver code myself.
--J.
Founder at PartsBox: https://partsbox.io/ (keep track of your electronic components)

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: 4 Camera Modules with Compute Module & IO Board

Thu Jun 30, 2016 5:26 pm

Are you thinking of using the FSA642 MIPI switch to swap the SOC's CSI input lines between two physical cameras without the SOC being aware that it is getting even frames from one camera and odd frames from a different camera? It seems like a rather bold idea. I suppose it could work, if the cameras were running closely in sync, otherwise you'd get split and mis-registered frames that were partly from camera A and partly from B, and noise/ invalid signals at the moment of switching, depending on the switching speed of the mux.

Have you looked at this device?
http://www.ivmech.com/magaza/en/develop ... exer-p-107

Code: Select all

This is IVPort V2, the first Raspberry Pi (It is compatible with all models of Raspberry Pi 1, 2 and 3) 8MP Camera Module V2 Multiplexer designed to connect more than one Camera Module V2 to Raspberry Pi.
Connectors: 15 Pin FFC CSI
Pinout: RaspberryPi CSI Connector Camera Pinouts
Protocol: MIPI
Propagation Delay between input and output: 0.5 ns
Switching Time fromone channel to other channel: 50ns (it can increase with depending to GPIO delay)
Image

jrychter
Posts: 20
Joined: Tue Jun 28, 2016 11:03 am
Contact: Website

Re: 4 Camera Modules with Compute Module & IO Board

Fri Jul 01, 2016 8:04 am

jbeale wrote:Are you thinking of using the FSA642 MIPI switch to swap the SOC's CSI input lines between two physical cameras without the SOC being aware that it is getting even frames from one camera and odd frames from a different camera?
Yes. I see no reason for it not to work, if you don't push everything to unreasonable limits (e.g. don't expect to double your CSI bandwidth just because you added a mux). But, like I said, it does require software support.

The device you mentioned can be bought from a number of vendors, actually. I saw one made in China, and while in the good long-standing tradition it had the part numbers sanded off, it looked like it used FSA642s to mux CSI connections.
--J.
Founder at PartsBox: https://partsbox.io/ (keep track of your electronic components)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23918
Joined: Sat Jul 30, 2011 7:41 pm

Re: 4 Camera Modules with Compute Module & IO Board

Fri Jul 01, 2016 8:58 am

If this were possible, which I doubt (I suspect timing would be extremely critical given CSI-2 speeds), it would likely require extensive changes to the camera software, which runs on the GPU, which is closed source, so therefore only the RPF can make the changes. For a use case such as this, it's simply not worth doing.

ISTR remember someone talking about this a few years ago, not sure how far they go, worth a Google search.

Best idea? Either two CM's on usual carrier board- 2 cameras on each. Or 4 Pi3 with one camera each, and sync them somehow. Depends on the actual use case.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7431
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: 4 Camera Modules with Compute Module & IO Board

Fri Jul 01, 2016 9:24 am

I came to the conclusion that the V1 of those boards didn't mux the I2C, so all N sensors see the same transactions, and the mechanism for NAKing invalid messages will be broken if even one sensor sees it correctly. It's also why it doesn't work on V2.1 camera boards as we read unique data back from those modules (V1.3 boards I think we only read the sensor ID, which will always be 0x5647).
Seeing as they're advertising a V2 board that supposedly works, I guess they've made all except one I2C bus "unidirectional", so sensor 1 replies on behalf of all of them.

The resync interval on switching is undefined. You will certainly corrupt the frame that was being received during the switch, and I couldn't say for definite that you won't get that corrupted frame delivered (there are CRCs on each line, I can't remember about the whole frame).
Watching the youtube clip embedded in the IVmech page I see a glitch about 42 seconds that appears to be part of two different frames, so I'd say that's a definite that you can get through partially corrupt frames. The magenta frame at about 1:10 and other points are also a classic symptom of CSI2 corruption.

Auto gain and exposure control will be quite badly broken if you're switching between sensors looking at significantly different scenes. AWB will have no knowledge of the switch either. About 56 seconds in that youube clip is a good example of that where it is having to take several frames to adjust after a switch from the desktop and hands to the office wall and posters.

There is also some latency in the GPU side, so on switching the GPIOs from userspace there is no guarantee as to exactly how many frames later the resulting frames will have switched to the new sensor.


Currently software support would have to be part of the GPU firmware, and that isn't going to happen I'm afraid. I don't have the spare time, and RPF are unlikely to see the business case for them to fund it.
There is the work ongoing to expose the CSI2 receiver via V4L2, but you then lose the ISP and will be getting raw Bayer frames, or have to use an alternate sensor with built in ISP.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

johnf
Posts: 30
Joined: Fri Dec 23, 2011 10:35 pm

Re: 4 Camera Modules with Compute Module & IO Board

Mon Sep 23, 2019 8:58 pm

My current project is to produce a close-up FINGER CAM enabling a blind person to scan textiles, foods, labels and so on.
This is as an adjunct to " The vOICe " which is a video-to- stereo sound aid to blind persons. Nothing to do with any TV talent show. The system is hosted on Pi, phones, special vr camera glasses, etc.
Blind persons are successfully interpreting immediate surroundings and indeed winning in competitions against brain/eye implant candidates.
There is the point of view which derives from the head mounted camera in normal usage, but there is also the experience of close up inspection of textiles, labels , prices , OCR and so on. Try to imagine doing that with a head mounted camera.
I have successfully fitted an Adafruit mini spy camera into a cigar tube. With Tindie adaptors to a HDMI cable, I have made a finger cam, powered by the HDMI . This lurks in a jacket sleeve and is easily brought down into use.
I wish to select either head mounted camera or the finger camera , for instance in the context of a shopping activity.
I would be content to have momentary blanking during changeovers. The vOICe is not a high resolution system .
Naturally I am not keen on hacking into the CSI camera hardware.
Is anyone else working on this kind of thing? The Fairchild FSA 642 switch is a candidate.

Return to “Compute Module”