Frame synchronisation

[PiGraham]

I see that the OV05647

Support for internal and external frame synchronisation.

Can anyone shed any light on that? Is it possible to frame-lock multiple cameras?
Is it possible to externally control frame rate or reset timing?

[jamesh]

I don't believe we have incorporated that. And the Raspi only exposes one CSI-2 port anyway, so you can only attached one camera as standard anyway.

[PiGraham]

I don't believe we have incorporated that. And the Raspi only exposes one CSI-2 port anyway, so you can only attached one camera as standard anyway.

Yes, but you can have more than one Pi.

One possibly popular use case is stereo 3D camera using two Pis with cameras modules. You can do it without frame sync, but then you have to sych two videos in post and you won't get frame-perfect results so a few artifacts are likely.

Someone did a 3D movie with two Pis that was on the blog a littel wile back and IIRC they did resyncing in post.

Presumably there is an Ext frame sync line on the FFC connector. I see a XVCLCK into the PLL feeding the timing generator. I guess that's it. Maybe it just needs to be driven to force the PLL, or maybe it needs to be switched in by software.

I there a pinout for the module in the public domain? I have a second PiCam on backorder. I might try it out.

[jbeale]

The blog post you were thinking of, The View from Gordon's Helmet showed a rather different, smaller early prototype of a "R-Pi" which was never produced. It used post work afterwards to manually time-align the two videos to make a 3D version. You could do that with two production R-Pi boards + 2 cameras, as well, although the real R-Pi board is larger than that early prototype.

I have not seen any pinout description of the tiny flex connector on the R-Pi camera module. The S5 connector on the main R-Pi board (where the 15x1mm camera flex cable connects) is shown on p.4 of the R-Pi schematic, but that's just the 2-data-lane MIPI CSI-2 plus I2C, power and a line labelled "CAM_CLK" which is actually being used as the red LED indicator light on the camera board.

If you ask google about "PRELIMINARY OV5647 filetype:pdf" you'll find an early version of the full data sheet for the camera chip, which is more extensive than the product brief on the company's site. It offers a reference schematic, but with a 32-pin connector, not the 24 lines on that tiny 0.4mm yellow flex connector on the module we have, so it's hard to guess what signals were left in and what was not.

If you have steady hands and a microscope, you could try probing those pins and see if you find any signals matching frame-start (eg. a 25 Hz pulse train, if you are recording video at 25 fps). But that's just the output sync. To take that as input to control frame-start on a second camera, as far as I know, you would need support from the GPU which is not currently on offer.

If you were truly dedicated, you could imagine installing extra hardware that sits between the R-Pi and the camera board and delays the control signal to the 2nd camera until it sees the 1st camera has started. IF the GPU allows a suitably long timeout before throwing an error, this could actually work. Of course, more effort than it is likely worth.

If you're willing to wait for a while until the cameras sync, you could also pull the R-Pi board's 19 MHz crystal X2 (or use an external VCO clock signal) and a PLL setup to gradually slow down or speed up the second R-Pi's system clock, until the accumulated phase lag causes the frame start times of the two cameras to match. Note, that clock signal is multiplied up internally many times to get the 700 MHz, etc. system clock so it likely has stringent duty cycle and jitter requirements.

[rkinch]

... it's hard to guess what signals were left in and what was not.

Clearly someone should sacrifice their camera and unsolder the BGA to ID what's connected to the ribbon. For some reason this is held as an important secret.

[steve_gulick]

some time ago on this forum I asked if a schematic of the camera board was available. No reply.

I wanted to find out what subset of the 50 pins were brought out to the 24 pin connector.

I was interested in what capabilities would be available. Like the possibility of synchronizing multiple cameras. So finding things like the vert sync signal would be useful.

I took one of the cameras I have (which had destroyed in an early attempt to remove the filter (-but learned enough to remove other filters)) and traced out the connections. This is what I came up with


OV5647 pad numbers and function names (from figure 1-1 in Omnivison' s data sheet)
1 16 DVDD
2 27 HREF
3 29 VSYNC
4 18 AVDD
5 22, 23 DVDD
6 24 DOVDD
7 26 AVDD
8 25 DOGND
9 NC ?
10 25 DOGND
11 45 XCLK
12 25 DOGND


13 5 SDA
14 4 SCL
15 35 D9/MDN0
16 36 D8/MDP0
17 38 D7/MCN
18 39 D6/MCP
19 41 D5/MDN1
20 42 D4/MDP1
21 34 D3
22 33 D2
23 32 D1
24 31 D0

The numbers in the left column are the pins on the 24-pin connector. I don't know the proper pin numbers. My arbitrary numbering scheme was looking at the top of the board with the lettering in normal reading position. I called pin 1 the pin at the top left, pin 12 bottom left, 13 bottom right, 24 top right (counter-clockwise from top left - like ic's)
like so...
1 24
.
12 13


pads on ov5747 NOT available on 24 pin connector:

15 PWDN
17 RESETB
28 PCLK
10 STROBE
11 FREX

The STROBE and FREX might have been useful for triggering and exposure control.

After tracing this out, I took a working module and tried to see the VSYNC signal with a scope on pin 3. No luck.

However, I think I saw in the manual (but can't find it at the moment) that the output on the VSYNC pad is disabled by default and needs to be enabled by writing an I2C register.

It would be useful if an official schematic could be made available.

Steve

[rkinch]

The STROBE and FREX might have been useful for triggering and exposure control.

Yes, sad that these features are unusable, as it rules out many valuable applications of an otherwise fine camera.

Thank you for the details on the connections. Good for us that you had this donor module, and the ability to sacrificially analyze and report on it in this sort of detail.

[fastmapper]

pads on ov5747 NOT available on 24 pin connector:

15 PWDN
17 RESETB
28 PCLK
10 STROBE
11 FREX

For the digital video port (DVP), the timing of the pixel data bits (D0 to D9) is related to PCLK. The VSYNC and HREF signals are present, so the only signal missing would be PCLK. I strongly suggest that PCLK would be on the 24-pin connector along with the other DVP signals.

[jamesh]

The STROBE and FREX might have been useful for triggering and exposure control.

Yes, sad that these features are unusable, as it rules out many valuable applications of an otherwise fine camera.

Thank you for the details on the connections. Good for us that you had this donor module, and the ability to sacrificially analyze and report on it in this sort of detail.

Build you own board with the same camera on.

As it stands, it's a cheap and cheerful camera board for the masses. Exactly what is required. You want something more specific, you are perfectly entitled to build your own. It's not that difficult. Don't expect other people to do your job for you for free.

I can add register stuff to the driver if you need slight software changes (nothing big though - I don't get paid and I do have a life and a real job).

[PiGraham]

James,

Don't take "if only the Pi did X" comments to heart. I'm sure people, when they stop to think about it, well realise that maximum functionality was not the design goal for the Pi team. It's very cheap and what we get for the money is very good.

You can't really blame us for asking if it can do this or that, though.

[jamesh]

James,

Don't take "if only the Pi did X" comments to heart. I'm sure people, when they stop to think about it, well realise that maximum functionality was not the design goal for the Pi team. It's very cheap and what we get for the money is very good.

You can't really blame us for asking if it can do this or that, though.

I have no problem all with people asking if the camera can do this or that. That's education.

Complaining when something doesn't do something, well, that's something else.

And just for the fun of it, I just found a camera board schematic whilst tidying my desk, I think its an oldish one, but from it, the camera socket has the following connections

1 PWDN to gound via 18k
2 NC
3 NC
4 NC
5 D1V5
6 D2V8
7 A2V8
8 Ground
9 NC
10 Ground
11 CLK
12 Ground
13 SDA
14 SCL
15 DN0
16 DP0
17 CN
18 CP
19 DN1
20 DP1
21 NC
22 NC
23 NC
24 NC

[jbeale]

However, I think I saw in the manual (but can't find it at the moment) that the output on the VSYNC pad is disabled by default and needs to be enabled by writing an I2C register.

Great work with the pinout Steve. According to the "PRELIMINARY" document, VSYNC defaults to an input after reset. The high bit of register 0x3002 SC_CMMN_PAD_OEN2 is Bit[7]: io_vsync_oen (OUTPUT ENABLE) with default value 0x00. There also appears to be a memory-mapped IO data register 0x300D SC_CMMN_PAD_OUT2 which is

Code: Select all

Bit[7]: io_vsync_o
Bit[6]: io_href_o
Bit[5]: io_pclk_o
Bit[4]: io_frex_o
Bit[3]: io_strobe_o
Bit[2]: io_sda_o
Bit[1]: io_gpio1_o
Bit[0]: io_gpio0_o

and then there is 0x3010 SC_CMMN_PAD_SEL2 which has the same bit mappings except being SELECT instead of data, eg Bit[7]: io_vsync_sel and so forth. And a *lot* more registers with various control functions. If you are in MIPI CSI-2 mode, as the R-Pi camera operates with, then maybe you cannot also get a VSYNC signal out even if you set the output enable? I do not know, and as far as I can tell, the datasheet I have does not say. As JamesH mentioned elsewhere, welcome to the world of inadequate camera data sheets.

It might be fun to make an OV5647 breakout board, although I have never done a BGA PCB before, I know that hobbyists can and have done reflowed BGA parts (check out reflow your PS3 motherboard with a hot air gun...) On the other hand, if it took the foundation a year to make their version of the camera board, there may be more difficulties than first appear.

By the way, by day I am board-level circuit designer at a small electronics company, we are an OEM and sell in the consumer market. My company actually has a possible application for a camera device such as this, and I made an inquiry to OmniVision but they have not been in touch. I suppose they are interested mostly in huge volume users, mobile phones etc. As far as I know, the OmniVision chips are not available in small quantities. But there are people claiming to sell it at least. http://www.alibaba.com/product-gs/69463 ... Image.html

[jbeale]

One more thing: you actually can buy a hobbyist-friendly PCB with the OV5642, from Ebay for $26 with free shipping. This is also a 5-MPixel camera module from OmniVision, with somewhat similar specs to the R-Pi camera. It uses the more widely available M12x0.5 lens type so getting other focal lengths is easier.

http://www.ebay.com/itm/5-Mega-pixel-Ca ... 3f20a676d1

There is an "ArduCam" board for Arduino(!) that can use this module, although not nearly to its full speed capacity. http://www.arducam.com/

Also: the full data sheet for the OV5642 mentions support for a second camera module. Typically this would be front & rear camera on a tablet or phone, I think this is an "either-or" situation, not intended for stereo (both at same time).

[renambot]

Hi,

Here's a quick video captured with two PIs and two cameras for stereo 3D:
http://youtu.be/S61ktNCexwg

Both cameras driven with a modified version of raspivid blocking before the capture command:
MMAL_PARAMETER_CAPTURE
The cameras are triggered using a physical switch wired to a GPIO pin on each Pi board.
The cameras are set to the same white balance (fluorescent if I remember) and auto-exposure.

The sync seems to be pretty OK (considering). I get a bad case of video flickering. I checked the flicker mode of the cameras and it seems to be set properly. I'll try on switching the power supplies as mentioned in other threads.

The only correction done in AfterEffect is a rough vertical alignment and stereo separation adjustment (no color correction, no advanced geometric correction, ...).

I'll try to post some photos of the setup later.

Luc

[jbeale]

Is that posted to YouTube in 3D? There is a "3D" button but I don't seem to be able to select any 3D controls by clicking on it.
The only flickering I see on that example is the autoexposure adjusting things (in step-wise fashion) as the chair rotates from darker backside to lighter frontside, and back again.

[renambot]

Yes, it's posted in 'squished aspect ratio' side-by-side stereo, then, at display time, youtube let you select your favorite stereo rendering (anaglyph, side-by-side, interlaced, top-bottom, off, ...) by clicking on the 3D button.

I guess the flickering is reduced by the youtube compression, but it's very noticeable in the source files, especially in the dark area.

Luc

[steve_gulick]

The YouTube video looks very good in 3-D to me using red /cyan glasses! Good job.

It would be very useful for more 3-D experimentation, and other neat stuff if the VSYNC could be enabled at the same time the video was being transmitted via the CSI mode. The VSYNC line from the OV5647 chip does connect to pin 3 on the 24 pin connector. It is not connected to any traces on the green PCB, but one can carefully solder a fine wire to the connector pin on the top side.

Steve

[rkinch]

Don't expect other people to do your job for you for free.

Of course, I don't expect any such thing. But shutter control and flash sync are fundamental to still photography, all the way back to daguerreotypes. The OmniVision chip properly provides these functions. So I don't know how else to characterize this near-miss in the current camera module, other than in some way "disappointing".

We understand such features were never in the scope of the project or your job or your pay, and the camera is useful for other many other educational and commercial applications. Nevertheless, it matters. Users will hear "programmable camera" and frequently ask the question, "how to synchronize it to external events", which is what any sensing device materially does. That FAQ apparently needs the answers, (1) it doesn't do that, (2) no plans to improve it to do that, (3) it can't be hacked because the functionality is part of proprietary and closed firmware, and (4) even if improved firmware or hacking were feasible, the physical BGA connections are missing.

Build you own board with the same camera on.

Hence the tragic aspect. The work is 99.99 percent done, but we users cannot contribute the 0.01 percent, much less plead for it. We are just freeloaders and complainers, told to *bleep* off and pay for our own, down to the first dollar, er, pound. Nobody's getting rich off this; it is a charitable enterprise. Charities tend to exhaustion and uncharitable consequences.

[PiGraham]

It is understandable, and reasonable, that RasPi doesn't exploit every feature. The real frustration is that the features are there but the information we would need to diy is not available.
So much around the Pi is open source that the closed nature of the hardware is a stark contrast.
If only it were a perfect world.

[jamesh]

Don't expect other people to do your job for you for free.

Of course, I don't expect any such thing. But shutter control and flash sync are fundamental to still photography, all the way back to daguerreotypes. The OmniVision chip properly provides these functions. So I don't know how else to characterize this near-miss in the current camera module, other than in some way "disappointing".

We understand such features were never in the scope of the project or your job or your pay, and the camera is useful for other many other educational and commercial applications. Nevertheless, it matters. Users will hear "programmable camera" and frequently ask the question, "how to synchronize it to external events", which is what any sensing device materially does. That FAQ apparently needs the answers, (1) it doesn't do that, (2) no plans to improve it to do that, (3) it can't be hacked because the functionality is part of proprietary and closed firmware, and (4) even if improved firmware or hacking were feasible, the physical BGA connections are missing.

Build you own board with the same camera on.

Hence the tragic aspect. The work is 99.99 percent done, but we users cannot contribute the 0.01 percent, much less plead for it. We are just freeloaders and complainers, told to *bleep* off and pay for our own, down to the first dollar, er, pound. Nobody's getting rich off this; it is a charitable enterprise. Charities tend to exhaustion and uncharitable consequences.

You are *such* a concern troll. I bet your glass is ALWAYS half empty. Just because something doesn't do exactly what YOU want, its disapointing? To everyone? No, I don't think so.

The module does what the majority wants. Flash sync was never even slightly considered. Shutter control simply doesn't work YET. That's just a software job which I havent had a chance to look at yet.

Synchronising of a capture to external events is easy - plumb in a control to a GPIO. Not difficult. Microsecond timing may not be possible.

The closed source GPU/ISP debate has been done to death,. Please do not bring up the subject again.

[alexeames]

rkinch - while you raise some valid points, and keep raising them - over and over and over again, ad nauseam, you don't seem to take the hint, even after a couple of warnings, that "concern trolling" is not appreciated here. (I'm not calling you a troll - that's not allowed here. Just suggesting that you may be sailing close to the wind.*)

I have absolutely no authority here, but I am getting frustrated by your continual goading of James, which must be extremely demotivating (or at the very least annoying) after months of spare time work. Please cut it out. It'll be done when and if it's done. And even if it's not, the camera is an excellent achievement as it is now.

I don't want you to 'bleep off', because I believe you have expertise to bring to the party.
But I would ask that you consider, with empathy and self-restraint, the impact of what you write before hitting the submit button.

Sorry I felt I had to bring this up, it's been brewing for a while.
(p.s. I was writing this while James wrote the post above this one.)

* Just my opinion, your mileage may vary. The negative impact of a concern troll post can go up as well as down, and shouldn't be seen as constructive criticism.

[drumminhands]

Here's a quick video captured with two PIs and two cameras for stereo 3D:
http://youtu.be/S61ktNCexwg

...

I'll try to post some photos of the setup later.

That's a lot of fun. Love to see the setup photos and if you are willing to share any code.

[rkinch]

You are *such* a concern troll. ... Please do not bring up the subject again.

Ouch. I'll admit to being more persistent than is welcome. But not to the insincerity of a troll.

Synchronising of a capture to external events is easy - plumb in a control to a GPIO.

That provides *asynchronous* event ordering. The actual moment of the captured frame is neither controllable nor detectable in real time, and is bracketed with arbitrarily long delays. Hence the others hacking into the cable signals, trying to impose control (via man-in-the-middle) or detection (via signal recognition). It's not just me.

[jamesh]

You are *such* a concern troll. ... Please do not bring up the subject again.

Ouch. I'll admit to being more persistent than is welcome. But not to the insincerity of a troll.

Synchronising of a capture to external events is easy - plumb in a control to a GPIO.

That provides *asynchronous* event ordering. The actual moment of the captured frame is neither controllable nor detectable in real time, and is bracketed with arbitrarily long delays. Hence the others hacking into the cable signals, trying to impose control (via man-in-the-middle) or detection (via signal recognition). It's not just me.

Hence the use of the phrase 'concern troll' rather than just troll.

Think about it for a moment. There are about 10 people at most on here so far discussing synchronisation. That a very small market for a lot of required work up front. Even if that market were to increase by a factor of 100, it would still not be worth spending the time on it. Some things are just not worth doing from a financial perspective, or from a time perspective when you are spending you own unpaid time on it.

[rkinch]

There are about 10 people at most on here so far discussing synchronisation. That a very small market ... not worth doing ...

And one of them an unlovely troll.

I'll just shuffle down now to my exile at the bottom of the wishlist.