User avatar
puppy101puppy
Posts: 16
Joined: Mon Feb 02, 2015 8:19 am

[Help] Shading Correction, Camera v2 after Lens Removed

Mon Jul 25, 2016 4:32 am

Hi,

I'm recently exploring the possibility to change lens of the rapsberry camera v2 and/or remove the lens completely for some contact imaging. I removed the lens and lens holder of camera v2 completely with the help of a razor blade (would love to share details if anyone is interested).
Image

However, after testing the lens-removed sensor response by illuminating with a plane wave (uniform illumination), the sensor response doesn't look quite uniform. The raw image shows some slowly varying response (a non-uniformity of ~30%), as can be seen in the red (R), green (G1) and blue (B) channels of this raw image below. (The raw image data is extracted from the large jpeg either using the "dcraw" or the documented python code. Thanks to those dedicated engineers and pioneering hackers!!!)
Image Image Image
(For instance, the green (G1) channel seems to be brighter in the middle instead of a uniform shade.) I'm wondering if anyone has seem similar non-uniformity problem of camera v2.

According to the Sony IMX219PQ document http://www.sony.net/Products/SC-HP/new_ ... 219_e.html,
The IMX219PQ has a lens shading correction function with four independent colors. It can be used for not only normal lens shading correction, and also can serve as an individual module adjustment of deviation in the lens assembly process during the module fabrication.
So this shading might be due to:
(1) The sensor fabrication defects.
(2) Shading correction for the lens, which is removed in my case.

Either of these seems to be related to some shade correction. Does anyone know if this correction is a "pre-processing" by adjusting the gain of each pixels (before digitized), or if this is a "post-processing" on the raw image (after digitized). (I've tried previously similar lens-removal for camera v1, OV5647 and it had uniform response. Not sure why this Sony IMX219 has such a non-uniformity in the raw image. :geek: )

Also from the Sony IMX219PQ document page,
Calibration data is written in a built-in OTP during the module inspection to contribute to the short loading time when the camera is turned on. One from three tables can be selected according to color temperature and there exists two-table blending mode for interpolation.
Is there a way to get or modify this built-in OTP, in order to beat or get around this auto- shade correction?

Thanks! :)

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Mon Jul 25, 2016 6:06 am

Congratulations on your advanced camera-hacking. I'm impressed you managed to take the housing off completely without damaging the sensor itself. It is interesting how much different the R,G,B channels are. I was going to suggest the difference was due to a radial offset of the microlens alignment relative to the pixel centers, but I don't know why each color channel would be so different as a result.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23869
Joined: Sat Jul 30, 2011 7:41 pm

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Mon Jul 25, 2016 9:47 am

I would have thought the internal LS would have been turned off, and it all done by the ISP on the VC4. So not sure what's going on.

My suspicion is that the microlenses on the sensor surface are generally tuned to a particular lens distance, and expect a lens to be there. ie they are offset from the pixel center the further you are from the sensor centre. So if there is no lens then you would get artifacts similar to what you see above.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7408
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Mon Jul 25, 2016 10:22 am

A quick read of the datasheet and I can't see anything being used for deliberate lens shading correction on the sensor - marketing spiel can often differ from the technical data. There is a defect correction block referenced, but that is using the OTP and I very much doubt it has been programmed (too great a production line cost).

What is shown is the Chief Ray Angle graph, and that indicates that the CRA is greater on IMX219 than OV5647 by about 20%. That may account for some of the variation in your captures.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

engberg
Posts: 1
Joined: Sat Aug 13, 2016 7:31 am

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Sat Aug 13, 2016 9:10 am

Have there been new insights into the v2 shading issues? I am getting very similar channels with a uniform field, perhaps with a bit more cyan vignetting with the v2 board. I had expected the cyan vignetting to be similar to the v1 sensor issues (viewtopic.php?f=43&t=73998), but then I would not have expected the result to show up in the RAW image channels.

@6by9: Is there a comparable "ov5647_acls=1" auto lens shading option for the v2 Sony sensor? I have tried that option along with/without an -st pass, with the most current updates through apt-get update/upgrade and rpi-update, but none solve the issue.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7408
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Sat Aug 13, 2016 9:34 pm

engberg wrote:@6by9: Is there a comparable "ov5647_acls=1" auto lens shading option for the v2 Sony sensor? I have tried that option along with/without an -st pass, with the most current updates through apt-get update/upgrade and rpi-update, but none solve the issue.
No, it is enabled by default for imx219.
I couldn't confirm for definite that all was working as expected on OV5647, hence why it isn't enabled by default there.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ned5173
Posts: 6
Joined: Thu Sep 01, 2016 11:46 pm

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Tue Sep 06, 2016 2:35 am

I was attracted to your posting because I am interested in changing the focal distance of the rpi (spy) camera to allow focusing of the image formed by two plano-concave lenses placed in front of the rpi lens. I am able to do this with a web camera but it's all very bulky. I have seen postings of IR filter removal involving separation of the camera backing from the plastic housing. I need to remove the filter too but wondered if the plastic housing was cut through behind the lens then perhaps I could retrofit a means of moving the lens back and forward and extract the filter as well.
Since you have removed a lens I expect you know what is behind it and whether my plan to remove the IR filter in this way is possible. I could sacrifice the camera to find out what is inside the plastic housing but some prior information might prevent needless butchery.

User avatar
Gavinmc42
Posts: 3878
Joined: Wed Aug 28, 2013 3:31 am

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Tue Sep 06, 2016 5:07 am

You might be better getting the NOIR version.
It seems the IR block is a bit of film on the sensor itself not in the lens assembly.

But if you are talking about the tiny Adafruit one, no idea.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Rado
Posts: 9
Joined: Mon Apr 08, 2013 12:51 pm
Location: Slovakia

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Wed Oct 19, 2016 9:26 pm

The new IMX219-based Pi Camera looked so promising to replace the old one for a film scanning project, when I ran across this discussion... I was concerned when I saw the notes on lens shading correction, but I didn't expect it to be this bad. Puppy101puppy (or anyone else) did you ever figure out a way to disable it? My project involves lens removal / replacement so this would be a show stopper.

According to the Raspberry Pi Blog, it seems it was quite an effort to implement / fine tune the lens shading for the Pi Camera v2: "Regarding the tuning process, I guess you could say the bulk of the effort went into the lens shading and AWB tuning". (Source)

So perhaps someone from the foundation could comment? If I understand this right, without disabling, it would make the camera hard (if not impossible) to use with projects that involve different lenses.

Kozuch
Posts: 64
Joined: Sun Aug 10, 2014 10:35 am

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Sun Feb 19, 2017 12:10 am

Any news on this topic? If the lens shading can not be disabled then as Rado says third party lenses may easily be out of game...

The only way seems to be using the raw Bayer output as suggested here - can someone confirm it will turn the shading off? Also, if I understand correctly, shading is only problem of v2 camera, v1 should be fine, right?

What about lens distortion correction that is being mentioned in picamera's docs? I think it may introduce a similar problem as shading...

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7408
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Sun Feb 19, 2017 8:20 am

Kozuch wrote:The only way seems to be using the raw Bayer output as suggested here - can someone confirm it will turn the shading off?
It has been stated many times on these forums already. (I don't think any of the engineers from Pi Towers routinely frequent stackexchange as the forums take up enough time).
The raw image is direct off the sensor with no additional processing at all.
The only parameters that will make a difference are those which operate directly on the sensor - exposure time, analogue gain, and flips.
Kozuch wrote:Also, if I understand correctly, shading is only problem of v2 camera, v1 should be fine, right?
Er, how does physics know that the lens is in front of the camera is a v1 or v2? That would be magic, and I want a similar option to turn gravity on and off :roll:
Read the Pixel vignetting section of https://en.wikipedia.org/wiki/Vignetting. That is what is being compensated for.
Kozuch wrote:What about lens distortion correction that is being mentioned in picamera's docs? I think it may introduce a similar problem as shading...
No geometric distortion is configured as the stock lens isn't a fisheye that would need correction. It is an option in the ISP should a future sensor need it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Kozuch
Posts: 64
Joined: Sun Aug 10, 2014 10:35 am

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Sun Feb 19, 2017 11:21 am

Hi 6by9,
6by9 wrote:Er, how does physics know that the lens is in front of the camera is a v1 or v2? That would be magic, and I want a similar option to turn gravity on and off :roll:
Read the Pixel vignetting section of https://en.wikipedia.org/wiki/Vignetting. That is what is being compensated for.
well I meant the inability to turn off shading correction - if that was a problem of v2 camera only... But in the meantime I read that v1 camera also can not turn it off (???) but probably v1 algo is not so aggressive so third party lenses seem to work quite well based on the images I have seen.

Talking to you - would you mind replying to my post in the flash driver thread? Thanks in advance!

User avatar
Rado
Posts: 9
Joined: Mon Apr 08, 2013 12:51 pm
Location: Slovakia

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Sun Feb 19, 2017 4:33 pm

Kozuch wrote:well I meant the inability to turn off shading correction - if that was a problem of v2 camera only... But in the meantime I read that v1 camera also can not turn it off (???) but probably v1 algo is not so aggressive so third party lenses seem to work quite well based on the images I have seen.
I think this may be true. When I've been looking closely at v1 camera pictures taken with a 3rd party lens, the corners were slightly brighter (i.e. the inversion of the usual lens vignetting) so perhaps the v1 camera does corrections as well, but they are less pronounced and more smooth (and thus harder to notice?)
6by9 wrote:No geometric distortion is configured as the stock lens isn't a fisheye that would need correction. It is an option in the ISP should a future sensor need it.
Barrel or pincushion distortion corrections are quite common these days, as imaging processors have become powerful enough to make them "on the fly" - so in-camera JPEGs are usually corrected as the photo is taken in today's compact cameras. The reason is that no real-world lens is perfectly rectilinear and a bit of distortion is always there (this is especially true for zoom lenses). That is not to say that the Pi Camera does anything like that, just wanted to comment on the fisheye bit. And thanks for the note on raw image data!

kif38
Posts: 1
Joined: Mon Jun 05, 2017 3:02 pm

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Mon Jun 05, 2017 3:24 pm

What about having those "correction" tabulated and apply the correction the other way around ?

I have myself a RPI camera v1 with a CS mount which have "good" result (for timelaps) excepted a darker band in the lower part of the image (corrected in software in post-processing). This is probably a hardware bug on my camera.
My plan was to replace this module with a camera v2, print a 3D housing for a M12 lens and add a wide-angle lens (not fish-eye).
I figured out threading the M12 was a little tricky. Now I have a correct focal plan but images exhibit some colored smooth circles which correspond to the issue described here.

Assuming replacement lenses are "much" better than the stock one, can we tabulate this correction and apply the reverse one
on the RGB image ?

Cheers,

Jerome

Return to “Camera board”