Page 1 of 1

[Help] Shading Correction, Camera v2 after Lens Removed

Posted: Mon Jul 25, 2016 4:32 am
by puppy101puppy
Hi,

I'm recently exploring the possibility to change lens of the rapsberry camera v2 and/or remove the lens completely for some contact imaging. I removed the lens and lens holder of camera v2 completely with the help of a razor blade (would love to share details if anyone is interested).
Image

However, after testing the lens-removed sensor response by illuminating with a plane wave (uniform illumination), the sensor response doesn't look quite uniform. The raw image shows some slowly varying response (a non-uniformity of ~30%), as can be seen in the red (R), green (G1) and blue (B) channels of this raw image below. (The raw image data is extracted from the large jpeg either using the "dcraw" or the documented python code. Thanks to those dedicated engineers and pioneering hackers!!!)
Image Image Image
(For instance, the green (G1) channel seems to be brighter in the middle instead of a uniform shade.) I'm wondering if anyone has seem similar non-uniformity problem of camera v2.

According to the Sony IMX219PQ document http://www.sony.net/Products/SC-HP/new_ ... 219_e.html,
The IMX219PQ has a lens shading correction function with four independent colors. It can be used for not only normal lens shading correction, and also can serve as an individual module adjustment of deviation in the lens assembly process during the module fabrication.
So this shading might be due to:
(1) The sensor fabrication defects.
(2) Shading correction for the lens, which is removed in my case.

Either of these seems to be related to some shade correction. Does anyone know if this correction is a "pre-processing" by adjusting the gain of each pixels (before digitized), or if this is a "post-processing" on the raw image (after digitized). (I've tried previously similar lens-removal for camera v1, OV5647 and it had uniform response. Not sure why this Sony IMX219 has such a non-uniformity in the raw image. :geek: )

Also from the Sony IMX219PQ document page,
Calibration data is written in a built-in OTP during the module inspection to contribute to the short loading time when the camera is turned on. One from three tables can be selected according to color temperature and there exists two-table blending mode for interpolation.
Is there a way to get or modify this built-in OTP, in order to beat or get around this auto- shade correction?

Thanks! :)

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Mon Jul 25, 2016 6:06 am
by jbeale
Congratulations on your advanced camera-hacking. I'm impressed you managed to take the housing off completely without damaging the sensor itself. It is interesting how much different the R,G,B channels are. I was going to suggest the difference was due to a radial offset of the microlens alignment relative to the pixel centers, but I don't know why each color channel would be so different as a result.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Mon Jul 25, 2016 9:47 am
by jamesh
I would have thought the internal LS would have been turned off, and it all done by the ISP on the VC4. So not sure what's going on.

My suspicion is that the microlenses on the sensor surface are generally tuned to a particular lens distance, and expect a lens to be there. ie they are offset from the pixel center the further you are from the sensor centre. So if there is no lens then you would get artifacts similar to what you see above.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Mon Jul 25, 2016 10:22 am
by 6by9
A quick read of the datasheet and I can't see anything being used for deliberate lens shading correction on the sensor - marketing spiel can often differ from the technical data. There is a defect correction block referenced, but that is using the OTP and I very much doubt it has been programmed (too great a production line cost).

What is shown is the Chief Ray Angle graph, and that indicates that the CRA is greater on IMX219 than OV5647 by about 20%. That may account for some of the variation in your captures.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Sat Aug 13, 2016 9:10 am
by engberg
Have there been new insights into the v2 shading issues? I am getting very similar channels with a uniform field, perhaps with a bit more cyan vignetting with the v2 board. I had expected the cyan vignetting to be similar to the v1 sensor issues (viewtopic.php?f=43&t=73998), but then I would not have expected the result to show up in the RAW image channels.

@6by9: Is there a comparable "ov5647_acls=1" auto lens shading option for the v2 Sony sensor? I have tried that option along with/without an -st pass, with the most current updates through apt-get update/upgrade and rpi-update, but none solve the issue.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Sat Aug 13, 2016 9:34 pm
by 6by9
engberg wrote:@6by9: Is there a comparable "ov5647_acls=1" auto lens shading option for the v2 Sony sensor? I have tried that option along with/without an -st pass, with the most current updates through apt-get update/upgrade and rpi-update, but none solve the issue.
No, it is enabled by default for imx219.
I couldn't confirm for definite that all was working as expected on OV5647, hence why it isn't enabled by default there.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Tue Sep 06, 2016 2:35 am
by ned5173
I was attracted to your posting because I am interested in changing the focal distance of the rpi (spy) camera to allow focusing of the image formed by two plano-concave lenses placed in front of the rpi lens. I am able to do this with a web camera but it's all very bulky. I have seen postings of IR filter removal involving separation of the camera backing from the plastic housing. I need to remove the filter too but wondered if the plastic housing was cut through behind the lens then perhaps I could retrofit a means of moving the lens back and forward and extract the filter as well.
Since you have removed a lens I expect you know what is behind it and whether my plan to remove the IR filter in this way is possible. I could sacrifice the camera to find out what is inside the plastic housing but some prior information might prevent needless butchery.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Tue Sep 06, 2016 5:07 am
by Gavinmc42
You might be better getting the NOIR version.
It seems the IR block is a bit of film on the sensor itself not in the lens assembly.

But if you are talking about the tiny Adafruit one, no idea.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Wed Oct 19, 2016 9:26 pm
by Rado
The new IMX219-based Pi Camera looked so promising to replace the old one for a film scanning project, when I ran across this discussion... I was concerned when I saw the notes on lens shading correction, but I didn't expect it to be this bad. Puppy101puppy (or anyone else) did you ever figure out a way to disable it? My project involves lens removal / replacement so this would be a show stopper.

According to the Raspberry Pi Blog, it seems it was quite an effort to implement / fine tune the lens shading for the Pi Camera v2: "Regarding the tuning process, I guess you could say the bulk of the effort went into the lens shading and AWB tuning". (Source)

So perhaps someone from the foundation could comment? If I understand this right, without disabling, it would make the camera hard (if not impossible) to use with projects that involve different lenses.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Sun Feb 19, 2017 12:10 am
by Kozuch
Any news on this topic? If the lens shading can not be disabled then as Rado says third party lenses may easily be out of game...

The only way seems to be using the raw Bayer output as suggested here - can someone confirm it will turn the shading off? Also, if I understand correctly, shading is only problem of v2 camera, v1 should be fine, right?

What about lens distortion correction that is being mentioned in picamera's docs? I think it may introduce a similar problem as shading...

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Sun Feb 19, 2017 8:20 am
by 6by9
Kozuch wrote:The only way seems to be using the raw Bayer output as suggested here - can someone confirm it will turn the shading off?
It has been stated many times on these forums already. (I don't think any of the engineers from Pi Towers routinely frequent stackexchange as the forums take up enough time).
The raw image is direct off the sensor with no additional processing at all.
The only parameters that will make a difference are those which operate directly on the sensor - exposure time, analogue gain, and flips.
Kozuch wrote:Also, if I understand correctly, shading is only problem of v2 camera, v1 should be fine, right?
Er, how does physics know that the lens is in front of the camera is a v1 or v2? That would be magic, and I want a similar option to turn gravity on and off :roll:
Read the Pixel vignetting section of https://en.wikipedia.org/wiki/Vignetting. That is what is being compensated for.
Kozuch wrote:What about lens distortion correction that is being mentioned in picamera's docs? I think it may introduce a similar problem as shading...
No geometric distortion is configured as the stock lens isn't a fisheye that would need correction. It is an option in the ISP should a future sensor need it.

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Sun Feb 19, 2017 11:21 am
by Kozuch
Hi 6by9,
6by9 wrote:Er, how does physics know that the lens is in front of the camera is a v1 or v2? That would be magic, and I want a similar option to turn gravity on and off :roll:
Read the Pixel vignetting section of https://en.wikipedia.org/wiki/Vignetting. That is what is being compensated for.
well I meant the inability to turn off shading correction - if that was a problem of v2 camera only... But in the meantime I read that v1 camera also can not turn it off (???) but probably v1 algo is not so aggressive so third party lenses seem to work quite well based on the images I have seen.

Talking to you - would you mind replying to my post in the flash driver thread? Thanks in advance!

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Sun Feb 19, 2017 4:33 pm
by Rado
Kozuch wrote:well I meant the inability to turn off shading correction - if that was a problem of v2 camera only... But in the meantime I read that v1 camera also can not turn it off (???) but probably v1 algo is not so aggressive so third party lenses seem to work quite well based on the images I have seen.
I think this may be true. When I've been looking closely at v1 camera pictures taken with a 3rd party lens, the corners were slightly brighter (i.e. the inversion of the usual lens vignetting) so perhaps the v1 camera does corrections as well, but they are less pronounced and more smooth (and thus harder to notice?)
6by9 wrote:No geometric distortion is configured as the stock lens isn't a fisheye that would need correction. It is an option in the ISP should a future sensor need it.
Barrel or pincushion distortion corrections are quite common these days, as imaging processors have become powerful enough to make them "on the fly" - so in-camera JPEGs are usually corrected as the photo is taken in today's compact cameras. The reason is that no real-world lens is perfectly rectilinear and a bit of distortion is always there (this is especially true for zoom lenses). That is not to say that the Pi Camera does anything like that, just wanted to comment on the fisheye bit. And thanks for the note on raw image data!

Re: [Help] Shading Correction, Camera v2 after Lens Removed

Posted: Mon Jun 05, 2017 3:24 pm
by kif38
What about having those "correction" tabulated and apply the correction the other way around ?

I have myself a RPI camera v1 with a CS mount which have "good" result (for timelaps) excepted a darker band in the lower part of the image (corrected in software in post-processing). This is probably a hardware bug on my camera.
My plan was to replace this module with a camera v2, print a 3D housing for a M12 lens and add a wide-angle lens (not fish-eye).
I figured out threading the M12 was a little tricky. Now I have a correct focal plan but images exhibit some colored smooth circles which correspond to the issue described here.

Assuming replacement lenses are "much" better than the stock one, can we tabulate this correction and apply the reverse one
on the RGB image ?

Cheers,

Jerome