jamesh wrote:Image System Pipeline is one definition (there are others). It's the path the image takes from being captures by the sensor to being fully processed and decent. On the GPU there are >20 stages to the ISP. The CSI-2 port where the camera attaches is just the first stage. That produces a raw bayer image, which is bebayered, black level corrected, lens shading corrected, denoised, sharpened, colour balanced, runs through the gain control, the white balance control. And that's just a subset of the processing...all done on the GPU at 30fps for 1080p.
Note that some camera sensors have all the processing built in - in fact the one being used does, but we don't actually use the internal ISP in this case, we use the one on the GPU, and just take bayer data from the camera.
It still sounds like many of the processes mentioned above could be done by cleverly written fragment shaders (render-to-texture) if public interfaces to use GLSL or Cg exist.
jamesh wrote:ref: You may be right should you 'you are right'
<<<Attack packet dropped by firewall>>>
They probably could be done with fragment shaders, but would be slower (general purpose code vs dedicated HW - code usually loses). Would take ages to write them all as well, and would''t provide the same level of programmability as available to the tuning on the GPU. The GPU support OGLES2.0, so you could do some GPGPU stuff if you were so inclined.
It's going to come down to speed. Rather than a fully GPU pipeline, you will be transferring the (large) bayer chunk of data to the Arm where it would be processed by a 700Mhz processor (compared with the 16way Vector code at 250Mhz = 4000Mhz , of which there are two, but not used exclusively for the ISP) with some bits passed back to the GPU for shader processing with the associated overheads. Lots of work for something that is slower than what is there already. Only benefit is user control of the whole tuning process.
In fact, as ARM's get faster there is benefit to running parts of the ISP on them. Not on this chip though.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."