eggn1n3
Posts: 106
Joined: Fri Jul 29, 2011 10:36 am

Re: Camera for R-Pi

Fri Sep 09, 2011 12:58 pm

I found a nice 1.3M USB camera for R-Pi projects:
http://www.delock.com/produkte.....95852.html. It's small and supported by Linux; I have seen it around for 30-35 euro's.

Update: Found another one, 2Mb: http://www.e-consystems.com/20.....rd_usb.asp, 45 USD.

User avatar
liz
Raspberry Pi Foundation Employee & Forum Moderator
Raspberry Pi Foundation Employee & Forum Moderator
Posts: 5202
Joined: Thu Jul 28, 2011 7:22 pm
Contact: Website

Re: Camera for R-Pi

Fri Sep 09, 2011 1:03 pm

We're talking to some optical sensor people at the moment - all being well, we'll be able to offer a camera module for sale (two, actually - we're looking at one with a very high megapixel count and a less expensive one with a much lower count, because we know you'll have different needs for different projects) next year. I'm not allowed to tell you more at the moment, but thought you'd like to know that you'll have the option soon.
Director of Communications, Raspberry Pi

rjw57
Posts: 33
Joined: Wed Sep 07, 2011 1:21 pm
Contact: Website

Re: Camera for R-Pi

Fri Sep 09, 2011 1:04 pm

I bought this cheapy webcam and headset thing from Maplin for work which is a bit cheaper (if not as cool): http://www.maplin.co.uk/exis-c.....cam-510695

It also works with Linux. Thankfully most webcams now do!

Lakes
Posts: 267
Joined: Wed Aug 24, 2011 2:17 pm

Re: Camera for R-Pi

Fri Sep 09, 2011 1:08 pm

I`m thinking DIY camcorder...

User avatar
liz
Raspberry Pi Foundation Employee & Forum Moderator
Raspberry Pi Foundation Employee & Forum Moderator
Posts: 5202
Joined: Thu Jul 28, 2011 7:22 pm
Contact: Website

Re: Camera for R-Pi

Fri Sep 09, 2011 1:15 pm

Helmet cam!
Director of Communications, Raspberry Pi

eggn1n3
Posts: 106
Joined: Fri Jul 29, 2011 10:36 am

Re: Camera for R-Pi

Fri Sep 09, 2011 1:46 pm

Thanks Liz. Good idea to have two types of camera's. I would be interested in a high resolution one. Although you are not allowed to tell anything about it, can you share what type of connector you are thinking of? USB or something with a special connector like we have seen in the previous R-Pi prototype using the 12Megapixel camera module.

User avatar
liz
Raspberry Pi Foundation Employee & Forum Moderator
Raspberry Pi Foundation Employee & Forum Moderator
Posts: 5202
Joined: Thu Jul 28, 2011 7:22 pm
Contact: Website

Re: Camera for R-Pi

Fri Sep 09, 2011 1:55 pm

Probably a special connector as on the prototype.
Director of Communications, Raspberry Pi

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23918
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera for R-Pi

Fri Sep 09, 2011 2:08 pm

I can give you some generic camera related answers, but nothing specific.

USB is simply not fast enough for larger resolution cameras (30fps at 1080P)- so a special interface is needed - usually CSI2 (although lower resolution cameras are sometimes CCP2). The GPU supports both, although how that is brought out to headers is yet to be decided. Because the camera attaches to the GPU the drivers are on the GPU, it requires the binary blob to be updated to include drivers and tuning for the particular camera. The GPU contains a very sophisticated ISP - the pipeline of image processing to get some raw camera data to pretty JPG's. This pipeline also needs to be 'tuned' to the particular camera and this can take some times (months for a specialised team to do a really good job). Look up reviews on the Nokia N8 for how good the imaging system is.

All the work above needs to be done for each new camera module - and will be done for any camera sold for the Raspi.

The other problem with cameras is cost in low numbers. Mobile phone manufactures buy in the millions and get huge discounts. Otherwise, for example, a 12MP sensor would cost £50 or more. It's finding someone to sell you one at the sort of prices normally reserved for large quantities that's the difficult bit!

Having said all that - it will be done! Even if I have to write camera drivers and tune it myself! But hopefully that gives people some idea of the complexities involved - it's not just a simple 'grab a module and plug it in' job.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23918
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera for R-Pi

Fri Sep 09, 2011 2:15 pm

Forgot to add, you could still use a USB camera as long as its supported by Linux without any of the above hassles. You won't get the same resolution or quality levels though.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

toxibunny
Posts: 1382
Joined: Thu Aug 18, 2011 9:21 pm

Re: Camera for R-Pi

Fri Sep 09, 2011 2:53 pm

I'm glad to hear the camera hasn't been dropped. Nice one raspers!
note: I may or may not know what I'm talking about...

jasonl
Posts: 21
Joined: Tue Aug 30, 2011 8:26 am

Re: Camera for R-Pi

Sat Sep 10, 2011 9:19 am

Quote from jamesh on September 9, 2011, 15:08
I can give you some generic camera related answers, but nothing specific.

USB is simply not fast enough for larger resolution cameras (30fps at 1080P)- so a special interface is needed - usually CSI2 (although lower resolution cameras are sometimes CCP2). The GPU supports both, although how that is brought out to headers is yet to be decided. Because the camera attaches to the GPU the drivers are on the GPU, it requires the binary blob to be updated to include drivers and tuning for the particular camera. The GPU contains a very sophisticated ISP - the pipeline of image processing to get some raw camera data to pretty JPG's. This pipeline also needs to be 'tuned' to the particular camera and this can take some times (months for a specialised team to do a really good job). Look up reviews on the Nokia N8 for how good the imaging system is.

All the work above needs to be done for each new camera module - and will be done for any camera sold for the Raspi.
[...]
Having said all that - it will be done! Even if I have to write camera drivers and tune it myself! But hopefully that gives people some idea of the complexities involved - it's not just a simple 'grab a module and plug it in' job.

This is intriguing - but I'm a little bit concerned that we are seeing more binary blobs, but inevitable given the nature of the device and available tech. Hope some thought goes for future maintainability - that the software accessing the blobs can be recompiled so they would work if future ABI's change. There have been issues with binary webcam drivers and the linux kernel in the past. Philips cameras spring to mind.

I'd like to see the camera software be able deliver uncompressed raw data from the sensor, not just JPEG or lossy encoded video. Framerate will obviously be lower due to the larger data transfer involved, but quality sometimes is required over speed. Think astronomical imaging. A lot of amateur astronomers use machine vision cameras for imaging - these cost hundreds of pounds and it could be possible make the R-Pi into a high quality image capture system at a fraction of the price.

Being able to use the camera with standard software interfaces is important. In addition to the exisiting Linux Video For Linux 2 interface, there exists the libdc1394 library. Both are used by OpenCV. OpenCV is quite an exciting library if you're into machine vision and robotics. There are good Python bindings, too.

And it would be good if the R-Pi camera drivers could provide an interface with libdc1394 so existing software using the library would work. dc1394 isn't just about Firewire - some cameras use this protocol over USB and gigabit ethernet. I've been using a dc1394 camera connected via USB (Point Grey Research Chameleon) to capture raw images for my 8mm telecine project - it's been quite a learning curve.

http://sourceforge.net/project.....libdc1394/
http://damien.douxchamps.net/i.....libdc1394/
http://opencv.willowgarage.com/wiki/

I'm not much into the internals of these libraries work - just as a geeky user who wants to plug a camera in, and write some Python code to do some interesting stuff with the images. As a photographer I like to work with the highest quality I can with the data from the sensor. ;-)

USvER
Posts: 21
Joined: Sun Aug 14, 2011 6:45 pm

Re: Camera for R-Pi

Thu Sep 29, 2011 6:57 pm

Ok, using embeded ISP for camera modules is hard task... But what about hardware video encoding?
I have idea of HD IP camera that can stream 720p30 H264 over the network. This job can be done even with USB webcam from logitech... but what about encoding?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23918
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera for R-Pi

Thu Sep 29, 2011 7:01 pm

Quote from USvER on September 29, 2011, 19:57
Ok, using embeded ISP for camera modules is hard task... But what about hardware video encoding?
I have idea of HD IP camera that can stream 720p30 H264 over the network. This job can be done even with USB webcam from logitech... but what about encoding?

Although the GPU is quite capable of encoding that stream in real time, I'm not sure what API is being provided to give you encode access to the GPU. This sort of thing is still being decided.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

david13lt
Posts: 20
Joined: Thu Sep 08, 2011 4:55 am

Re: Camera for R-Pi

Fri Sep 30, 2011 7:41 am

Using HD web camera will probably require a lot of CPU resource from the system, not GPU. I have heard that there should be some webcams with co-processor inside, so they don't require CPU resource to process the data. As I understand, such webcams would cost more than traditional ones, but minimizes requirements for the machine.

S_Doomfist
Posts: 5
Joined: Mon Sep 19, 2011 9:51 am

Re: Camera for R-Pi

Fri Sep 30, 2011 9:30 am

I wonder if creating a Raspberry Pi camera would be a good project (goes off to look up some info)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23918
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera for R-Pi

Fri Sep 30, 2011 9:33 am

Quote from david13lt on September 30, 2011, 08:41
Using HD web camera will probably require a lot of CPU resource from the system, not GPU. I have heard that there should be some webcams with co-processor inside, so they don't require CPU resource to process the data. As I understand, such webcams would cost more than traditional ones, but minimizes requirements for the machine.

True. You needs 1280*720*10 (bpp) * 30 = 26MB/s transfer/processing rate before encoding, which might be a bit much for the CPU.

Not problem on the GPU which can process 1080p30, so best bet is a camera attached to that. Plan is to have one eventually I believe, but won't be available at first release.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

USvER
Posts: 21
Joined: Sun Aug 14, 2011 6:45 pm

Re: Camera for R-Pi

Fri Sep 30, 2011 1:20 pm

Quote from jamesh on September 30, 2011, 10:33
Quote from david13lt on September 30, 2011, 08:41
Using HD web camera will probably require a lot of CPU resource from the system, not GPU. I have heard that there should be some webcams with co-processor inside, so they don't require CPU resource to process the data. As I understand, such webcams would cost more than traditional ones, but minimizes requirements for the machine.

True. You needs 1280*720*10 (bpp) * 30 = 26MB/s transfer/processing rate before encoding, which might be a bit much for the CPU.

Not problem on the GPU which can process 1080p30, so best bet is a camera attached to that. Plan is to have one eventually I believe, but won't be available at first release.

Picture coming from camera is MJPEG not raw data... Then jpeg decoding come into play and it shuld be hardware decoded(no one decode JPEG on ARM cpu)... Then it shuld be encoded to H264... And then stream...
As far as i know there is no data copy... there only reference passing(in theory, with a proper API)... So it shuld be do'able!
I realy want to see this functionality... So please make my year! :D

PS. Some interesting info on web cams http://www.quickcamteam.net
PPS. I'm not the logitech emloye or even fan... But i found their cheap UVC HD web cams a good choice for this!

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23918
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera for R-Pi

Fri Sep 30, 2011 1:58 pm

Quote from USvER on September 30, 2011, 14:20
Quote from jamesh on September 30, 2011, 10:33
Quote from david13lt on September 30, 2011, 08:41
Using HD web camera will probably require a lot of CPU resource from the system, not GPU. I have heard that there should be some webcams with co-processor inside, so they don't require CPU resource to process the data. As I understand, such webcams would cost more than traditional ones, but minimizes requirements for the machine.

True. You needs 1280*720*10 (bpp) * 30 = 26MB/s transfer/processing rate before encoding, which might be a bit much for the CPU.

Not problem on the GPU which can process 1080p30, so best bet is a camera attached to that. Plan is to have one eventually I believe, but won't be available at first release.

Picture coming from camera is MJPEG not raw data... Then jpeg decoding come into play and it shuld be hardware decoded(no one decode JPEG on ARM cpu)... Then it shuld be encoded to H264... And then stream...
As far as i know there is no data copy... there only reference passing(in theory, with a proper API)... So it shuld be do'able!
I realy want to see this functionality... So please make my year! :D

Certainly doable, just not sure it will be done. Trying to think if there is a decode JPEG API - not sure. What's really needed is a transcode from MJPEG to H264 which keeps everything in the GPU , which I am pretty sure isn't in place at the moment. Might be possible with an OpenMAX chain...
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Scribe
Posts: 98
Joined: Wed Sep 28, 2011 12:00 pm
Contact: Website

Re: Camera for R-Pi

Sat Oct 01, 2011 8:16 pm

Really looking forward to hearing more about the high megapixel option, we've just been looking at omniVision 5mp but if it's not going to be possible to integrate it with relative ease could really do with knowing.

jshriver
Posts: 16
Joined: Sat Oct 08, 2011 2:41 am
Contact: Website

Re: Camera for R-Pi

Fri Oct 14, 2011 3:31 am

In all fairness since this is linux and this has USB support I'd wage pretty much any webcam will work via v4l2 , also look into gphoto2 project for regular consumer hand held digital camera control.

jshriver
Posts: 16
Joined: Sat Oct 08, 2011 2:41 am
Contact: Website

Re: Camera for R-Pi

Fri Oct 14, 2011 3:46 am

Quote from jamesh on September 29, 2011, 20:01

Although the GPU is quite capable of encoding that stream in real time, I'm not sure what API is being provided to give you encode access to the GPU. This sort of thing is still being decided.

Maybe mencoder or ffmpeg via v4l2 interface. Not sure if it offers GPU optimization but a possibility and open source so if someone was interested it could be added.

Rasz
Posts: 24
Joined: Thu Oct 13, 2011 10:28 pm

Re: Camera for R-Pi

Fri Oct 14, 2011 4:25 am

The GPU supports both, although how that is brought out to headers is yet to be decided. Because the camera attaches to the GPU the drivers are on the GPU, it requires the binary blob to be updated to include drivers and tuning for the particular camera. The GPU contains a very sophisticated ISP - the pipeline of image processing to get some raw camera data to pretty JPG's. This pipeline also needs to be 'tuned' to the particular camera and this can take some times (months for a specialised team to do a really good job).

:(
was hoping they were standardized like parallel ones

Look up reviews on the Nokia N8 for how good the imaging system is.

Wait, did you use same module as in Nokia N8?

The other problem with cameras is cost in low numbers. Mobile phone manufactures buy in the millions and get huge discounts. Otherwise, for example, a 12MP sensor would cost £50 or more. It's finding someone to sell you one at the sort of prices normally reserved for large quantities that's the difficult bit!

Shouldnt be a problem. You could leverage economy of scale just like you did with Broadcom. Just pick camera module used in a popular cellphone (SE/Nokia) and chinese replacement parts suppliers will take care of the rest.
Example http://www.ebay.co.uk/sch/i.ht.....era+Module
http://www.ebay.co.uk/sch/i.ht.....era+Module

cant beat £6 8Mpix module (im not sure if its MIPI, it is Sony and half of their high mpix modules are mipi. + would probably require additional 1.2v and 1.8v regulators).

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23918
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera for R-Pi

Fri Oct 14, 2011 11:13 am

Quote from Rasz on October 14, 2011, 05:25
The GPU supports both, although how that is brought out to headers is yet to be decided. Because the camera attaches to the GPU the drivers are on the GPU, it requires the binary blob to be updated to include drivers and tuning for the particular camera. The GPU contains a very sophisticated ISP - the pipeline of image processing to get some raw camera data to pretty JPG's. This pipeline also needs to be 'tuned' to the particular camera and this can take some times (months for a specialised team to do a really good job).

:(
was hoping they were standardized like parallel ones

Look up reviews on the Nokia N8 for how good the imaging system is.

Wait, did you use same module as in Nokia N8?

The other problem with cameras is cost in low numbers. Mobile phone manufactures buy in the millions and get huge discounts. Otherwise, for example, a 12MP sensor would cost £50 or more. It's finding someone to sell you one at the sort of prices normally reserved for large quantities that's the difficult bit!

Shouldnt be a problem. You could leverage economy of scale just like you did with Broadcom. Just pick camera module used in a popular cellphone (SE/Nokia) and chinese replacement parts suppliers will take care of the rest.
Example http://www.ebay.co.uk/sch/i.ht.....era+Module
http://www.ebay.co.uk/sch/i.ht.....era+Module

cant beat £6 8Mpix module (im not sure if its MIPI, it is Sony and half of their high mpix modules are mipi. + would probably require additional 1.2v and 1.8v regulators).

What do you mean same module as N8? The GPU in the Raspi is a souped up version of the one in the N8 (it's considerably faster and has a better ISP). The N8 camera module isn't available to the general public AFAIK. Also the tuning is copyright Nokia I believe.

Not sure why you think it 'shouldn't be a problem'. Because it most certainly is. We don't have leverage with the camera people, but we do have sway with Broadcom. Economies of scale for these things comes in the 100's of thousands, not in the thousands. We also need to pick a camera that we have already tuned (an untuned camera looks awful, and we won't write new drivers and tune one specifically for this project), which limits what we can get, and it needs to be one we can actually buy. Also needs to be CSI2 (Mipi) or CCP2.

That said, might be some news forthcoming on cameras in the near future...
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Rasz
Posts: 24
Joined: Thu Oct 13, 2011 10:28 pm

Re: Camera for R-Pi

Fri Oct 14, 2011 3:14 pm

Quote from jamesh on October 14, 2011, 12:13
What do you mean same module as N8?

I wrongly assumed you meant exact N8 camera module, not comparable solution.

Quote from jamesh on October 14, 2011, 12:13
The N8 camera module isn't available to the general public AFAIK.
It is custom. Sadly I cant even source replacement modules so its no good to us.
Edit: $30
http://www.ebay.com/itm/Origin.....45f90c69c6
Pricey :(

Quote from jamesh on October 14, 2011, 12:13
Also the tuning is copyright Nokia I believe.

If by tuning you mean register setup for camera ASIC then its debatable. Its not like they wrote a program, its just settings. Anyway its irrelevant as community can come up with excellent settings for any camera module we/you pick.
Good example would be Sony IMX020 used in Sony Ericsson K850 (easily >50milion units sold). Pretty ancient phone from 2007. Replacement modules available at $8. Registers reverse engineered by enthusiasts, new tweaks to this day
http://www.esato.com/board/vie.....pic=200545
Sony calls it "2 channel sub-LVDS" interface. No idea if its compatible with CSI/CCP

N95 module MT9P001 is CCP, costs $15.
http://www.isgchips.com/pdf/mt.....0.book.pdf

Not sure why you think it 'shouldn't be a problem'. Because it most certainly is. We don't have leverage with the camera people, but we do have sway with Broadcom. Economies of scale for these things comes in the 100's of thousands, not in the thousands.

Start with a module that is already used in a popular cellphone similar to my examples above. It will be easy to obtain, with guaranteed supply for a long time and very low price.

We also need to pick a camera that we have already tuned (an untuned camera looks awful, and we won't write new drivers and tune one specifically for this project), which limits what we can get, and it needs to be one we can actually buy. Also needs to be CSI2 (Mipi) or CCP2.

We need V4L2 /dev/video, basically a buffer with data received from camera module. I have no experience with CSI/CCP, but driving paraller cameras only required the driver to initialize the transfer. All cameras had their own unique pool of registers that you had to set, but once that was done they all behaved the same spitting nice data into paraller interface. Im hoping CSI/CCP is just a serializer and the data looks the same. Expose I2C to the user, let us know what formats DSP accepts (RGB422,YUV422,RGB565,raw,jpeg), give us V4L2 /dev/video buffer with received data (maybe some option to switch between raw and encoded by the DSP) and im sure people will be able to figure out the rest.
It should be possible to use something like SN65LVDS315 and connect all the paraller modules with existing linux drivers.

That said, might be some news forthcoming on cameras in the near future...

Nice.
Anyway its not like camera is the most important feature. We can live with hardcoded support for one commercial module at start. I just hope we will eventually get proper V4L2 driver so we can hack random CSI2/CCP2 cameras.

jonititan
Posts: 30
Joined: Sat Aug 06, 2011 2:05 pm
Contact: Website

Re: Camera for R-Pi

Sun Nov 06, 2011 7:34 pm

Has there been any advance on what the camera options there will be?

Return to “General discussion”