bmw318
Posts: 13
Joined: Tue Aug 16, 2016 8:23 am

Extracting RAW image from JPG

Thu Aug 18, 2016 2:12 pm

Hi,
I used raspistill -r to generate an RAW image in the metadata of the JPG image.
Now I want the extract this RAW image....how can I do this quickly?
Is there an exe file to do this?
Using ExifPro gives me only the metadata excl. the RAW image.

Thanks.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6331
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Extracting RAW image from JPG

Thu Aug 18, 2016 3:22 pm

What are you trying to process it with? The Bayer raw10 format is not fun to play with - waveform80 provides a good explanation in the PiCamera docs, or the version of dcraw at https://github.com/6by9/RPiTest/tree/master/dcraw now has support for processing raws off both revisions of Pi Camera module.

If you want the lowest level format, then

Code: Select all

tail -c 10237440 8MPix.jpg > 8MPix.raw
tail -c 6371328 5MPix.jpg > 5MPix.raw
should just give you the packed raw data.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

bmw318
Posts: 13
Joined: Tue Aug 16, 2016 8:23 am

Re: Extracting RAW image from JPG

Thu Aug 18, 2016 4:47 pm

Hi,
thank you very much. I'm a beginner with RPi so...have a lot of questions.
I need the RAW image of the image sensor for academic research.
I get a RAW image but when I use the size (2592x1944) than I see that the image is misformed. So clearly the wrong format.
How do you explain the 6371328 number in the tail command of te 5Mpix camera? Is this the total amount of pixels?

Is there a description of the metadata format that includes the RAW image? Where is the RAW image located in the metadata?
Sorry for all the questions....

Best regards.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6331
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Extracting RAW image from JPG

Thu Aug 18, 2016 5:11 pm

I was suspecting that you weren't fully understanding what that raw image contained. I suspect you really want the YUV or RGB representation of the image, not raw Bayer, in which case look at raspiyuv and raspividyuv from the userland repo. Both can now capture RGB as well as YUV.

If you really want the true raw, then please read up on Bayer colour filters first. viewtopic.php?f=43&t=44918 also covers a lot of stuff about the data format, as does the PiCamera doc that I linked to last time.
10bits/pixel
(2592 * 10 / 8) rounded up to the nearest multiple of 32 = 3264
1944 rounded up to the nearest multiple of 16 = 1952
3264 * 1952 = 6371328 bytes
(That only applies to the V1.3 sensor, not the 8MPix V2.1 sensor. The same maths applies on V2.1, but with 3282x2466 as the base numbers -> 4128*2480)

There is a header block of 32kB immediately before the image data at the end of the JPEG - that is what dcraw is processing. That also gives you the Bayer order, which you need in order to process the data correctly.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

bmw318
Posts: 13
Joined: Tue Aug 16, 2016 8:23 am

Re: Extracting RAW image from JPG

Fri Aug 19, 2016 1:26 pm

Hi,
I really need the true RAW image, the image from the pixels itself, no processing at all.
There is a lot of confusing going on regarding RAW images.

Thank you.

Bryan See
Posts: 11
Joined: Mon Apr 10, 2017 2:33 pm

Re: Extracting RAW image from JPG

Fri Apr 14, 2017 7:14 pm

Me too, from the IMX219 image sensor.

User avatar
schoolpost
Posts: 14
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: Extracting RAW image from JPG

Fri Apr 14, 2017 9:07 pm

Bryan See wrote:Me too, from the IMX219 image sensor.
Bryan double posting, wont answer your question any quicker.

The confusion in this situation is really starting to propagate onto me.

RAW = unprocessed 10-bit bayer pixel data ( in a BGGR arrangement )

BG
GR

You want RAW from your IMX219 camera? Here:

Code: Select all

raspistill -r -o out.jpg
Tacked onto the end of the .jpg is your RAW data.

From there you choose how you want to extract / process that data.

Here some options:
1. Picamera Bayer Example:
http://picamera.readthedocs.io/en/lates ... a-captures
2. Raspiraw:
https://github.com/illes/raspiraw
3. DCRAW:
https://www.cybercom.net/~dcoffin/dcraw/
4. Python DNG Converter Script:
https://github.com/schoolpost/pydng

There are no built-in / standard tools designed by RPF for capturing straight to a DNG or other RAW format like most cameras.

What else am I missing here? please clarify your intentions if this hasn't sufficed.

XFer012
Posts: 10
Joined: Tue Mar 15, 2016 3:17 pm

Re: Extracting RAW image from JPG

Wed Apr 19, 2017 9:35 am

schoolpost wrote: 3. DCRAW:
https://www.cybercom.net/~dcoffin/dcraw/
Straight DCRaw from Dave Coffin won't work on the "strange .jpg" which is the output of raspistill -r

He's going to need the custom version patched by 6by9:

https://github.com/6by9/RPiTest

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6331
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Extracting RAW image from JPG

Wed Apr 19, 2017 9:51 am

XFer012 wrote:
schoolpost wrote: 3. DCRAW:
https://www.cybercom.net/~dcoffin/dcraw/
Straight DCRaw from Dave Coffin won't work on the "strange .jpg" which is the output of raspistill -r
It will, but only on slightly older raws from the OV5647 (V1 camera module). It doesn't support IMX219 (V2 camera) nor flips changing the Bayer order.
XFer012 wrote:He's going to need the custom version patched by 6by9:
https://github.com/6by9/RPiTest
I have tried a couple of times in contacting Dave Coffin about updating the main version, but had no response.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

XFer012
Posts: 10
Joined: Tue Mar 15, 2016 3:17 pm

Re: Extracting RAW image from JPG

Wed Apr 19, 2017 11:10 am

6by9 wrote:
Straight DCRaw from Dave Coffin won't work on the "strange .jpg" which is the output of raspistill -r
It will, but only on slightly older raws from the OV5647 (V1 camera module). It doesn't support IMX219 (V2 camera) nor flips changing the Bayer order.
The OP has, in fact, a V2 camera. That's why I replied.

thatneat
Posts: 4
Joined: Tue Dec 04, 2018 5:52 pm

Re: Extracting RAW image from JPG

Tue Dec 04, 2018 10:14 pm

Just to add another tool to the mix - my team just released a Python library which lets you load up the RAW data from a raspistill RAW+jpeg file into a Python numpy array for easy analysis. We have been using a fork of dcraw but we wanted to skip the .dng file step and pull the data straight into Python.

Check out https://github.com/OsmoSystems/picamraw/ and drop us a line if you have any questions or comments!

Return to “Camera board”