RonR
Posts: 459
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Image File Utilities

Thu Aug 01, 2019 9:12 pm

Attached are several utilities I created and use to manipulate image files.


image-backup:

image-backup creates a backup of a running Raspbian system to a standard 'raw' image file that can be written to an SD card or a USB device card with Etcher, imageUSB, etc. It will also perform incremental updates to an existing backup image file.

Running image-backup with no parameters will create a full backup. To create the smallest possible image, specify an Image ROOT filesystem size of 0 to determine the minimum allowed size. If you plan to incrementally update the image file, specify a considerably larger size to allow for additional growth.

Running image-backup with a parameter of an existing image filename will incrementally update that image file.


Image-check:

image-check will check the integrity of a standard 'raw' image file. Usage is:

image-check imagefile [W95|Linux]

where W95 checks the BOOT partition and Linux checks the ROOT partition. If neither is specified, Linux is assumed.


image-compare:

image-compare compares a running Raspbian system to an existing standard 'raw' image file and displays the incremental changes that image-backup would perform if run. Usage is:

image-compare [imagefile]


image-mount:

image-mount mounts a standard 'raw' image file to allow it to be read or written as if it were a device. Usage is:

image-mount imagefile mountpoint [W95|Linux]

where W95 mounts the BOOT partition and Linux mounts the ROOT partition. If neither is specified, Linux is assumed.


image-set-ptuuid:

image-set-ptuuid sets the Partition Table UUID value of a standard 'raw' image file. Usage is:

image-set-ptuuid imagefile ptuuid

where ptuuid is 8 hex digits


image-shrink:

image-shrink shrinks a standard 'raw' image file to its smallest possible size (plus an optional additional amount of free space). Usage is:

image-shrink imagefile [Additional MB]

where Additional MB is an additional amount of free space to be added.
Attachments
image-utils.zip
(8.67 KiB) Downloaded 30 times
Last edited by RonR on Sat Aug 31, 2019 4:02 am, edited 1 time in total.

User avatar
rin67630
Posts: 443
Joined: Fri Mar 04, 2016 10:15 am

Re: Image File Utilities

Tue Aug 13, 2019 4:40 pm

Super! very useful applets!

Could you please tell me where to put the unzipped files? Just in /home/pi)?
Are they executable directly or must we first chmod them?

A detailed noob-proof description of what to do would really be useful.

Thank you!

RonR
Posts: 459
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Wed Aug 14, 2019 12:56 am

rin67630 wrote:
Tue Aug 13, 2019 4:40 pm
Could you please tell me where to put the unzipped files? Just in /home/pi)?
Are they executable directly or must we first chmod them?

They're not picky about where you place them (/home/pi would be fine). You must make them executable (chmod +x ...).
rin67630 wrote:
Tue Aug 13, 2019 4:40 pm
A detailed noob-proof description of what to do would really be useful.

I'll try to write up a few examples shortly.

itsmedoofer
Posts: 356
Joined: Wed Sep 25, 2013 8:43 am

Re: Image File Utilities

Wed Aug 14, 2019 11:19 am

Again, fantastic work, really appreciated !

User avatar
scruss
Posts: 2418
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Image File Utilities

Sat Aug 24, 2019 4:57 pm

RonR wrote:
Wed Aug 14, 2019 12:56 am
They're not picky about where you place them (/home/pi would be fine). You must make them executable (chmod +x ...).
They could also go in ~/bin (that's /home/pi/bin). Raspbian automatically adds ~/bin to your path at login if the folder exists.

Code: Select all

mkdir ~/bin
is all people need to create it. The system will add it to your path the next time you login or reboot.

RonR — thanks very much for writing these. I see that some older links have expired and people can't seem to find them, so I created a github repo of your code as of 2019-08-01: RonR-RaspberryPi-image-utils. I hope that's okay with you.

I've also stored a copy of the attachment on the Internet Archive so it should stick around better.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

bls
Posts: 205
Joined: Mon Oct 22, 2018 11:25 pm

Re: Image File Utilities

Sat Aug 24, 2019 6:40 pm

image-mount is particularly sweet! Takes away all the hassle of mounting an SD card image. Thanks!

RonR
Posts: 459
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sat Aug 24, 2019 6:58 pm

bls wrote:
Sat Aug 24, 2019 6:40 pm
image-mount is particularly sweet! Takes away all the hassle of mounting an SD card image. Thanks!

Thanks for the feedback.

I use image-mount every time a new Raspbian image is released to inject an ssh and/or wpa_supplicant.conf file into the image file. Then I don't have to manually add them to the SD card or USB device each time I write one. It's also handy for extracting files from an image file created by image-backup.

bls
Posts: 205
Joined: Mon Oct 22, 2018 11:25 pm

Re: Image File Utilities

Sat Aug 24, 2019 7:13 pm

RonR wrote:
Sat Aug 24, 2019 6:58 pm
bls wrote:
Sat Aug 24, 2019 6:40 pm
image-mount is particularly sweet! Takes away all the hassle of mounting an SD card image. Thanks!

Thanks for the feedback.

I use image-mount every time a new Raspbian image is released to inject an ssh and/or wpa_supplicant.conf file into the image file. Then I don't have to manually add them to the SD card or USB device each time I write one. It's also handy for extracting files from an image file created by image-backup.
That's great! I never thought of that approach. Mine is slightly different. I have a script that adds all my stuff (scripts, config files, etc) to a newly-built SD card. I then boot it and install the packages I want. At that point I dd that beautiful SD card to an IMG file and use it for building new 'virgin' SD cards, as needed. image-mount lets me make certain classes of updates to my SD "golden master" without having to go through a complete rebuild.

User avatar
Botspot
Posts: 149
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas
Contact: Yahoo Messenger

Re: Image File Utilities

Fri Aug 30, 2019 2:50 pm

These are brilliant! I have used portions of image-mount in my bash script to let it mount and boot a Raspbian disk image using systemd-nspawn.

Bug report:
image-mount does not accept disk images located in a directory that contains the space character. (And yes, I tried using quotation marks.)

Code: Select all

pi@raspberrypi:~ $ sudo /home/pi/image-utils/image-mount "/home/pi/Web drives/DMA Google Drive/image files/2019-06-20-raspbian-buster-full.img" /media/pi/fs Linux

Mount /home/pi/Web drives/DMA Google Drive/image files/2019-06-20-raspbian-buster-full.img on /media/pi/fs (y/n)? y

/home/pi/image-utils/image-mount: line 66: drives/DMA * 512: division by 0 (error token is "DMA * 512")

Unable to create loop device
My doctor told me my brain is as useful as a Raspberry Pi. Is that a compliment?

User avatar
scruss
Posts: 2418
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Image File Utilities

Fri Aug 30, 2019 10:22 pm

Yeah, it doesn't quite get the quoting right around starting round about line 60.

Of course, no-one I know would admit the creating or owning any file system with spaces in the name …
(joke, in case it was too obtuse.)
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

RonR
Posts: 459
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sat Aug 31, 2019 4:13 am

The problem was totally unrelated to quoting. The problem was due to using fdisk to display disk parameters and awk to parse them. The addition of spaces in path/file names caused awk to grab the wrong fields. The solution was to switch to sfdisk and sed.

An updated image-utils.zip file is now available in the first post of this topic which works properly with spaces in path/file names.

Thanks Botspot for reporting this.

Return to “Advanced users”