User avatar
100xyz
Posts: 19
Joined: Fri Aug 17, 2018 10:19 am
Location: US citizen, born in India living in Beijing
Contact: Website

Corrpution on system backup

Fri Aug 31, 2018 5:31 am

Hi,
Not sure this is the right section for this topic...

I have a Pi with Raspbian. It runs a web server using node.js and express. I want to make another copy of the SD card, so that I can run the same thing on the second pi.

To do this, I shutdown the server with
sudo service webServer stop

Then I stop the pi with
sudo shutdown --halt now

Then I pull out the SD card, stick it into my pc and do a dd.

So far, so good.

Then when I create a replica on another SD card (also using dd), the main webServer.js file is corrupt. It has some system log messages written at the very top of the file. What's happening?

Any suggestions, tips appreciated.
Cofounder 100-xyz.com
Platform for local information via WiFi

User avatar
RaTTuS
Posts: 10491
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: Corrpution on system backup

Fri Aug 31, 2018 7:17 am

what dd commands did you use ?
e.g.
sudo dd if=/dev/sda of=/dev/sdb conv=sync,noerror bs=4k
is your new sdcard a fake ?

or then you can
use the rpi-clone tool?
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

User avatar
100xyz
Posts: 19
Joined: Fri Aug 17, 2018 10:19 am
Location: US citizen, born in India living in Beijing
Contact: Website

Re: Corrpution on system backup

Fri Aug 31, 2018 9:11 am

sudo dd of=2018-06-27-100xyz.img if=/dev/sdb bs=4M status=progress
sudo sync /dev/sdb


sudo dd if=2018-06-27-100xyz.img of=/dev/sdb bs=4M status=progress
sudo sync /dev/sdb

Above are the commands I use for the dd. I have tried several times on several SD cards.
Cofounder 100-xyz.com
Platform for local information via WiFi

User avatar
B.Goode
Posts: 8864
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: Corrpution on system backup

Fri Aug 31, 2018 9:55 am

I don't know for certain that it will fix your problem, but if your implementation of dd supports it perhaps using the conv=fsync option on the dd command itself might help?

(Somewhere back in the forum archives there is a thread about dd on a Mac not doing the right thing by default and creating unusable images. Maybe this is related, even if you don't have a Mac.)

Alternatively, use the SD Card Copier utility provided by the Raspberry Pi Foundation.

Or maybe you are getting a faithful backup of something that is already broken? If you do the shutdown steps you describe and then reboot from the 'master' microSD card does the server recover cleanly?

epoch1970
Posts: 3794
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Corrpution on system backup

Fri Aug 31, 2018 10:15 am

dd on the Mac is fine.
It is risky to clone an expanded SD to another one, even of the same brand/batch, they may have slight differences in capacity.
Shrink the original, clone it, expand the clone. Should work.
(Variant, you can shrink the expanded original image if you loop-mount it)
Gparted could help.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

User avatar
100xyz
Posts: 19
Joined: Fri Aug 17, 2018 10:19 am
Location: US citizen, born in India living in Beijing
Contact: Website

Re: Corrpution on system backup

Sat Sep 01, 2018 12:03 pm

@B. Goode
The master image is fine. It boots up and the files are clean. Problem is with the copies.

The disk sizes are different. Master is 4gb. Copies I have tried are 8, 16 and 32gb all resulting in corruption.

Now trying the other suggestions.

Thanks for the input.
Cofounder 100-xyz.com
Platform for local information via WiFi

User avatar
100xyz
Posts: 19
Joined: Fri Aug 17, 2018 10:19 am
Location: US citizen, born in India living in Beijing
Contact: Website

Re: Corrpution on system backup

Mon Sep 03, 2018 3:14 am

Looks like the problem is now resolved. I unmounted the two partitions before making the backup and before making the copy.

Here's the explanation as provided by the tech guru:

As soon as you insert the card, it would have mounted the partitions automatically. And lets say you dd and take a copy (this in itself will create some writes on the file system metadata area). Now you take the dd. This is OK, but ideally you should take a dd with none of its partitions being mounted.
Now lets say you insert the new SD card. Lets say it is not really blank and has some FS in it already. It shall be auto mounted. Now, you dd the image back to this disk. But, it has mount points which the OS will update when you sync it. Hence, you end up with corruption.
Hence, if you make sure that you unmount the partitions that are already mounted before taking a copy or while making a copy, it will avoid corruption.
Cofounder 100-xyz.com
Platform for local information via WiFi

Return to “Raspbian”