359
Posts: 16
Joined: Thu May 29, 2014 7:55 pm

Re: HOWTO: Move the filesystem to a USB stick

Wed Mar 04, 2015 8:57 pm

Well, finally got it running. Old b pi sd card wouldn't boot on pi2. Downloaded new raspbian image, installed on the card then once that was working, changed cmdline.txt to look to sda. Worked a treat. Not sure why original files on SD wouldn't work but all happy now. Cheers

bartonlp
Posts: 2
Joined: Sun Feb 08, 2015 9:06 pm

Re: HOWTO: Move the filesystem to a USB stick

Fri Mar 13, 2015 8:49 pm

I just got a new PI 2 and a 500GB WD USB external drive. When I tried the PARTUUID method it failed. I looked and looked and then I thought "This drive is msdos not gtp". So I converted it to gtp and now it works OK.

While doing the conversion I got the message about overlapping partitions. Luckily I had a 500Mb partition at the end so I used fdisk to remove it and rewrite the partition table. Then I used gdisk and converted from msdos to gtp.

I tried all of the other suggestions on this thread but converting to gtp was the only thing that worked.

Hope this helps.

User avatar
Nastanir
Posts: 41
Joined: Tue Dec 24, 2013 3:16 pm
Location: IIT Guwahati, India
Contact: Website

Re: HOWTO: Move the filesystem to a USB stick

Sun Mar 15, 2015 8:53 am

Wonderful tutorial. The extended procedure worked well with all three pi. Thank you.
extra note:
1. I have tried to apply the gdisk based procedure on the SD card as well i.e. Formating it with gdisk to have gpt information on SD card, modify the cmdline.txt and fstab of SD card. Could not get it work.
2. Used on a USB SSD drive (Kingston 16gb SATA 1. Interface speeds up dramatically on pi B version. Problem is power consumption is very high so most power supply does not work. Sustained power consumption <1A but at some point it needs high current. Only a 2.1 A supply worked (mi power bank). With pi 2 did not work as individual usb ports could not supply enough current.

tekyboy
Posts: 6
Joined: Tue Apr 14, 2015 3:25 am

Re: HOWTO: Move the filesystem to a USB stick

Tue Apr 14, 2015 3:29 am

guys i attached 200GB HDD with RPi was working like a charm but accidentally i formatted the SD card of the captioned Pi instead of another. i installed fresh wheezy on SD card it detect USB but wont show as in df -h command.
i havent used fdisk command afraid it will erase any stuff from HDD
Now any other advise by which i can retrive my HDD stuff back

User avatar
rpdom
Posts: 14688
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: HOWTO: Move the filesystem to a USB stick

Tue Apr 14, 2015 4:11 am

tekyboy wrote:Now any other advise by which i can retrive my HDD stuff back
All you should need to do is to change the cmdline.txt on the card so "root=" points at the USB disk, the same as you did when you copied it over in the first place.

You should also check that the same kernel version is installed, otherwise you'll get module mismatches.
Check the contents of lib/modules on the card and on the USB disk. If both have the same contents - should probably be 3.18.7+ and 3.18.7-v7+, you should be good to go. If the one on the card contains something different then you might want to copy that over to the disk before rebooting.

tekyboy
Posts: 6
Joined: Tue Apr 14, 2015 3:25 am

Re: HOWTO: Move the filesystem to a USB stick

Tue Apr 14, 2015 4:45 am

thanks for the advc. and due to lack of my knowldge i have to ask following:

how to check kernel versions
how to Check the contents of lib/modules on the card and on the USB disk

EDIT:
i just changed root files and i guess i already got it THANKS

User avatar
rpdom
Posts: 14688
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: HOWTO: Move the filesystem to a USB stick

Tue Apr 14, 2015 5:06 am

tekyboy wrote:how to check kernel versions
how to Check the contents of lib/modules on the card and on the USB disk
Check kernel version and modules on card

Code: Select all

pi@raspi3 ~ $ uname -r # Current kernel version
3.18.7+
pi@raspi3 ~ $ ls /lib/modules # contents of modules on current root fs (card)
3.18.7+  3.18.7-v7+
Check modules on disk

Code: Select all

pi@raspi3 ~ $ sudo mount -t ext4 /dev/sda1 /mnt # Mount disk partition (may not be "1")
pi@raspi3 ~ $ ls /mnt/lib/modules # contents of modules on disk
3.18.7+  3.18.7-v7+
pi@raspi3 ~ $ sudo umount /dev/sda1
If there is an entry on the disk that matches the uname command, then you you are good to go, otherwise you need to copy stuff across like this

Code: Select all

pi@raspi3 ~ $ sudo mount -t ext4 /dev/sda1 /mnt # Mount disk partition (may not be "1")
pi@raspi3 ~ $ cp -a /mnt/lib/modules/$(uname -r) /lib/modules # Copy over modules
pi@raspi3 ~ $ sudo umount /dev/sda1
Replace "/dev/sda1" with the partition of your hard disk.
Everything from the "#" to the end of the line are comments and can be omitted.

pradeepsingh072
Posts: 1
Joined: Wed May 27, 2015 5:25 am

Re: HOWTO: Move the filesystem to a USB stick

Wed May 27, 2015 5:35 am

Very helpful.........Thank you.
I have to encrypt the USB stick after moving file system to it.
I am currently trying with cryptsetup, but unable to do so.

Please, help me out.
Regards,
Pradeep

alvaro.canelo
Posts: 1
Joined: Fri Jun 05, 2015 4:32 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jun 05, 2015 4:34 pm

Really, really well explained. No problems. Thank you very much, mate!

sudhir_brahma
Posts: 28
Joined: Fri Jun 19, 2015 12:08 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jun 19, 2015 12:16 pm

Thanks a lot for the details- it works quite well.
Some additional suggestions: Write a cron script that does the following: In my case it runs every minute
1. Unmount the sd and run fsck -a on the mmcblk0p1 partition.
2. Do a touch /forcefsck: Reason-On every boot, before the filesystem on the USB takes over, the fsck scans and repairs any error that may occur on the USB stick filesystem. Just ensure that yr /etc/fstab has the appropriate configuration to allow this fsck to run at boot time

paulv
Posts: 558
Joined: Tue Jan 15, 2013 12:10 pm
Location: Netherlands

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jun 19, 2015 12:31 pm

Hi sudhir,

Suggestions are good, but a little cryptic.
Do you care to elaborate so other less skilled can follow what you're suggesting?

Tks,
paulv

Joe Schmoe
Posts: 4277
Joined: Sun Jan 15, 2012 1:11 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jun 19, 2015 12:32 pm

a cron script ... runs every minute
Whenever anyone uses the terms "cron" and "every minute" in the same sentence, you know they are creating a disaster about to happen.
And some folks need to stop being fanboys and see the forest behind the trees.

(One of the best lines I've seen on this board lately)

sudhir_brahma
Posts: 28
Joined: Fri Jun 19, 2015 12:08 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jun 19, 2015 3:11 pm

Sure Paluv. My apologies for being cryptic. The details as follows:
Do the following once u have booted into the USB filesystem. You will need to create an auto login for a user say 'pi'. To do that follow the instructions: http://www.opentechguides.com/how-to/ar ... start.html

Create a shell script in say /home/pi/corruption_prevent.sh
enter the following in that shell script: This is assuming yr sd appears as mmcblk0 and mmcblk0p1 is the boot partition on the sd
sudo umount /dev/mmcblk0p1 >/dev/null 2>&1
sudo fsck -a /dev/mmcblk0p1 >/dev/null 2>&1

sudo touch /forcefsck

Save the file, make it executable for all.

U now need to do 2 things:
1. Get this script to run at boot time
2. Be doubly sure get it to run via cron as well

To get this to run at boot time do as follows:
How to automatically run a script after login.
Step 1: Open a terminal session and edit the file /etc/profile
sudo nano /etc/profile
Step 2: Add the following line to the end of the file. PS: Dont miss the dot before /home below
. /home/pi/corruption_prevent.sh
Step 3: Save and Exit
Press Ctrl+X to exit nano editor followed by Y to save the file. U can use any editor- not nano necessarily
Now u are all set to get this working at boot time. This should be fine for most cases. If however u are paranoid that these can change and u need to keep ensuring the files on the sd are ok then proceed to the next step (setting up the cron Job)
Login as a pi user in the commandline
enter:
crontab -e
Go right down at the bottom of the file and enter the following to run the script every minute
* * * * * /home/pi/corruption_prevent.sh
press cntrl+O to save and cntrl+X to exit
Test: check in a little more than a minute's time for a filewith the name forcefsck in the / directory . That is being put by the command sudo touch /forcefsck in your script /home/pi/corruption_prevent.sh
Hope this helps
PS: I have run reboots (over 1000 reboots ) and it is going strong. I stop it and check and resume the reboots
Update: 20th June 2015: I have even tried abrupt/ungraceful power off by yanking off the 5 volt supply line to the micro-usb. Did that 75 times, each time changing the plug-pull time (starting from 1 sec of powering up to when the unit starts responding to pings. Specifically on instances when the green led lights up signalling IO to the sd) : It is going strong- no issues.

Earlier, the problem reported by fsck while running on sd: It always found the dirty flag set.
Last edited by sudhir_brahma on Sat Jun 20, 2015 3:49 am, edited 2 times in total.

sudhir_brahma
Posts: 28
Joined: Fri Jun 19, 2015 12:08 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Fri Jun 19, 2015 3:13 pm

Joe Schmoe wrote:
a cron script ... runs every minute
Whenever anyone uses the terms "cron" and "every minute" in the same sentence, you know they are creating a disaster about to happen.
hello Joe- can u let us know why you think so?
Thanks in adv

paulv
Posts: 558
Joined: Tue Jan 15, 2013 12:10 pm
Location: Netherlands

Re: HOWTO: Move the filesystem to a USB stick/Drive

Sat Jun 20, 2015 12:25 pm

Thanks sudhir, your method is clear to me now.

In the meantime, Joe published an undocumented feature that allows you to do a complete file check of the p1 and p2 partitions during the boot sequence. With his permission, I created a separate post : viewtopic.php?f=29&t=113669 but I also added this link to my original how-to post.

Obviously this will only force the extended file check at boot time (before mounting), so if you are concerned that problems can occur more frequently, I suggest you to consider to regularly reboot the Pi under your (programmatic) control, and not force a file check so often through cron. Chances are that you introduce more problems by frequently mounting/unmounting than you can catch and fix. in addition, by scheduling a file check regardless, you will probably never find out what is causing a possible corruption in the first place. My advice is that checking the file system every minute is most likely a bit too much, unless you have a very specific reason.

Enjoy!

sudhir_brahma
Posts: 28
Joined: Fri Jun 19, 2015 12:08 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Jun 23, 2015 5:47 pm

I just discovered that windows natively does not allow more than 1 partition on a removable drives, like a USB/sd etc. Could that be for good reason ie that corruption is seen in such attempts. Is that the reason for our issue as well ie that /boot and /root partitions when created on the same sd card runs into these issues? I am doing an experiment : On the SD card, I have removed the /root partition and have extended the /boot partition to cover all of the 8gb sd card. So now I have a /boot partition that is 8gb in size and made into FAT32. The /root is on the USB and I am doing mutiple boots. No issues seen so far.
Questions:
1. Fundamentally is there an issue with having multiple partitions on removable drives?
2. Do u see an issue with a 8 gb sized /boot partition?
3. Do u see an issue with the /boot being converted to FAT 32 (I do not so far. fsck runs all right)
4,. Can I create directories in this now enlarged /boot partition for general usage by the root user only? Is there an issue in doing that?
Thanks
Sudhir

User avatar
DougieLawson
Posts: 35519
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Jun 23, 2015 6:10 pm

sudhir_brahma wrote: 2. Do u see an issue with a 8 gb sized /boot partition?
...
4,. Can I create directories in this now enlarged /boot partition for general usage by the root user only? Is there an issue in doing that?
Your filesystem can't have any security, to be usable every file, every directory has to be world writeable by every user on the system (or everything has to run as root). FAT, FAT16, FAT32 and vFAT do NOT support the Unix ownership and permissions system.
Note: Having anything remotely humorous in your signature is completely banned on this forum. Wear a tinfoil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
rpdom
Posts: 14688
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: HOWTO: Move the filesystem to a USB stick/Drive

Tue Jun 23, 2015 6:21 pm

sudhir_brahma wrote:Questions:
1. Fundamentally is there an issue with having multiple partitions on removable drives?
Only in windows.
2. Do u see an issue with a 8 gb sized /boot partition?
No, except I have no use for one.
3. Do u see an issue with the /boot being converted to FAT 32 (I do not so far. fsck runs all right)
No, FAT32 should be fine
4,. Can I create directories in this now enlarged /boot partition for general usage by the root user only?
Yes, although the default settings will mean that any file or directory in /boot is readable by any user, but only writeable by root. Is that the sort of "general usage" you meant?
Is there an issue in doing that?
Yes. You will be restricted by the limitations of the FAT32 filesystem as to what you can put on there. IIRC you can't have a file called "FOO.TXT" and another files called "foo.txt" as the filesystem isn't case sensitive.

sudhir_brahma
Posts: 28
Joined: Fri Jun 19, 2015 12:08 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Wed Jun 24, 2015 5:16 am

Thanks DougieLawson and rpdom.
Ran the reboot tests overnight, 674 times. No issues seen.
Mounted the /boot partition and it goes belly up. The green light turns on (no ssh so can not see what is going on). Most likely a long fsck. Turned it off (pulled the plug and put it back) and the same unit with no changes started working perfectly (did the dirty flag get turned off?).. Next I re-flashed the sd and this time increased the boot partition to only 1 GB size. Did the same tests with more vigour like copying files and pulling the power plug during copying and deleting...absolutely no issue.....the unit came bk working each time....any ideas on how I can make life worse for the unit to make it fail?
Lesson learned: Keep the boot partition small, never mount it and keep doing fsck on it on every boot or several times during running. I will appreciate views and comments on this. Just need to make this raspberry pi fail safe across power plug pulls

User avatar
yy502
Posts: 30
Joined: Sun Aug 03, 2014 2:15 am
Location: Cambridge, UK

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jun 25, 2015 9:52 am

Hi all,

Could someone who boots their Pi from USB disk confirm if I need to change root=PARTUUID=xxx (existing working solution) to root=UUID=yyy in cmdline.txt when I upgrade from 3.18 to 4.0?

Thanks.

User avatar
bleep42
Posts: 145
Joined: Wed Mar 07, 2012 12:43 pm
Location: Sussex
Contact: Website

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jun 25, 2015 12:24 pm

yy502 wrote:Hi all,

Could someone who boots their Pi from USB disk confirm if I need to change root=PARTUUID=xxx (existing working solution) to root=UUID=yyy in cmdline.txt when I upgrade from 3.18 to 4.0?

Thanks.
Mine simply says.

Code: Select all

root=/dev/sda1

User avatar
DougieLawson
Posts: 35519
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jun 25, 2015 12:42 pm

bleep42 wrote: Mine simply says.

Code: Select all

root=/dev/sda1
The flaw with that is that your HDD MUST enumerate as /dev/sda (with partitions /dev/sda1 to /dev/sdaN). What happens when it enumerates as /dev/sdb?
Note: Having anything remotely humorous in your signature is completely banned on this forum. Wear a tinfoil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
bleep42
Posts: 145
Joined: Wed Mar 07, 2012 12:43 pm
Location: Sussex
Contact: Website

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jun 25, 2015 12:54 pm

DougieLawson wrote:
bleep42 wrote: Mine simply says.

Code: Select all

root=/dev/sda1
The flaw with that is that your HDD MUST enumerate as /dev/sda (with partitions /dev/sda1 to /dev/sdaN). What happens when it enumerates as /dev/sdb?

Yes possible, but it never does! and I've been using /root on a external disk since well before this thread was started. :-)
Regards,
Kevin.

Joe Schmoe
Posts: 4277
Joined: Sun Jan 15, 2012 1:11 pm

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jun 25, 2015 1:09 pm

The flaw with that is that your HDD MUST enumerate as /dev/sda (with partitions /dev/sda1 to /dev/sdaN). What happens when it enumerates as /dev/sdb?
Yes possible, but it never does! and I've been using /root on a external disk since well before this thread was started.
Yeah, that's the question, isn't it? Like you, I've never had a problem using /dev/sdXX. I even posted a post to that effect a year or so ago, saying, pretty much "Yeah, we all know this can happen in theory, but does it ever happen in practice?". Nobody ever responded to that, so one might conclude that it does never happen in practice. Then again, one ought to assume that since they went to all this trouble to code up this UUID stuff, that it did happen, somewhere, somehow, to someone...

(Or maybe not...) Maybe it is all just a response to a theoretical problem that no one's ever actually seen.

Obviously, this can never be a problem unless you have more than one USB Mass Storage Device connected to your Pi at boot time. So that's one way to ensure it never happens. Another way, is to put the thing you want to put from connected directly to the Pi (through one of the USB ports on the board) and put the other one(s) on a USB hub. That should ensure that they enumerate more slowly than the first one. Then the first one (the one you want to be /dev/sda) will always enumerate first.

In the end analysis, each person has to decide for themselves whether the added complexity of the UUID stuff is worth the trouble...
And some folks need to stop being fanboys and see the forest behind the trees.

(One of the best lines I've seen on this board lately)

User avatar
DougieLawson
Posts: 35519
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: HOWTO: Move the filesystem to a USB stick/Drive

Thu Jun 25, 2015 3:36 pm

I've had it happen when I left a USB stick inserted and rebooted my RPi. Using UUID meant that the system came up even though the hard drive was assigned as /dev/sdb & /dev/sdb1.
Note: Having anything remotely humorous in your signature is completely banned on this forum. Wear a tinfoil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

Return to “Advanced users”