Page 13 of 14

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Thu May 25, 2017 1:13 pm
by dvaitman
paulv wrote: If you look at the very first post, there is a note about booting directly from a disk for the RPi Model 3.
I already boot from a USB using raspbian. After enabling booting from a usb on the pi (steps on the rpi site), just burn the image with etcher to the usb, plug the usb to the pi, and boot.
It works fine, no issues. The last raspbian version can be burnt into a usb straight away. The previous version had many steps as per the instructions on the first post.
My question is not for raspbian, but for the google AIY image from this site:
http://www.androidauthority.com/build-g ... pi-770296/
image file: https://dl.google.com/dl/aiyprojects/vo ... est.img.xz

i want to write this image to a USB and boot from there. But changing the cmdline.txt (root=PARTUUID=....) and fstab (replace /dev/mmcblk0p1 & dev/mmcblk0p2 with PARTUUID=.......) doesn't work

I do that and when I boot I get these errors:

Code: Select all

cat: /sys/block/*/PARTUUID=432b3309-02/partition: No such file or directory
expr: syntax error
Error: The device /dev/autofs is so small that it cannot possibly store a filesystem....
Warning: Error fsyncing/closing /dev/autofs: Invalid argument
so, not sure how to make this work :?
help/direction/advise is appreciated

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Thu May 25, 2017 10:00 pm
by arglebargle
.

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Fri May 26, 2017 6:08 pm
by RickyBaker
@paulv thanks so much for putting this together!

A couple questions about backups:

Would it make sense to have a backup script that essentially clones the USB rootfs filesystem back to the SD card every now and then? So you have essentially a running backup of your system without causing too many writes to the SD card?

If I wanted to backup the USB then, would I image it wiht the same method I usually use when the image is on an SD card? or is it simpler than that because I'm not booting off of it?

And finally: I feel like the mere existence of this thread should answer my question but I still feel like the reasons I can deign don't do it for me: Is the USB flash drive really that much more stable than an SD Card? Seems like they are the same underlying technology with a different connection, but I'm far from an expert...

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Sat May 27, 2017 4:27 am
by gkreidl
RickyBaker wrote:...
And finally: I feel like the mere existence of this thread should answer my question but I still feel like the reasons I can deign don't do it for me: Is the USB flash drive really that much more stable than an SD Card? Seems like they are the same underlying technology with a different connection, but I'm far from an expert...
You are right, but USB hard disks are faster, more reliably and provide much more capacity/$.

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Tue May 30, 2017 7:09 pm
by RickyBaker
@gkreidl Thanks! I get that, I guess I was curious why that's the case? Is it the connector?

Also, what's the easiest way to make sure the files are successfully moved to the USB drive? Everything seemed to go off without an error, and when I rebooted it came back up, but I'm just not sure its using the USB drive and not the sd card...

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Sun Jun 04, 2017 7:08 am
by GuiPoM
Hello :)

That the third or four time I am using this thread to move my RPI2 filesystem, and it is still so useful, thanks for that !

This time there is a last step I did not manage to do: usually what I do is that I use a good SD card (with 32GB, fast enough ...) and I install everything I need and make some tests. Then, I start applying the indications on this thread to move system to USB. And finally, I switch SD card to a very small one (1GB or less) because as it is only used to boot, there is no need to have an high end one.

This time I do not manage to replace SD card with another, I tried:
- as usual, one FAT32 partition created on windows, named boot, and copy paste everything
- install raspbian lite on the SD, the change command and boot.config with the one from the other card
- dd the partition and extract it on the SD using Win32diskimager.

RPI2 does not boot, whatever the solution is. And I don't know how to debug that. What would be the correct procedure ? Thanks !

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Sat Jun 24, 2017 5:18 am
by zipplet
If it is any help, I wrote a tool to automate the task of moving your root partition from an SD card to a USB device and making the necessary changes to the boot configuration files. You still need the SD card present to load the boot files (firmware and kernel), but it will run the OS from the USB device afterwards. It works on all Pi models.

The only caveat is you must run a tool (included) every time you apt-get dist-upgrade, update /boot (which is a virtual copy sitting on the USB device) or update the kernel/firmware to move those changes to the SD card. The tool is supplied, and the scripts inform you of this during installation.

One good thing about this method of operation is the tool installs (yet another) tool that runs every boot, and copies any changes you have made to /boot on the SD card (using your computer) to the "virtual" /boot on the USB device, so you can fix a broken overclock/etc - the settings on the SD card take priority.

At all other times the SD card remains unmounted to avoid data corruption during power loss/etc.

If installation goes wrong, reverting is easy - just pull out the SD card and rename a couple of files. For those interested, it can be found in my pitools repository under "sd_to_usb_boot" here:

https://github.com/zipplet/pitools

I repeat, this tool STILL requires the SD card after. This will not magically make a Pi 3 bootable USB device that works without the SD card. Also, you should follow the caveats given when you run the tool - run it in console mode only with as many services stopped as possible to ensure a clean filesystem transfer. Ideally, you'd do this immediately after installing the OS of your choice on the SD card.

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Sat Jun 24, 2017 4:14 pm
by RickyBaker
@zipplet Thanks! that's pretty awesome! I'll try it out next time I want to do this

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Sat Jun 24, 2017 5:07 pm
by rpdom
zipplet wrote:The only caveat is you must run a tool (included) every time you apt-get dist-upgrade, update /boot (which is a virtual copy sitting on the USB device) or update the kernel/firmware to move those changes to the SD card. The tool is supplied, and the scripts inform you of this during installation.
You could add some apt-get hooks that do the updating of /boot on the SD card after an upgrade or dist-upgrade.

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Sun Jun 25, 2017 2:34 am
by zipplet
rpdom wrote:You could add some apt-get hooks that do the updating of /boot on the SD card after an upgrade or dist-upgrade.
Good idea - I'll revisit the tool at some point and add support for that - then it becomes completely transparent to the end-user.

Re: HOWTO: Move the filesystem to a USB stick/Drive [updated

Posted: Fri Jul 07, 2017 9:51 pm
by Protoroto
I was making a new thread but I figured out I could ask here, let's hope it's the right place...

My situation:
Arch installed on a 16gb sd card, / is f2fs . I am scared that someday it will get corrupted and I will lose everything on it.

What I want:
Move everything to an USB hd (and boot totally from USB).

So, as far as I understand, I need to:
- add program_usb_boot_mode=1 to /boot/config.txt
- power down the raspberry pi3, take out the sd card, put it on laptop as well as the USB drive
- create a /boot partition and a / partition on the USB drive (Iet's say I will use ext4 for / this time).
- copy /boot partition from the sd to the USB drive
- copy / partition from the sd to the USB drive
- edit /boot/cmdline.txt on USB drive and change root=/dev/mmcblk0p2 to whatever the USB is when mounted, and edit rootfstype=f2fs to ext4
- edit /etc/fstab entries to mount USB drive partitions instead of sd ones.

Is there anything else I should know? Some other suggestions? If something goes wrong, if I plug back the sd will it boot regardless the changes to /boot/config.txt?

Thanks in advance for any help, and sorry for the english, is not my primary language.

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Wed Aug 02, 2017 8:35 pm
by donbrew
Mounting the new drive during boot

The next step is to actually mount the rootfs system during the boot process. This is handled by the file system during the later stages of the boot process itself. It uses the information located in the file /etc/fstab.
NOTE We are only going to change the data on the new drive, so we keep the SD card unchanged. This will make it easier to go back and forth with only changing the /boot/cmdline.txt file on the SD card.

Code: Select all

sudo nano /mnt/etc/fstab


The location of fstab should be /etc/fstab

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Wed Aug 02, 2017 8:47 pm
by DirkS
donbrew wrote:
Wed Aug 02, 2017 8:35 pm
Mounting the new drive during boot

The next step is to actually mount the rootfs system during the boot process. This is handled by the file system during the later stages of the boot process itself. It uses the information located in the file /etc/fstab.
NOTE We are only going to change the data on the new drive, so we keep the SD card unchanged. This will make it easier to go back and forth with only changing the /boot/cmdline.txt file on the SD card.

Code: Select all

sudo nano /mnt/etc/fstab


The location of fstab should be /etc/fstab
No. Please read the instructions carefully.
This is to change fstab on the new rootfs partition that's mounted on /mnt

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Sun Aug 20, 2017 1:39 pm
by bmendon
I just tested your process on an Raspberry Pi 3 using Raspbian Stretch Lite and it ran flawlessly. A couple of notes...

1. I had problems until I performed a low-level format of the drive I used. You might suggest that as a step, especially for users like myself who are using drives that have been repartitioned a number of times. I used the tool (free edition) from here to do the format: http://hddguru.com/

NOTE: it took about 6 hours to low-level format the drive.

2. Stretch uses the UUID format for the partitions. I just went with sda1 and sda2.

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Fri Aug 25, 2017 12:59 pm
by S0litaire
Currently running a Pir0w in a WD node.

The file system is currently on the WD hard drive.

How do reverse this procedure so that the currant install runs form the SD card (without loosing data if possible)

Basically it's running an archived php3 based forum with a 1.6Gb SQL database. it's a total pain to restore that size of a SQL database on the r-pi's. It's possible but rather not have the hassle if it can be avoided

Is it just a case of moving the filesystem off of the WD drive and changing the fstab mount?

Or am i stuck starting from scratch?

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Thu Nov 09, 2017 7:17 am
by StNick
Amazing thread!

Thanks TS for this post.
Even now this theme is up to date.

I'm struggling with direct SSD boot on RPI3 for last couple of months,
the issue is that RPi3 has some restrictions on supported hardware,
I've already bought tons or USB-SATA adapters and none of them is working on RPi3.
My SSD is Kingston HyperX Fury 120 GB.

Here is detailed post, who is interested please read ALL comments! People have large variety of issues

https://www.raspberrypi.org/blog/pi-3-b ... rage-boot/

So as my second chance - I'm going to try just move ONLY root filesystem to SSD
and stay with SDCARD as primary boot device(probably even USB stick will work instead of SDCARD).

Actually USB sticks works pretty well as single USB boot device, but SSD (via adaptors) is not!

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Tue Nov 28, 2017 4:18 am
by topgear
Followed the steps and successfully moved the file system to an external hdd. Created only one partition which /dev/sda1.

I can boot from it and execute commands.

But can't write anything to the drive. Getting " Read-only file system " error.

can't install anything or create any file on the usb hdd because of this but I can boot from it using = RPi2 + MicrosdCard + USB HDD - Super Top M6116 SATA Bridge + Raspbian Stretch

Out Put of lsusb

Code: Select all

pi@raspberrypi:~ $ lsusb
Bus 001 Device 004: ID 14cd:6116 Super Top M6116 SATA Bridge
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp. SMC9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Out put of df -h

Code: Select all

pi@raspberrypi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        73G  4.6G   65G   7% /
devtmpfs        460M     0  460M   0% /dev
tmpfs           464M     0  464M   0% /dev/shm
tmpfs           464M   12M  452M   3% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           464M     0  464M   0% /sys/fs/cgroup
/dev/mmcblk0p1   42M   21M   21M  51% /boot
tmpfs            93M     0   93M   0% /run/user/1000
out put of fdisk -l

Code: Select all

pi@raspberrypi:~ $ sudo fdisk -l
Disk /dev/ram0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram1: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram2: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram3: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram4: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram5: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram6: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram7: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram8: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram9: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram10: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram11: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram12: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram13: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram14: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/ram15: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mmcblk0: 7.4 GiB, 7948206080 bytes, 15523840 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xf0f659f5

Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk0p1       8192    93814    85623 41.8M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      94208 15523839 15429632  7.4G 83 Linux


Disk /dev/sda: 74.5 GiB, 80025278976 bytes, 156299373 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6160303D-BBC0-4BFF-A236-FDCA2405B3B1

Device     Start       End   Sectors Size Type
/dev/sda1   2048 155191295 155189248  74G Linux filesystem
Anyway to fix this ?

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Mon Dec 04, 2017 8:07 pm
by JimBobHarris
Thanks for this excellent help page. So many pitfalls, so much need for experts.

Even so:

sudo nano /mnt/etc/fstab

should be, I think:

sudo nano /etc/fstab

Sorry if this has been pointed out before; too many pages to scan to prevent duplicating it.

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Mon Jan 08, 2018 3:37 pm
by Maverix
Hi,

I would like to boot my RPI3 from a SSD drive via USB because of the better lifespan of a harddisk compared to a SD card.

In the moment I'm not sure how to migrate my FHEM software from SD card to a completly new SSD drive.

Is it enough to just flash the image (created from the SD card) to the SSD drive?

Thanks in advance.

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Mon Jan 08, 2018 5:34 pm
by gkreidl
Maverix wrote:
Mon Jan 08, 2018 3:37 pm
Hi,

I would like to boot my RPI3 from a SSD drive via USB because of the better lifespan of a harddisk compared to a SD card.

In the moment I'm not sure how to migrate my FHEM software from SD card to a completly new SSD drive.

Is it enough to just flash the image (created from the SD card) to the SSD drive?

Thanks in advance.
Read the original (first) post.

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Tue Jan 23, 2018 1:51 am
by nalbee
Whats the smallest size sd card that can be used with this configuration? Any recommendations?

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Sat Apr 07, 2018 5:14 pm
by paulv
Hi Nalbee,

The smallest size SD card for this application is the one that your selected OS version will fit on. There is no definite number because the size changes with every OS revision.

However, keep in mind that if you are going to use the card for "normal" use (running the OS from it), the larger the card is the better, because of the mostly internal cell management of the card itself. Flash cells can only be used (written to and erased) a finite number, and so the SD card manages this. The more free space it has to work with, to more reliable the long term usage will be. Google for it to learn more.

My recommendation is to get the largest SD card, from the best vendors, that you can afford. There are way too many recommendations to quote, look at the forum or Google for the recommended ones.

Enjoy!

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Thu May 10, 2018 2:22 pm
by LloydPi3
Following the sticky I was able to move the stretch OS to a usb WD hard drive (HDD driven by a powered usb hub). Stretch appears to go with partuuid instead of sda. Note that there is a UUID and a PARTUUID which are different. Use the PARTUUID when setting things up.

paulv what is the reasoning on making two or more partitions with the rootfs partition small and my_data large? Isn't rootfs where all the programs one will add are going to be put in? It seems to me it would make sense to have it relatively large?

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Thu May 10, 2018 6:55 pm
by paulv
Putting your data files in one place makes a backup all the more easy, and you can move the data easier between different target computers.
If you don't need/want that, keeping the partition as large as possible helps the card with wear leveling.

Enjoy!

Re: STICKY: HOWTO: Move the filesystem to a USB stick/Drive

Posted: Sat May 12, 2018 4:57 am
by bluewhale9
I was following this step by step, but something is different on my end.

Right after the 2 partitionings with gdisk n, w, and v.... I'm running into issues with formatting.

I'm running an RPi 3B+ on latest version of Raspian Stretch, and using a brand new USB 3 Flash Drive

Code: Select all

pi@raspberrypi:~ $ sudo mke2fs -t ext4 -L rootfs /dev/sda1
mke2fs 1.43.4 (31-Jan-2017)
/dev/sda1 contains a hfsplus file system labelled 'NO NAME'
Proceed anyway? (y,N) y
/dev/sda1 is mounted; will not make a filesystem here!
pi@raspberrypi:~ $ umount /dev/sda1
pi@raspberrypi:~ $ sudo mke2fs -t ext4 -L rootfs /dev/sda1
mke2fs 1.43.4 (31-Jan-2017)
/dev/sda1 contains a hfsplus file system labelled 'NO NAME'
Proceed anyway? (y,N) y
Creating filesystem with 7507808 4k blocks and 1880480 inodes
Filesystem UUID: 6bf1534c-7665-4c6b-840a-618410f29390
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done   

pi@raspberrypi:~ $ sudo mke2fs -t ext4 -L my_data /dev/sda2
mke2fs 1.43.4 (31-Jan-2017)
The file /dev/sda2 does not exist and no size was specified.
pi@raspberrypi:~ $ sudo mount /dev/sda1 /mnt
pi@raspberrypi:~ $ sudo mount /dev/sda2 /mnt
mount: special device /dev/sda2 does not exist