RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Sat Oct 12, 2019 3:05 am

Brad Q wrote:
Fri Oct 11, 2019 8:48 pm
You seeing any performance degradation after today's update? It included a bootloader update. It could just me. But it feels a little off? It would not necessarily be due to usb-boot.

I don't believe I'm seeing anything different or unusual here. I'll be doing some FreePBX builds this weekend and will let you know if anything out of the ordinary surfaces.

HypnoToad
Posts: 77
Joined: Thu Dec 11, 2014 6:00 pm

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 14, 2019 1:05 am

Hi, Does this script work OK with the Raspberry Pi 4?

I'm trying to boot from an SSD connected to the USB3 port, I installed the latest Raspbian Lite - 2019-09-26-raspbian-buster-lite.zip to an SD card, and ran the script which copied the files to the SSD.

After the first reboot I got the following screen:

Image

Have I done somthing wrong here?

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 14, 2019 1:13 am

HypnoToad wrote:
Mon Oct 14, 2019 1:05 am
Does this script work OK with the Raspberry Pi 4?
...
Have I done somthing wrong here?

Yes, it works fine with the Raspberry Pi 4

It appears you removed the SD card:

[ TIME ] Timed out waiting for device /dev/mmcblk0p1

HypnoToad
Posts: 77
Joined: Thu Dec 11, 2014 6:00 pm

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 14, 2019 2:23 am

RonR wrote:
Mon Oct 14, 2019 1:13 am
HypnoToad wrote:
Mon Oct 14, 2019 1:05 am
Does this script work OK with the Raspberry Pi 4?
...
Have I done somthing wrong here?

Yes, it works fine with the Raspberry Pi 4

It appears you removed the SD card:

[ TIME ] Timed out waiting for device /dev/mmcblk0p1
That's great that it's OK with the Pi 4 :)

I did leave the SD card in while it was rebooting, and left it in - I did not remove it at all.

I'll try another SD card and see if it does the same thing.

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 14, 2019 2:55 am

HypnoToad wrote:
Mon Oct 14, 2019 1:05 am
I installed the latest Raspbian Lite - 2019-09-26-raspbian-buster-lite.zip to an SD card, and ran the script which copied the files to the SSD.

I downloaded usb-boot (to make sure I'm using the same one you used) and did the same thing (except I used a USB 3 flash drive as I don't have a spare SSD at the moment). After the copy, I rebooted and it came right up on the USB flash drive:

Code: Select all

pi@raspberrypi:~ $ mount
/dev/sda2 on / type ext4 (rw,noatime)

There are compatibility problems with the Raspberry Pi 4 USB 3 ports and some USB-SATA adapters, so you might want to try your SSD on a USB 2 port or try a USB flash drive if you still have problems.

HypnoToad
Posts: 77
Joined: Thu Dec 11, 2014 6:00 pm

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 14, 2019 3:19 am

RonR wrote:
Mon Oct 14, 2019 2:55 am
HypnoToad wrote:
Mon Oct 14, 2019 1:05 am
I installed the latest Raspbian Lite - 2019-09-26-raspbian-buster-lite.zip to an SD card, and ran the script which copied the files to the SSD.

I downloaded usb-boot (to make sure I'm using the same one you used) and did the same thing (except I used a USB 3 flash drive as I don't have a spare SSD at the moment). After the copy, I rebooted and it came right up on the USB flash drive:

Code: Select all

pi@raspberrypi:~ $ mount
/dev/sda2 on / type ext4 (rw,noatime)

There are compatibility problems with the Raspberry Pi 4 USB 3 ports and some USB-SATA adapters, so you might want to try your SSD on a USB 2 port or try a USB flash drive if you still have problems.
Ok, I tried it on a USB 2 port and it worked straight away, it looks like my USB-to-SATA adaptor is the issue.

After some searching using the 'lsusb' command it turns out that using adding `usb-storage.quirks=0080:a001:u` to the `cmdline.txt` seems to fix things for me. I've now plugged it back up to the USB 3.0 port and it's now working.

Thanks! :)

Phil_C
Posts: 8
Joined: Mon Oct 14, 2019 10:46 am

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 14, 2019 10:48 am

I recently bought an RPi4, so I'm very new to it. I used NOOBS to set it up and I'm running a webserver on it. A few days ago I read that USB memory sticks are better than SD card in terms of longevity, so I decided to make the change.

By doing some dancing back and forth through this thread, I managed to get it all working, so I thought I'd consolidate the steps I needed to take into one place for other NOOBS users. All this information is in this thread, but somewhat scattered. Here are the steps...

1. Download usb-boot (from the first post) and put it into your user's Home folder (e.g. /home/pi).

2. Make it executable with chmod +x usb-boot

3. If the USB stick in already connected, remove it.
3a. Open the File Manager
3b. Click on Edit->Preferences
3c. In the Volume Management tab, uncheck 'Mount removeable media automatically when they are inserted'. This will prevent the USB stick from being automatically mounted when you connect it.
3d. Connect the USB memory stick.

4. Run usb-boot (./usb-boot).
4a. Select the USB device by pressing the space bar.
4b. Use the tab key to navigate to the Yes button and press the space bar.

It will take about 15 minutes to copy everything from the SD card to the USB stick. When it's done, reboot the Pi and it will run from the USB stick.

Leave the SD card in. Apparently it's needed to be there.

Check that the Pi is now running from the USB stick by typing:-
mount | grep " on / "
into the console. It should come back with, "/dev/sda2 on type ext4 (rw, noatime)", which means that it's running on the USB stick. Also two new icons appear on the GUI - '256 MB Volume' and 'root'.

The auto-mount option in Edit->Preferences can be checked again. The Pi continues to run from the USB stick after reboots.

Phil_C
Posts: 8
Joined: Mon Oct 14, 2019 10:46 am

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 14, 2019 4:23 pm

I forgot to include one thing from my previous post, and that is a thank you to RonR, but I can't see a way to edit the post. So...

Thank you, RonR, for the thread, the script, and the time you've put into it since you started the thread.

LTolledo
Posts: 2010
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Running Raspbian from USB Devices : Made Easy

Sat Oct 19, 2019 9:17 am

First of all.... THANKS to RonR for the script.

Tried the script for the first time on my RPi4B and a NVMe SATA drive (just have one lying around to play around)
had trouble trying to run the script.... but managed to do it after a few research.

and it did run as advertised... great!

however I did noticed something...
my Raspbian Buster on the microSD was already fully setup, including required mount points for network shares and its symlinks to desktop
after completing the script and rebooting....
the mount points in /mnt were not included... therefore the symlinks were invalid...

if the whole root fs was supposed to be transferred to target USB device.... the mount points were not....
checked the original root fs on the microSD... its indeed there.... but not on the transferred root fs on the USB device

was this by design or do I need to tweak the script to truly transfer the whole entirety of the root fs to the target USB drive.

if, however, the script is ran right after the initial Rasbian install.... then this is not a problem.....
..... and still highly recommended for those who want to have their root fs on a USB drive.... and automating the process to do it.

a progress indicator would be a nice "improvement" on the script... (if it is possible)

once again thanks to RonR!
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Sun Oct 20, 2019 7:14 pm

LTolledo wrote:
Sat Oct 19, 2019 9:17 am
my Raspbian Buster on the microSD was already fully setup, including required mount points for network shares and its symlinks to desktop
after completing the script and rebooting....
the mount points in /mnt were not included... therefore the symlinks were invalid...

/media and /mnt are intentionally excluded from the rsync copy. You would need to manually recreate the mount points in those directories.

LTolledo
Posts: 2010
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 21, 2019 10:09 am

RonR wrote: /media and /mnt are intentionally excluded from the rsync copy. You would need to manually recreate the mount points in those directories.
...well, that's already taken care of.... no problem there.... :D

are there any other folders that are intentionally excluded from the rsync process? just for us to know...
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 21, 2019 4:39 pm

LTolledo wrote:
Mon Oct 21, 2019 10:09 am
are there any other folders that are intentionally excluded from the rsync process?

/dev
/media
/mnt
/proc
/run
/sys
/tmp
/lost+found

User avatar
rpdom
Posts: 15385
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 21, 2019 7:50 pm

Why not just use the -x one --one-file-system option for rsync? That will copy everything in the selected file system including mount points, but not anything that is mounted in them - that excludes anything in /dev, /sys, /proc etc...
Ok, so it will copy the contents of ./lost+found, but that should be empty unless there have been problems.

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Mon Oct 21, 2019 9:54 pm

rpdom wrote:
Mon Oct 21, 2019 7:50 pm
Why not just use the -x one --one-file-system option for rsync? That will copy everything in the selected file system including mount points, but not anything that is mounted in them - that excludes anything in /dev, /sys, /proc etc...
Ok, so it will copy the contents of ./lost+found, but that should be empty unless there have been problems.

I'm aware of all that, but -x will also exclude copying the contents of /boot. This can be solved by doing two separate rsync's, but that would require a little reworking of the current script. I've been considering the options.

PosterMeal
Posts: 1
Joined: Wed Nov 06, 2019 3:05 pm

Re: Running Raspbian from USB Devices : Made Easy

Wed Nov 06, 2019 3:07 pm

Hello

Is it a software (firmware) problem of booting from SSD, dont u know?
I ask because if i get pi4 now, will i be able to upgrade it to ssd when “ssd boot” update will be ready or it will be hardware modified?

Thanks

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Wed Nov 06, 2019 6:54 pm

PosterMeal wrote:
Wed Nov 06, 2019 3:07 pm
Is it a software (firmware) problem of booting from SSD, dont u know?
I ask because if i get pi4 now, will i be able to upgrade it to ssd when “ssd boot” update will be ready or it will be hardware modified?

The ability to boot directly from a USB device without the presence of an SD card is strictly a software change (boot EEPROM). No hardware modifications will be necessary.

In the meantime, there is no downside to leaving an SD card inserted and booting from a USB device using usb-boot in the first post of this topic. The end result is the same.

Brad Q
Posts: 34
Joined: Mon Aug 12, 2019 12:10 am

Re: Running Raspbian from USB Devices : Made Easy

Wed Nov 06, 2019 9:22 pm

Ron
Thanks for making this. It has allowed me to use my Pi4 for my daily drivers. Just wondering if, when you update this, do those that have already used it to set up their systems need to do anything?

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Wed Nov 06, 2019 9:44 pm

Brad Q wrote:
Wed Nov 06, 2019 9:22 pm
Just wondering if, when you update this, do those that have already used it to set up their systems need to do anything?

No, it's pretty much a one-shot process.

If you're already booting from a USB device, you can also use usb-boot to move everything from the current USB device to a different USB device (for example, a faster or larger USB device).

LTolledo
Posts: 2010
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Running Raspbian from USB Devices : Made Easy

Wed Nov 06, 2019 10:01 pm

Hi there RonR!

one thing I recently noticed on my RPi4B-4G running raspbian from USB-SSD is that

it ignores any drive plugged into any of the vacant USB ports....
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Wed Nov 06, 2019 10:21 pm

LTolledo wrote:
Wed Nov 06, 2019 10:01 pm
one thing I recently noticed on my RPi4B-4G running raspbian from USB-SSD is that

it ignores any drive plugged into any of the vacant USB ports....

On my RPi4B-4G, usb-boot sees all USB drives that are plugged in.

Brad Q
Posts: 34
Joined: Mon Aug 12, 2019 12:10 am

Re: Running Raspbian from USB Devices : Made Easy

Wed Nov 06, 2019 10:48 pm

I just checked on my 4-4GB and it sees all four ports as well. Tested by moving USB stick from port to port.

ccpetersen
Posts: 9
Joined: Fri Dec 04, 2015 4:03 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Thu Nov 07, 2019 3:01 am

Just a big thanks to RonR for the script and forum support. The script works like a champ and I learned a few tricks from it.

Excellent work! :D :D

LTolledo
Posts: 2010
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Running Raspbian from USB Devices : Made Easy

Thu Nov 07, 2019 9:54 am

RonR wrote:
Wed Nov 06, 2019 10:21 pm
LTolledo wrote:
Wed Nov 06, 2019 10:01 pm
one thing I recently noticed on my RPi4B-4G running raspbian from USB-SSD is that

it ignores any drive plugged into any of the vacant USB ports....

On my RPi4B-4G, usb-boot sees all USB drives that are plugged in.
Well it does see any drive connected if it was plugged before boot-up (auto-mounted in /media/pi)
after boot-up... ejected the detected drive and re-plug it to the same port... its not auto-mounted (however I see it as /dev/sda, but no mount point)

ok here is the lsblk output when plugged before boot

Code: Select all

pi@RPi4B-4G:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 111.8G  0 disk 
├─sda1        8:1    0    94M  0 part /media/pi/BOOT
└─sda2        8:2    0 111.7G  0 part /media/pi/55521fec-d6c2-42f5-bb6e-b34c0c71
sdb           8:16   0 119.2G  0 disk 
├─sdb1        8:17   0   256M  0 part /media/pi/BE1E-AFEC
└─sdb2        8:18   0   119G  0 part /
mmcblk0     179:0    0  59.6G  0 disk 
├─mmcblk0p1 179:1    0   256M  0 part /boot
└─mmcblk0p2 179:2    0  59.4G  0 part /media/pi/rootfs
ejecting and re-plugging again

Code: Select all

pi@RPi4B-4G:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 111.8G  0 disk 
├─sda1        8:1    0    94M  0 part 
└─sda2        8:2    0 111.7G  0 part 
sdb           8:16   0 119.2G  0 disk 
├─sdb1        8:17   0   256M  0 part /media/pi/BE1E-AFEC
└─sdb2        8:18   0   119G  0 part /
mmcblk0     179:0    0  59.6G  0 disk 
├─mmcblk0p1 179:1    0   256M  0 part /boot
└─mmcblk0p2 179:2    0  59.4G  0 part /media/pi/rootfs
so the "ignoring" means "not auto-mounting"....

the drive used in this test is a Crucial BX500 120GB SSD (containing my old USB-boot Raspbian Stretch system)

trying it with a Transcend 16GB USB 2.0 pen drive , FAT32 formatted

Code: Select all

pi@RPi4B-4G:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    1  15.1G  0 disk 
└─sda1        8:1    1  15.1G  0 part 
sdb           8:16   0 119.2G  0 disk 
├─sdb1        8:17   0   256M  0 part /media/pi/BE1E-AFEC
└─sdb2        8:18   0   119G  0 part /
mmcblk0     179:0    0  59.6G  0 disk 
├─mmcblk0p1 179:1    0   256M  0 part /boot
└─mmcblk0p2 179:2    0  59.4G  0 part /media/pi/rootfs
rebooted it then checked

Code: Select all

pi@RPi4B-4G:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 119.2G  0 disk 
├─sda1        8:1    0   256M  0 part /media/pi/BE1E-AFEC
└─sda2        8:2    0   119G  0 part /
sdb           8:16   1  15.1G  0 disk 
└─sdb1        8:17   1  15.1G  0 part /media/pi/ECE4-C138
mmcblk0     179:0    0  59.6G  0 disk 
├─mmcblk0p1 179:1    0   256M  0 part /boot
└─mmcblk0p2 179:2    0  59.4G  0 part /media/pi/rootfs
pi@RPi4B-4G:~ $ 
so it was auto-mounted as /media/pi/ECE4-C138
but not auto-mounted if "hot-plugged"

the port used for this test is a vacant USB2.0 port.
tried it with the the USB3.0 port... same results as above.

any place I need to tweak to "auto-mount" hot plugged drives?
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

RonR
Posts: 550
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian from USB Devices : Made Easy

Thu Nov 07, 2019 5:59 pm

LTolledo wrote:
Thu Nov 07, 2019 9:54 am
any place I need to tweak to "auto-mount" hot plugged drives?

usb-boot uses a simple 'ls -l /dev/sd?' command to find USB mass storage devices.

As stated in a previous post:
RonR wrote:
Tue Aug 20, 2019 3:24 am
usb-boot (in the first post of this topic) has been updated.

usb-boot now supports NOOBS.

If you answer 'Yes' to 'Replicate BOOT/ROOT contents from /dev/mmcblk0 to /dev/sdX?', the selected USB device must not be mounted anywhere (Raspbian Desktop auto-mounts USB devices). Previous versions of usb-boot did not check and the script would report failures if the USB device was mounted. usb-boot now checks and refuses to continue if the selected USB device is mounted. Simply eject or unmount (umount) the selected USB device and run usb-boot again.

I don't understand what

"so the "ignoring" means "not auto-mounting"...."

and

"so it was auto-mounted as /media/pi/ECE4-C138 but not auto-mounted if "hot-plugged""

have to do with usb-boot.

usb-boot doesn't get involved with auto-mounting.

usb-boot simply checks and refuses to proceed with a "Replicate BOOT/ROOT contents" if the target device is mounted anywhere:

Code: Select all

    if [ $(mount | grep -c "^${USB_DEST}") -ne 0 ]; then
      errexit "USB mass storage device in use (mounted)"
    fi

LTolledo
Posts: 2010
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Running Raspbian from USB Devices : Made Easy

Thu Nov 07, 2019 10:19 pm

Just to clarify:

The condition stated was observed on an RPi4B-4G with Raspbian Buster Full Desktop system after it was successfully transferred from microSD to USB-SSD using usb-boot. The system is running fine with the rootfs located on the USB-SSD.

prior to the change (still on native microSD boot), any USB device hot-plugged to RPi4B's vacant USB port is auto-mounted in /media/pi

after using usb-boot to transfer the system to USB-SSD, any USB drive hot-plugged to any USB port is not auto-mounted

however, any USB drive plugged to RPi4B's USB before bootup, the USB drive is always auto-mounted.


so...when running the usb-boot script, was there modification in the system setting that prevents a successfully transferred system from auto-mounting any hot-plugged drives?
if there is... would like to undo that setting myself, to re-enable hot-plugging on a successfully transferred system.
if none... will look at something else....

thanks
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

Return to “Advanced users”