AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Large partition sizes, Raspbian and Windows

Sun Mar 10, 2019 9:53 pm

I’m trying to understand how the Linux/Raspbian operating system works with regard to the extended partition (s) beyond the boot partition. Since windows can’t see and deal with the (. ) partitions, I cant load them into Windows to copy or delete data there. If I format the SD card so it is usable in Windows AND Raspbian. I seem to remember Raspbian makes a series of 2gb partitions in the extended space on the bootable SD card.

My deliema is that I’m trying to build a small portable music player with a storage capacity of 256-512gb for music. I could save a TON of space inside this player by utilizing the extra space on the SD boot card. But moving music to and from that card after setting it up for use by Raspbian is difficult to impossible.

Any suggestions on how I can save space by using the bootable SD card to store music AND have it accessible to both Raspbian and Windows (8.1)?

LTolledo
Posts: 2154
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Large partition sizes, Raspbian and Windows

Sun Mar 10, 2019 10:05 pm

Your other post will likely be locked
and discussions continue here.

To better understand what your requirements are first let us know your setup:
1. What RPi model?
2. What boot up media?
3. Storage media installed?
4. Power supply unit
5. attached Sound card/DAC model/type
6. other peripherals?

Based on your requirements, it is possible for the music storage space to be accessible to both raspbian and windows. But first we need to know at least 1 to 4 of the above.
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

Ernst
Posts: 1257
Joined: Sat Feb 04, 2017 9:39 am
Location: Germany

Re: Large partition sizes, Raspbian and Windows

Sun Mar 10, 2019 10:20 pm

AirFiero wrote:
Sun Mar 10, 2019 9:53 pm
Any suggestions on how I can save space by using the bootable SD card to store music AND have it accessible to both Raspbian and Windows (8.1)?
I agree with the questions by LTolledo in the previous post, but I can provide a simple method without having the answers.

What you need is a memory card large enough for the operating system and anything else you intend to install, load raspbian on this card and boot up the Pi and complete the installation. When Raspbian is initially booted the partition is extended using the free space.

Now use Win32DiskImager on your Windows PC to create an image from the SD card, next write this image using Win32DiskImager to a larger memory card, or hard disk. Once completed the remaining space can be used to create a partition (volume) using the Windows disk manager. You will have to select a file system type and install the modules needed.
The road to insanity is paved with static ip addresses

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Sun Mar 10, 2019 11:15 pm

LTolledo wrote:
Sun Mar 10, 2019 10:05 pm
Your other post will likely be locked
and discussions continue here.

To better understand what your requirements are first let us know your setup:

Based on your requirements, it is possible for the music storage space to be accessible to both raspbian and windows. But first we need to know at least 1 to 4 of the above.
1. What RPi model?

For the portable music player, Pi Zero W using Volumio (for now, probably will try other players). I also plan to do music players using a Pi 3B+, currently using Raspbian.

2. What boot up media?

SD card, 256-512gb

3. Storage media installed?

If I can't get this to work on the bootable SD card, I'd use a USB stick or card reader. I've tested the latter method, and it works fine with Pi 3B+ and Raspbian. However, this takes up a lot of space in the player. Currently, the player should end up being no larger than a pack of cigarettes.

4. Power supply unit

For the portable, the Adafruit Power Boost 1000 with a 2500mAh LiPo battery.
https://www.adafruit.com/product/2465

5. attached Sound card/DAC model/type

IQaudio PiDACZero with headphone amp daughter board.

6. other peripherals?

Small display, 5 inches or less.

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Sun Mar 10, 2019 11:17 pm

Ernst wrote:
Sun Mar 10, 2019 10:20 pm
AirFiero wrote:
Sun Mar 10, 2019 9:53 pm
Any suggestions on how I can save space by using the bootable SD card to store music AND have it accessible to both Raspbian and Windows (8.1)?
I agree with the questions by LTolledo in the previous post, but I can provide a simple method without having the answers.

What you need is a memory card large enough for the operating system and anything else you intend to install, load raspbian on this card and boot up the Pi and complete the installation. When Raspbian is initially booted the partition is extended using the free space.

Now use Win32DiskImager on your Windows PC to create an image from the SD card, next write this image using Win32DiskImager to a larger memory card, or hard disk. Once completed the remaining space can be used to create a partition (volume) using the Windows disk manager. You will have to select a file system type and install the modules needed.
I believe I've tried this before. If I recall correctly, once Raspbian does the extension, the extended storage is not visible to Windows for copying files. I also believe that Raspbian makes a series of 2 or 4 gb partitions. Wouldn't I end up with 100 2gb partitions on a 256gb card?

User avatar
Imperf3kt
Posts: 2985
Joined: Tue Jun 20, 2017 12:16 am
Location: Australia

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 2:33 am

Do you not own a single usb flash drive?
That would be the easiest method and is how I've been transferring files between Linux, Mac, Android, Windows, iPhone, various models of Sony's playstation OS, even my watch.

As an added bonus, any media on the stick will also play on my car stereo.

USB flash drives live up to their plug and play name, in my experience.
55:55:44:44:4C
52:4C:52:42:41

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 3:49 am

Imperf3kt wrote:
Mon Mar 11, 2019 2:33 am
Do you not own a single usb flash drive?
That would be the easiest method and is how I've been transferring files between Linux, Mac, Android, Windows, iPhone, various models of Sony's playstation OS, even my watch.

As an added bonus, any media on the stick will also play on my car stereo.

USB flash drives live up to their plug and play name, in my experience.
Yes, I own several including a couple of 256gb drives. You may have missed the point that I’m trying to make this player as *small* as possible. Utilizing the boot SD card would save a LOT of space, not to mention saving some cost.

ghans
Posts: 7878
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:21 am

Buy an USB SD card reader and a second SD card. Plug both into your running Pi.
Use the SD card copier. Use gparted to increase the size of the first partition.

Now you can use the second SD card on Windows and Linux.

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org

User avatar
HawaiianPi
Posts: 4876
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:23 am

There are ways to transfer files between Windows and Linux. Things like FTP or SCP could be used, or share a folder on the Pi over your network. So there are three possible solutions off the top of my head (and I'm sure others I missed).

I use WinSCP to grab stuff from my Torrent box (because I hate SAMBA). SCP let's you transfer files using SSH protocol, and WinSCP is a GUI file manager that makes it simple. Not as fast as a network share, but as simple as SSH.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

User avatar
Imperf3kt
Posts: 2985
Joined: Tue Jun 20, 2017 12:16 am
Location: Australia

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:37 am

AirFiero wrote:
Mon Mar 11, 2019 3:49 am
Imperf3kt wrote:
Mon Mar 11, 2019 2:33 am
Do you not own a single usb flash drive?
That would be the easiest method and is how I've been transferring files between Linux, Mac, Android, Windows, iPhone, various models of Sony's playstation OS, even my watch.

As an added bonus, any media on the stick will also play on my car stereo.

USB flash drives live up to their plug and play name, in my experience.
Yes, I own several including a couple of 256gb drives. You may have missed the point that I’m trying to make this player as *small* as possible. Utilizing the boot SD card would save a LOT of space, not to mention saving some cost.
You asked for a way to transfer the files, right?
The USB stick does not need to remain in the Pi any longer than the transfer takes to complete.

Or is there some other reason you don't want to use this method that I am not seeing?
55:55:44:44:4C
52:4C:52:42:41

User avatar
rpdom
Posts: 15596
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:45 am

AirFiero wrote:
Sun Mar 10, 2019 11:17 pm
If I recall correctly, once Raspbian does the extension, the extended storage is not visible to Windows for copying files. I also believe that Raspbian makes a series of 2 or 4 gb partitions. Wouldn't I end up with 100 2gb partitions on a 256gb card?
Raspbian (without NOOBS) just has two partitions. The small boot partition (about 60-80MB), which is the one the Windows can see, and the main Linux partition, which is expanded to fill the card no matter how big it is.

In theory you could change the size of the partitions so that the boot one is a lot bigger and use that for your shared storage. You'd need to do that on a Linux system using something like gparted. Also you may have problems on the Pi because of the restricted permissions on that mount

You can get tiny USB memory sticks that barely protrude from the socket. Maybe that would be a better plan?

Ernst
Posts: 1257
Joined: Sat Feb 04, 2017 9:39 am
Location: Germany

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 9:37 am

AirFiero wrote:
Sun Mar 10, 2019 11:17 pm
Ernst wrote:
Sun Mar 10, 2019 10:20 pm
AirFiero wrote:
Sun Mar 10, 2019 9:53 pm
Any suggestions on how I can save space by using the bootable SD card to store music AND have it accessible to both Raspbian and Windows (8.1)?
I agree with the questions by LTolledo in the previous post, but I can provide a simple method without having the answers.

What you need is a memory card large enough for the operating system and anything else you intend to install, load raspbian on this card and boot up the Pi and complete the installation. When Raspbian is initially booted the partition is extended using the free space.

Now use Win32DiskImager on your Windows PC to create an image from the SD card, next write this image using Win32DiskImager to a larger memory card, or hard disk. Once completed the remaining space can be used to create a partition (volume) using the Windows disk manager. You will have to select a file system type and install the modules needed.
I believe I've tried this before. If I recall correctly, once Raspbian does the extension, the extended storage is not visible to Windows for copying files. I also believe that Raspbian makes a series of 2 or 4 gb partitions. Wouldn't I end up with 100 2gb partitions on a 256gb card?
You are right but for the wrong reasons. I have just now tried my suggestion on an SD card and it fails because Windows does not recognize the additional partition on a removable disk. If I remember correctly I have been able to do this in the past but it may have been on a hard disk that was not recognized as a removable disk.
The road to insanity is paved with static ip addresses

LTolledo
Posts: 2154
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 9:56 am

Ok this is my take on your requirements

1. RPiZW
2. 512GB MicroSD
3. 2500mAh LiPo Battery
4. iQAudio PiDACZero
5. 5" display (HDMI type?, GPIO header type?)

Stacking recommendation (feel free to change as you like)
Top stack ---> 5" display
Mid stack --> RPiZW
Bottom stack --> iQAudio DAC

If RPiZWH, it has to be at the bottom stack.

Probable procedure for the 512microSD
1. Flash the latest volumio image, and let it finish its initial installation on the RPiZW (including the sound card installation, try few web radio stations first)
2. once Volumio installation is finished, power off and take out the microSD card
3. in a linux pc (or Raspbian Stretch with Desktop) plug the microSD card to the USB port (using a compatible microUSB to microSD adapter)
4. use gparted to shrink the root directory to about 8GB (volumio system can run on just 8GB)
5. format the remaining space as NTFS
6. now on a winpc, copy all your favorite music files to the NTFS partition.

Once copying is finished, plug the 512GB microSD back to the RPiZW system and boot it up.
See if Volumio can scan the music file contents of the NTFS directory. If it does, then hope its up to your liking.

have fun!
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 4:54 pm

HawaiianPi wrote:
Mon Mar 11, 2019 6:23 am
There are ways to transfer files between Windows and Linux. Things like FTP or SCP could be used, or share a folder on the Pi over your network. So there are three possible solutions off the top of my head (and I'm sure others I missed).

I use WinSCP to grab stuff from my Torrent box (because I hate SAMBA). SCP let's you transfer files using SSH protocol, and WinSCP is a GUI file manager that makes it simple. Not as fast as a network share, but as simple as SSH.
This sounds like a good set of solutions, thank you.

I haven’t tried yet, but I need to see if Volumio has a method for file transfer under its GUI, or if I will need to turn off the GUI when I need to use the command line to have SSH access.

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 4:58 pm

Imperf3kt wrote:
Mon Mar 11, 2019 6:37 am
AirFiero wrote:
Mon Mar 11, 2019 3:49 am
Imperf3kt wrote:
Mon Mar 11, 2019 2:33 am
Do you not own a single usb flash drive?
That would be the easiest method and is how I've been transferring files between Linux, Mac, Android, Windows, iPhone, various models of Sony's playstation OS, even my watch.

As an added bonus, any media on the stick will also play on my car stereo.

USB flash drives live up to their plug and play name, in my experience.
Yes, I own several including a couple of 256gb drives. You may have missed the point that I’m trying to make this player as *small* as possible. Utilizing the boot SD card would save a LOT of space, not to mention saving some cost.
You asked for a way to transfer the files, right?
The USB stick does not need to remain in the Pi any longer than the transfer takes to complete.

Or is there some other reason you don't want to use this method that I am not seeing?
Sorry, I didn’t mean to come off snarky. I need to experiment with Volumio to see how easy it is to access files and do the transfer in this manner. Have a Pi 3B+ set up with Volumio as a test unit. I’ll fire it up and test it.

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 5:01 pm

rpdom wrote:
Mon Mar 11, 2019 6:45 am
AirFiero wrote:
Sun Mar 10, 2019 11:17 pm
If I recall correctly, once Raspbian does the extension, the extended storage is not visible to Windows for copying files. I also believe that Raspbian makes a series of 2 or 4 gb partitions. Wouldn't I end up with 100 2gb partitions on a 256gb card?
Raspbian (without NOOBS) just has two partitions. The small boot partition (about 60-80MB), which is the one the Windows can see, and the main Linux partition, which is expanded to fill the card no matter how big it is.

In theory you could change the size of the partitions so that the boot one is a lot bigger and use that for your shared storage. You'd need to do that on a Linux system using something like gparted. Also you may have problems on the Pi because of the restricted permissions on that mount

You can get tiny USB memory sticks that barely protrude from the socket. Maybe that would be a better plan?
I can see experiment with gparted and also see if there are permissions issues.

I’ve been searching for usb sticks and readers that are as small as possible. So far, the smallest ones are USB C, which would need an adapter, which negates the space saving. I’ll keep looking.

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 5:35 pm

LTolledo wrote:
Mon Mar 11, 2019 9:56 am
Ok this is my take on your requirements

1. RPiZW
2. 512GB MicroSD
3. 2500mAh LiPo Battery
4. iQAudio PiDACZero
5. 5" display (HDMI type?, GPIO header type?)
I found a 4.3 inch display, Here:

https://www.waveshare.com/4.3inch-hdmi-lcd-b.htm

I have no idea if that will work. I asked the question elsewhere on this forum...will HDMI carry the touch portion of the touchscreen data to a Pi?

I can’t use a GPIO display because the IQaudio DAC takes all the header pins.

Stacking recommendation (feel free to change as you like)
Top stack ---> 5" display
Mid stack --> RPiZW
Bottom stack --> iQAudio DAC

If RPiZWH, it has to be at the bottom stack.[/quote]

Gordon at IQAudio was nice enough to create a custom stack for me of a Pi Zero W, DAC and headphone amp daughter board all crunched down into a stack 3/4 inch or 19mm high. If I can post pictures here somehow, I’ll show you what it looks like.
Probable procedure for the 512microSD
1. Flash the latest volumio image, and let it finish its initial installation on the RPiZW (including the sound card installation, try few web radio stations first)
2. once Volumio installation is finished, power off and take out the microSD card
3. in a linux pc (or Raspbian Stretch with Desktop) plug the microSD card to the USB port (using a compatible microUSB to microSD adapter)
4. use gparted to shrink the root directory to about 8GB (volumio system can run on just 8GB)
5. format the remaining space as NTFS
6. now on a winpc, copy all your favorite music files to the NTFS partition.

Once copying is finished, plug the 512GB microSD back to the RPiZW system and boot it up.
See if Volumio can scan the music file contents of the NTFS directory. If it does, then hope its up to your liking.

have fun!
I can certainly try this, but I suspect that the Pi will not recognize the NTFS partition on the boot SD micro card.

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 5:42 pm

ghans wrote:
Mon Mar 11, 2019 6:21 am
Buy an USB SD card reader and a second SD card. Plug both into your running Pi.
Use the SD card copier. Use gparted to increase the size of the first partition.

Now you can use the second SD card on Windows and Linux.

ghans
Is the size of the partition limited under Linux to 2 or 4gb, including the boot partition? Or is it theoretically unlimited? I'm hoping to use a 256gb space (however that is implemented) for music storage.

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:18 pm

To put some size perspective on this project, which I'm calling "RasPod", here is a picture of the components so far, and an iPod Classic 160gb and Pi 3B+ for size comparison, along with some USB sticks and card readers...
The attachment 20190311_105833_resized_1.jpg is no longer available
The attachment IMG_7260.jpg is no longer available
20190311_105833_resized_1.jpg
20190311_105833_resized_1.jpg (221.9 KiB) Viewed 1966 times

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:21 pm

The white paper under the Pi Zero is the footprint I'm trying to get all of the components into.

Here is the Pi Zero stack:
The attachment IMG_7260.jpg is no longer available
IMG_7260.jpg
IMG_7260.jpg (66.05 KiB) Viewed 1965 times

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:22 pm

continued
Attachments
IMG_7262.jpg
IMG_7262.jpg (77.85 KiB) Viewed 1963 times

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 6:38 pm

Looking at the 4.3 inch monitor from Waveshare, it appears that it needs both an HDMI and USB connection. Since the Pi Zero only has one micro USB port, I would either need some kind of USB hub to accommodate two USB connections, or it would preclude me from using a USB stick/reader for music storage.

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 7:12 pm

rpdom wrote:
Mon Mar 11, 2019 6:45 am
AirFiero wrote:
Sun Mar 10, 2019 11:17 pm
If I recall correctly, once Raspbian does the extension, the extended storage is not visible to Windows for copying files. I also believe that Raspbian makes a series of 2 or 4 gb partitions. Wouldn't I end up with 100 2gb partitions on a 256gb card?
Raspbian (without NOOBS) just has two partitions. The small boot partition (about 60-80MB), which is the one the Windows can see, and the main Linux partition, which is expanded to fill the card no matter how big it is.

In theory you could change the size of the partitions so that the boot one is a lot bigger and use that for your shared storage. You'd need to do that on a Linux system using something like gparted. Also you may have problems on the Pi because of the restricted permissions on that mount

You can get tiny USB memory sticks that barely protrude from the socket. Maybe that would be a better plan?

I've located gparted, and it appears to have two versions, one to use on Windows, and one (each) for various flavors of Linux. Just so I understand, should I try gparted on an RPi, modifying the partion of an SD card attached to the USB port, or should I try this from Windows? Or both?

Another question, do we know the maximum size a boot partition can be under Raspbian?

AirFiero
Posts: 67
Joined: Tue Jan 15, 2019 10:56 pm

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 7:15 pm

In the process of trying to post pictures, part of my text explanation got lost.

Note the white piece of paper under the Pi Zero. This is the footprint I'm trying to fit all components into. I will also have about 3/4 inch or 19 mm vertically to stack or layer components into.
Attachments
20190311_105833_resized_1.jpg
20190311_105833_resized_1.jpg (221.9 KiB) Viewed 1939 times

User avatar
rpdom
Posts: 15596
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Large partition sizes, Raspbian and Windows

Mon Mar 11, 2019 7:21 pm

AirFiero wrote:
Mon Mar 11, 2019 7:12 pm
Another question, do we know the maximum size a boot partition can be under Raspbian?
The maximum size for a FAT32 partition is 2TB, so I guess that is all you can have.

Return to “Raspbian”