bob432626
Posts: 22
Joined: Thu May 30, 2019 2:51 pm

can you do a raid with pis?

Sun Oct 13, 2019 6:28 am

hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)

fruitoftheloom
Posts: 20900
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: can you do a raid with pis?

Sun Oct 13, 2019 8:08 am

bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)

Parallel / Distributed Computing is not the same as Hard Drive Raid, time for a Google:


Distributed Computing:

https://www.pidramble.com/


RAID:

https://unix.stackexchange.com/question ... rage-media
Retired disgracefully.....

LTolledo
Posts: 2006
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: can you do a raid with pis?

Sun Oct 13, 2019 10:38 am

One can do RAID with RPi... it can be done.... software RAID.....done that before... as RAID1

....but I cannot recommend it if the drives are USB based....


.... however ...if a hardware RAID HAT will be available (if ever)... and able to acquire and test it for myself....then maybe I may recommend it....
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

ejolson
Posts: 3681
Joined: Tue Mar 18, 2014 11:47 am

Re: can you do a raid with pis?

Sun Oct 13, 2019 3:02 pm

bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)
To use many Pi computers together for fault-tolerant storage, you need some sort of distributed filesystem. At the moment the popular choices seem to be Gluster, Andrew and Ceph. In not sure which run on Raspbian, but since all three are open source and work with Linux, it is likely all can be made somehow to run on the Raspberry Pi.

In many ways the most interesting is Ceph, partly because it's the newest. None are particularly easy, in my opinion, and all require at least three computers--possibly more--to work at all. While it makes more practical sense for home use to buy a cheap NAS, in certain large-scale commercial environments distributed filesystems are a necessity and learning how they work could be educational.

If you have success with a bunch of Pi computers and any of these distributed file systems, it would be great to share how you set things up on this forum. I would be quite interested and suspect there are others who would be too.

BeauSlim
Posts: 56
Joined: Mon Jul 31, 2017 10:02 am

Re: can you do a raid with pis?

Sun Oct 13, 2019 4:54 pm

The simplest RAID solution would probably be to use a hardware-RAID USB disk enclosure like the "RAID" products from Orico and Mediasonic et al. I don't own one so I haven't tried one with the Pi.

Multi-JBOD USB enclosures do work with software-raid via mdadm or btrfs or zfs but software RAID over USB is very much not recommended by experts due to reliability problems and (having tried it) performance tends to be terrible anyway.

Always remember that RAID IS NOT A BACKUP.

ejolson
Posts: 3681
Joined: Tue Mar 18, 2014 11:47 am

Re: can you do a raid with pis?

Sun Oct 13, 2019 6:06 pm

The original post where
bob432626 wrote:
Sun Oct 13, 2019 6:28 am
pretty much all the pis worked together as 1 computer
makes me think the question is not about traditional RAID with a single Pi but how to set up a fail-safe distributed filesystem using multiple Pi computers.

It is widely documented how to configure a single server with a redundant array of disks. In my opinion RAID works well up to about 100TB. For petabyte storage requirements, however, a distributed approach may be preferable for greater resiliency as well as bandwidth.

Ceph generally requires separate computers for the monitor, manager, metadata server and the online storage devices. A realistic setup might consist of three monitors, two managers, two metadata servers and three or more online storage devices. That's already at least 10 computers, which is quite a few just to set up an environment for learning about things at home.

Luckily, many people have the budget to buy and provide the electricity for that many Raspberry Pi computers--even the 4GB model. This allows someone, who wouldn't have had an opportunity otherwise, to become an expert in setting up and maintaining high-performance distributed filesystems.

At the same time, it might be more productive to set a Pi-controlled drip sprinkler system up for growing tomatoes in the garden.

bob432626
Posts: 22
Joined: Thu May 30, 2019 2:51 pm

Re: can you do a raid with pis?

Sun Oct 13, 2019 6:53 pm

fruitoftheloom wrote:
Sun Oct 13, 2019 8:08 am
bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)

Parallel / Distributed Computing is not the same as Hard Drive Raid, time for a Google:


Distributed Computing:

https://www.pidramble.com/


RAID:

https://unix.stackexchange.com/question ... rage-media
THANK YOU, you understand what i mean in this

Ernst
Posts: 1244
Joined: Sat Feb 04, 2017 9:39 am
Location: Germany

Re: can you do a raid with pis?

Mon Oct 14, 2019 6:35 am

bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)
https://media.ccc.de/v/1428-ceph-explai ... pberry-pis

https://bryanapperson.com/blog/the-defi ... pberry-pi/
The road to insanity is paved with static ip addresses

Return to “General discussion”