Pi-hole & OMV - Enclosure options for HDDs and the feasibility using two USB hubs to connect 4-8 HDDS

[PairedPrototype]

I've been thinking about building a NAS because I don't have any proper backups of my data. I've looked about at existing small NAS boxes and they offer limited redundancy or upgrade options and the ones that do, cost a lot. So I wanted to make my own. I think it would be a nice little project for a Pi 4 as well.

Ultimately, I intend to do 3 things with the Pi. I'd like it to operate as a NAS box, run Pi-hole and also be able to power on my desktop (probably just WOL) for when I'm not home. Yes, I could just leave my desktop running, but then it's unnecessarily putting my energy bill up. The Pi won't have an impact quite as much. So with that thought in mind, I'll want it powered 24/7.

I've not really jumped into Pi projects before, so I'm not sure of my available options to accomplish this. I know that USB 2 is too slow for a HDD, but I know USB 3 is significantly faster. So I'm thinking about grabbing a powered USB 3 hub to connect 2-4 drives on each of the ports giving me a total of 4-8 drives. The Pi would probably run Ubuntu Server (as I intend to run it completely headless) and have both OpenMediaVault and Pi-hole running as services.

I don't know of any Pi enclosures that could handle that many drives and space for 2 powered hubs. The most I've seen is just one 3.5" drive lol.

So out of this, I have 2 questions.
Is this plan practical and can it/will it work?
Are there any enclosures that could handle that many drives with some spare room to hide 2 USB hubs?

Thanks

[ejolson]

So out of this, I have 2 questions.
Is this plan practical and can it/will it work?
Are there any enclosures that could handle that many drives with some spare room to hide 2 USB hubs?

There is some discussion about a similar idea in the following thread:

https://www.raspberrypi.org/forums/view ... 5#p1526290

I've been using the four-disk enclosure that I linked in the previous thread for about a year and it works well.

More recently I've tried it with the Pi 4B.
The Pi had no trouble identifying all four drives in the enclosure as sda through sdd. I did, however, run into problems:

• The gigabit W Ethernet on the 4B was not compatible with my switch so I had to buy a new switch.
• The btrfs utility in Buster doesn't work properly with multi-drive disk pools, so I had to copy the older version of the program from Raspbian Stretch.

The enclosure has the advantage of a single power supply that guarantees the drives power on together. I suspect two such enclosures might work. I looked, but don't see an 8-drive version.

[PairedPrototype]

There is some discussion about a similar idea in the following thread:

https://www.raspberrypi.org/forums/view ... 5#p1526290

I've been using the four-disk enclosure that I linked in the previous thread for about a year and it works well. The Pi 4B had no trouble identifying all four drives in the enclosure as sda through sdd. It did, however, run into the following problems:

• The gigabit W Ethernet on the 4B was not compatible with my switch so I had to buy a new switch.
• The btrfs utility in Buster doesn't work properly with multi-drive disk pools, so I had to copy the older version of the program from Raspbian Stretch.

The enclosure has the advantage of cooling the drives, a reliable power supply and the fact that all drives are guaranteed to power on together. I suspect two such enclosures might work. I looked, but don't see an 8-drive version with similar specs.

Oooo, that looks like it would do the job! I just saw an 8 bay version through the Amazon link, it's more than x2 the price :O Would be cheaper to just get 2 x 4 bay model.

Thanks for the info. I'll do a little more searching to see if there's an alternative 8 bay box.

[ejolson]

There is some discussion about a similar idea in the following thread:

https://www.raspberrypi.org/forums/view ... 5#p1526290

I've been using the four-disk enclosure that I linked in the previous thread for about a year and it works well. The Pi 4B had no trouble identifying all four drives in the enclosure as sda through sdd. It did, however, run into the following problems:

• The gigabit W Ethernet on the 4B was not compatible with my switch so I had to buy a new switch.
• The btrfs utility in Buster doesn't work properly with multi-drive disk pools, so I had to copy the older version of the program from Raspbian Stretch.

The enclosure has the advantage of cooling the drives, a reliable power supply and the fact that all drives are guaranteed to power on together. I suspect two such enclosures might work. I looked, but don't see an 8-drive version with similar specs.

Oooo, that looks like it would do the job! I just saw an 8 bay version through the Amazon link, it's more than x2 the price :O Would be cheaper to just get 2 x 4 bay model.

Thanks for the info. I'll do a little more searching to see if there's an alternative 8 bay box.

Whatever you get, make sure it supports USB3, because the 8-drive enclosures I found only support eSATA.

[LTolledo]

as I have posted on another post:

this is my just newly completed DIY NAS project
contents are as follows:

• Raspberry Pi 4 Model B 2GB (will change to 1GB model when available)
  with heatsinks and fan (got myself a 30x30 heatsink with embedded 5v fan, installed over SoC)
  2 x 8TB 3.5" HDD
  SATA to USB 3.0 adapter boards (hacked from enclosures)
  in ITX case
  PSU is single 12V 5A "power brick"
  DIY buck converter board/adapter
  50x50x15 12v fan for HDD/case cooling
  wired GbE LAN connection to network

running Raspbian Buster Lite
instead of using it as a RAID1 unit, simple redundancy or "timed mirroring" is implemented

RPi4BNAS_project.jpg (190.37 KiB) Viewed 12072 times

pictured while doing initial transfer of files to HDD1 (top lid taken off to show the insides)

I could have used 2.5" drives but there were no 8TB or higher capacity versions that were reasonably priced (yet as of this writing) so as to have better space saving inside the case
do have the 90W ITX type PSU board.... but decided against its implementation (size/space requirement, and the use of the dummy load resistor), thus I made my own power distribution board...
at ambient of 27C am getting 47C ~50C during file transfer
to achieve better cooling of the SoC, removed the double sided thermal "insulator" tape and fastened the heatsink directly using thin thermal paste (there was a 7C to 9C difference in temp when the "insulator tape" was used )

may build another version of this later on.....

[PairedPrototype]

Whatever you get, make sure it supports USB3, because the 8-drive enclosures I found only support eSATA.

Indeed, I'll be keeping an eye out for that one

as I have posted on another post:

this is my just newly completed DIY NAS project
contents are as follows:

• Raspberry Pi 4 Model B 2GB (will change to 1GB model when available)
  with heatsinks and fan (got myself a 30x30 heatsink with embedded 5v fan, installed over SoC)
  2 x 8TB 3.5" HDD
  SATA to USB 3.0 adapter boards (hacked from enclosures)
  in ITX case
  PSU is single 12V 5A "power brick"
  DIY buck converter board/adapter
  50x50x15 12v fan for HDD/case cooling
  wired GbE LAN connection to network

running Raspbian Buster Lite
instead of using it as a RAID1 unit, simple redundancy or "timed mirroring" is implemented

RPi4BNAS_project.jpg


pictured while doing initial transfer of files to HDD1 (top lid taken off to show the insides)

I could have used 2.5" drives but there were no 8TB or higher capacity versions that were reasonably priced (yet as of this writing) so as to have better space saving inside the case
do have the 90W ITX type PSU board.... but decided against its implementation (size/space requirement, and the use of the dummy load resistor), thus I made my own power distribution board...
at ambient of 27C am getting 47C ~50C during file transfer
to achieve better cooling of the SoC, removed the double sided thermal "insulator" tape and fastened the heatsink directly using thin thermal paste (there was a 7C to 9C difference in temp when the "insulator tape" was used )

may build another version of this later on.....

That looks pretty sweet!

[Pimaxin]

instead of using it as a RAID1 unit, simple redundancy or "timed mirroring" is implemented

So I have 2 SSD's connected to the USB 3.0 ports and OMV installed and it won't even allow you configure any RAID with USB (it says it's too unreliable). Can you tell me how you do your "timed mirroring"?

[ejolson]

instead of using it as a RAID1 unit, simple redundancy or "timed mirroring" is implemented

So I have 2 SSD's connected to the USB 3.0 ports and OMV installed and it won't even allow you configure any RAID with USB (it says it's too unreliable). Can you tell me how you do your "timed mirroring"?

The reason USB is considered unreliable for RAID is because how likely it is for one drive to get unplugged while the others are left online. If you are certain this won't happen, I think it would be possible to override the recommendation made by OMV and configure the drives as RAID1 anyway. Unfortunately, I don't know exactly how to do this.

Sometimes it seems the only people who use OMV are the developers who are trying to debug it.

[default_user8]

instead of using it as a RAID1 unit, simple redundancy or "timed mirroring" is implemented

So I have 2 SSD's connected to the USB 3.0 ports and OMV installed and it won't even allow you configure any RAID with USB (it says it's too unreliable). Can you tell me how you do your "timed mirroring"?

Not sure what he considers "timed mirroring" to be, but I"m using a cronjob to run rsync to mirror my main NAS drive to another connected drive once a week at 3AM on Monday mornings. I also use hdparm to keep my mirror drive in a low power state. These are some good articles on that:
https://www.howtogeek.com/135533/how-to ... -on-linux/
http://man7.org/linux/man-pages/man8/hdparm.8.html

And here is another that is the basis on how i started my home NAS project when I first discovered the Raspberry Pi and abandoned my freeNAS project I had started with an old tower computer. This article has been updated, but i've been running a home NAS for at least 4 years now based on the raspberry pi platform.
https://www.howtogeek.com/139433/how-to ... ge-device/

Since i fell down this rabbit hole i've since moved onto running nextcloud and plex media server on a rock64 1gb board(did this before the Pi4 was introduced) for my NAS solution instead of my Raspberry PI and have retasked my raspi to run pihole and my pivpn server. Hope this helps and good luck.

[ejolson]

instead of using it as a RAID1 unit, simple redundancy or "timed mirroring" is implemented

So I have 2 SSD's connected to the USB 3.0 ports and OMV installed and it won't even allow you configure any RAID with USB (it says it's too unreliable). Can you tell me how you do your "timed mirroring"?

Not sure what he considers "timed mirroring" to be, but I"m using a cronjob to run rsync to mirror my main NAS drive to another connected drive once a week at 3AM on Monday mornings. I also use hdparm to keep my mirror drive in a low power state. These are some good articles on that:
https://www.howtogeek.com/135533/how-to ... -on-linux/
http://man7.org/linux/man-pages/man8/hdparm.8.html

And here is another that is the basis on how i started my home NAS project when I first discovered the Raspberry Pi and abandoned my freeNAS project I had started with an old tower computer. This article has been updated, but i've been running a home NAS for at least 4 years now based on the raspberry pi platform.
https://www.howtogeek.com/139433/how-to ... ge-device/

Since i fell down this rabbit hole i've since moved onto running nextcloud and plex media server on a rock64 1gb board(did this before the Pi4 was introduced) for my NAS solution instead of my Raspberry PI and have retasked my raspi to run pihole and my pivpn server. Hope this helps and good luck.

With a reasonable backup system there should always be at least two copies of every file at any time.

Mirroring with only one backup drive is difficult to get right because typically only one version of a file exists while the old copy is getting overwritten by the new. Moreover, it is surprisingly likely that the source drive will fail during the mirroring operation.

Most of these problems can be mitigated using a copy-on-write filesystem or a thinly-provisioned LVM block device with a suitable rotation of snapshots. A description of such a backup technique using an external BTRFS formatted hard disk is in the thread

https://www.raspberrypi.org/forums/view ... p?t=219525

[Pimaxin]

instead of using it as a RAID1 unit, simple redundancy or "timed mirroring" is implemented

So I have 2 SSD's connected to the USB 3.0 ports and OMV installed and it won't even allow you configure any RAID with USB (it says it's too unreliable). Can you tell me how you do your "timed mirroring"?

Not sure what he considers "timed mirroring" to be, but I"m using a cronjob to run rsync to mirror my main NAS drive to another connected drive once a week at 3AM on Monday mornings. I also use hdparm to keep my mirror drive in a low power state. These are some good articles on that:
https://www.howtogeek.com/135533/how-to ... -on-linux/
http://man7.org/linux/man-pages/man8/hdparm.8.html

And here is another that is the basis on how i started my home NAS project when I first discovered the Raspberry Pi and abandoned my freeNAS project I had started with an old tower computer. This article has been updated, but i've been running a home NAS for at least 4 years now based on the raspberry pi platform.
https://www.howtogeek.com/139433/how-to ... ge-device/

Since i fell down this rabbit hole i've since moved onto running nextcloud and plex media server on a rock64 1gb board(did this before the Pi4 was introduced) for my NAS solution instead of my Raspberry PI and have retasked my raspi to run pihole and my pivpn server. Hope this helps and good luck.

I'm thinking of using rsync instead of RAID now. Any reason why only once a week instead of daily?

[LTolledo]

I'm thinking of using rsync instead of RAID now. Any reason why only once a week instead of daily?

I think that's his preference
you can do it daily
or every 12hours, or every 4hours
or whatever timing you think suits your requirements...
you have full control over what you want do....

....so you have decided on the enclosure for your project?

[ejolson]

I'm thinking of using rsync instead of RAID now. Any reason why only once a week instead of daily?

I think that's his preference
you can do it daily
or every 12hours, or every 4hours
or whatever timing you think suits your requirements...
you have full control over what you want do....

....so you have decided on the enclosure for your project?

As I said earlier in this thread, please make sure you understand what will happen if rsync experiences a read error during backup.

https://www.raspberrypi.org/forums/view ... 0#p1550161

It can also happen that a read error occurs before backup, knocks the volume offline, rsync then sees the empty mount point and decides to delete all files in the backup to match.

Here is an thread about data loss worth reading while setting up any automated backup system.

https://www.raspberrypi.org/forums/view ... 2&t=244938

[default_user8]

I'm thinking of using rsync instead of RAID now. Any reason why only once a week instead of daily?

I think that's his preference
you can do it daily
or every 12hours, or every 4hours
or whatever timing you think suits your requirements...
you have full control over what you want do....

....so you have decided on the enclosure for your project?

As I said earlier in this thread, please make sure you understand what will happen if rsync experiences a read error during backup.

https://www.raspberrypi.org/forums/view ... 0#p1550161

It can also happen that a read error occurs before backup, knocks the volume offline, rsync then sees the empty mount point and decides to delete all files in the backup to match.

Here is an thread about data loss worth reading while setting up any automated backup system.

https://www.raspberrypi.org/forums/view ... 2&t=244938

I completely understand your warnings about using rsync, It would be a huge deal to lose all your data. It's my understanding that they way i'm using it, rsync compares the two drives and is only deleting files in the back up directories that no longer exist in the main directories, and that it's only altering files in the backup that were altered on the main drive. I'm absolutely no expert here, but i've been doing this for at least 4 years and never had an issue.

[Pimaxin]

I'm thinking of using rsync instead of RAID now. Any reason why only once a week instead of daily?

I think that's his preference
you can do it daily
or every 12hours, or every 4hours
or whatever timing you think suits your requirements...
you have full control over what you want do....

....so you have decided on the enclosure for your project?

Yes, here it is. 2x 2TB SSD

[default_user8]

That is very nice, did you print that yourself? I made my stack out of a sheet of lexan, but your's is much nicer than what I made. This is an old picture of mine, I don't use a hub anymore, or a raspberry pi for that matter. I switched to a rock64 1gb board about 6 months ago.
Since that picture was taken I had even started using an old 80gb hard drive instead of and sd card so it had even more cables than what's in that picture. But like i said about 6 months ago i switched to a rock64 board and went back to booting from and sd card so i don't need a hub anymore.

raspistack.jpg (60.06 KiB) Viewed 11193 times

[LTolledo]

I'm thinking of using rsync instead of RAID now. Any reason why only once a week instead of daily?

I think that's his preference
you can do it daily
or every 12hours, or every 4hours
or whatever timing you think suits your requirements...
you have full control over what you want do....

....so you have decided on the enclosure for your project?

Yes, here it is. 2x 2TB SSD

nice one you have there!
great looking!

I do suggest you cover the unused GPIO pins, like in the pic below
to prevent those accidental shorts to the GPIO pins...

GPIO cover example.jpg (123.21 KiB) Viewed 11169 times

this is on my Rock64-1GB NAS with 2.5" 4TB WesternDigital HDD (on USB3.0) and 2.5" 1TB HDD (on USB2.0)
same thing I did with the RPi2B NAS seen here

RPi2B_NAS_v3.0.jpg (198.74 KiB) Viewed 11169 times

also with 2.5" 4TB Seagate HDD

fan power is always taken directly from power-in port (not from GPIO header)

[Pimaxin]

That is very nice, did you print that yourself? I made my stack out of a sheet of lexan, but your's is much nicer than what I made. This is an old picture of mine, I don't use a hub anymore, or a raspberry pi for that matter. I switched to a rock64 1gb board about 6 months ago.
Since that picture was taken I had even started using an old 80gb hard drive instead of and sd card so it had even more cables than what's in that picture. But like i said about 6 months ago i switched to a rock64 board and went back to booting from and sd card so i don't need a hub anymore.raspistack.jpg

I downloaded the 3d printer files from here https://www.thingiverse.com/thing:3850735 and sent it to a 3d printing company to print for around $20. The company I sent it to didn't do that great of a job printing as the texture isn't smooth and there were a few defects near the screw holes. Had to drill a bigger hole to fit some screws as well but it works great and doesn't look too bad.

[Pimaxin]

I do suggest you cover the unused GPIO pins, like in the pic below
to prevent those accidental shorts to the GPIO pins...

fan power is always taken directly from power-in port (not from GPIO header)

Where did you get the GPIO pin cover? Can I punch a hole for the fan power? How do you power fan via the power-in port?

[ejolson]

I do suggest you cover the unused GPIO pins, like in the pic below
to prevent those accidental shorts to the GPIO pins...

fan power is always taken directly from power-in port (not from GPIO header)

Where did you get the GPIO pin cover? Can I punch a hole for the fan power? How do you power fan via the power-in port?

I like the way it currently looks with the gold plated pins of the GPIO glistening in the sunlight.

Chances of the pins suddenly shorting out is guaranteed if you let a three-year-old with a paperclip play nearby, otherwise the probability is zero.

[LTolledo]

Where did you get the GPIO pin cover? Can I punch a hole for the fan power? How do you power fan via the power-in port?

its a simple 40-pin header with the pins plucked out... leaving only the ones on the diagonal end and cutting off the excess.
on my other GPIO covers... I plucked out the pins on all non-GND points... and cut off the excess so that its level to the plastic.

40-pin header.jpg (162.95 KiB) Viewed 11105 times

power for the fans are taken on the board side
below is an example taken from my RPi3B+ (was not able to take pictures on the RPi4B version)

RPi3B+FanPower.jpg (195.21 KiB) Viewed 11105 times

why I do this? I usually have some 40-pin HAT installed on the RPi's GPIO and I also need a power for the fan. This is the solution I found.
if not confident about soldering skills, and no plan to install 40-pin HAT then trimming the "GPIO cover" above to accommodate the fan power maybe a probable solution....

[Pimaxin]

For those that have both Pi-hole & OMV installed, should OMV be installed first or Pi-hole?

[default_user8]

I'm running pivpn and pihole on my raspberry pi, since both are installed from scripts I started with the minimal raspbian install. I don't think it really matters which one you do first.