Files gone from an external HDD

[David-CZ]

I bought Raspberry Pi 3 model B along with a WD Duo external drive as a back-up system.

When I was copying some data on the WD drive, an error about disconnection popped up. After I ejected the drive (it was still visible among the devices), I replugged it, but the folder that previously already had some files in it is now completely empty. Both the newly added files along with everything else is gone.

The WD Duo is a two 4TB exfat drives in RAID1.

The folder already had over 100GB of data in it. There's no point in having a back up system where I risk losing everything simply by interrupting a copying process.

[MaxK1]

You need to report errors/prolems as accurately and completly as possible.
A pi 4 - Really? Where did you get that?
"... an error about disconnection popped up." Exact messages please!

In addition, we need to know what O/S and distro you are using as well as how the malfunctioning H/W is connected and what else is connected.

[David-CZ]

Meant version 3 model B. Don't know why I wrote 4. Changed it in OP.

If I had the message I'd post it here. But I'm not going to try replicating the error and losing more data.

I'm running Raspbian from the latest NOOBS package. The WD Duo is connected via a USB cable. Apart from that I have a wireless Microsoft keyboard+touchpad connected, again via USB, and a TV via HDMI.

[MaxK1]

I can only speculate on possible causes...
There is a problem with the drive.
There was some kind of USB problem with the keyboard dongle that caused a USB reset which the drive did not recover from.
Bad cable? Bad power?
The fact that existing files went away points to drive itself having a problem...

[RaTTuS]

you will be able to recover the files as they are exfat file system - stuff it in a windows machien and get it to rescan
or use testdisk on a linux system

N.B. exfat is rubbish as a backup file system

backup via rsync over Ethernet

[drgeoff]

Why are you using exFAT on hard drives?

[stderr]

The folder already had over 100GB of data in it. There's no point in having a back up system where I risk losing everything simply by interrupting a copying process.

That's like saying that it's no use avoiding transfats because everyone eventually dies of something. If you only have the one backup, you always are at some risk when you are doing a new backup to that disk since you've got to plug it in and spin it up and then write to it. Anything could happen.

Backup family photos and your revisions to your Great American Novel onto robust media like non-erasable CDs and DVDs. Especially with regards to those things that don't change, make two copies and verify they can be read on another drive, store in different locations.

No single hard drive to back things up will be a safe solution because what happens if you have a house break in and the bad guys steal your computers? Or if your main machine is compromised and you are running the backup on that main machine, you will likely have to give that main machine access to the backup network drive. Even if it is running linux, there is no reason why the compromised machine can't encrypt the data remotely. It's better to have the remote machine access your main machine and backup that way, then it would have to be compromised to get caught out.

I'm sure there will be a lot of arguments about what to do and what not to do but certainly the answer isn't to conclude that backing up is pointless. Put the really, really important stuff on offline known-good write-once media and you'll have greater peace of mind.

[peterlite]

A disappeared directory suggests the file system was updating the directory list when the error occurred. ExFAT is a light weight file system with no protection against that type of error. Ext4 and NTFS use journaling as a protection against that type of error. Major changes are recorded in a journal first then applied. If the change fails, a restart can recover from the journal. NTFS gives you compatibility with Windows and efficient storage of very small files. Everything else is the same between NTFS and Ext4.

When you add RAID on top, there are some problems related to RAID. The old NT RAID1 always worked. You could pop out either disk and use the disk in another computer. Every other version of RAID introduces problems that complicate shutdown and recovery. Use the file system recommended for your RAID and follow the configuration instructions then test the result. Switch off a disk while the system is running then switch it on to see if recovery works.