How to protect the filesystem without a save shutdown button?

[TrevorF]

rurwin wrote:

What about on a model A? No USB storage, no network.

USB network adapter, wireless if you have to. Programming is going increasingly network heavy so for the higher levels classes a Model B or a Model A with a USB-hosted network adapter (at least occasionally connected) would likely be where the *systems* programming classes would need to go.

Though programming education in the parts of the world who cannot afford networks (and the people/skills to maintain them) this is a challenge.

*At least* get the students to store their stuff on their own pen drives so that the Pi always starts (or a backup card is used and the corrupt card is "recycled" into the backup store) ...?

The corruption of the file system on an unmanaged restart of the OS might not be the only reason why the OS file system could fail.

[BlueClogger]

Marcus V. said:


So, this board needs a shutdown button, that's a simple, cheap thing to attach to one of the GPIO pins. Should need a bit of pressure to engage. Will cause interrupt and then "shutdown -h now" or whatever you tell it to do. After you get some feedback (LED?), you can safely pull the plug. (Could even restart the system if you press again, getting you out of some system locks in a safe way).

This should be fixed, looks like a flaw in the design.

Marcus



Not really a flaw if you read the above comments.  As with so many posts the simple answer to this is cost.  Not everyone will need a physical switch, so why charge everyone for putting one on.  The RPi as supplied will be the minimum configuration, and software and hardware solutions to issues like this will rapidly emerge.  After all, one of the key aims of the project is to get folk to develop solutions.

[jhhudso]

I believe there is not much risk of corrupting the Raspberry Pi's root filesystem by abruptly unplugging it. Most modern Linux distribution use the ext3 or ext4 journaling filesystem. Unlike a non-journaling filesystem (ext2, FAT16, FAT32), a journaling filesystem (NTFS, ext3, ext4, reiserfs) saves metadata that makes a filesystem consistent to a journal on the filesystem before a write system call is considered complete. This maintains the consistency of the filesystem even when the system using it is unplugged. When the system is turned back on the filesystem looks for this journal and "replays" the unsaved portion of the journal log. This keeps the filesystem metadata consistent. Now it is still possible for the contents of a file to be corrupt if the file had been saved but the filesystem not flushed from memory to disk before the system was abruptly turned off. There are ways to mitigate even this though. You can mount an ext3/ext4 filesystem with the option data=journal (the default is data=ordered) which causes the filesystem to save not just metadata but also the contents of files to the filesystem journal before a write system call is considered complete. This improves reliability at the expense of some performance. For more details about these options read the ext3/ext4 section of the mount(8) man page or /usr/src/linux/Documentation/filesystems/ext4.txt http://git.kernel.org/?p=linux.....fa;hb=HEAD

[bradburts]

JamesH said:


Well, I haven't trashed my SD card on my alpha board, and I was pressing the power button all the time on that at fairly random intervals. I don;t think I have ever issued the 'shutdown' command. I think the problem is being massively overstated.

Backup your work just on the off chance would be the correct action I think.



Agreed.

It does depend on your setup though. For example run an Apache webserver & every few seconds you will get a log for each connected browser. Even so the write will be open for a few mS every 10 seconds which gives an assumed risk of < 1:3,000. That's  the pessimistic worse case; not every write will not cause the FTL functions leaving most write interruptions protected by ext4.

As I said I created a setup which would switch a system off every 30 seconds or so whilst the system was logging. Could run for weeks without issue which suggested that the critical section was much less than a few mS for every write.

The test was run with new 'fresh' cards so I suppose a card which has been used for a while may behave differently.

[Canuck]

It would suck if unplugging killed my ReiserFS external HDD, I have a couple so I hope it won't be choked by the USB ports bandwidth and power limitations.

[error404]

The quality of ResierFS notwithstanding, physicals drives are immune to the issues we're discussing here since there is no further abstraction table that needs to be updated, the filesystem is in complete control, and it is possible to guard against incomplete writes.

I don't really see the big deal either. Just shut down the system by software, as is normal practice for most people anyway. If possible in your environment I would strongly suggest using network mounted home directories for user files and running the system partitions read-only, leaving some space for temporary files and whatnot on the SD card that is wiped at each boot.

[Canuck]

I heard it could cause mass corruption if the system is doing something disk intensive, then quickly looses power with ReiserFS.

[error404]

Canuck said:


I heard it could cause mass corruption if the system is doing something disk intensive, then quickly looses power with ReiserFS.



I've seen Reiser fail spectacularly in this kind of situation as well, but this is due to the quality of Reiser, not a failing of the protection mechanism. If integrity is important to you, use ext3/4.

[Steady_Bear]

slacer said:


What happens if a student simply switch of the device by pulling the power plug?

I guess the file system will be corrupt at the next start, or after the next student wants to leave this classroom in order to meet his friends between lessons?

... ...

Maybe it was too expensive to add a way to shutdown a raspberry without the need to logon as root first, but maintaing filesystems during lessons might upset some teachers and students.



What happens if a student does this with the current PC? (there are some ways of making this likely - but I'll not give ideas).

You don't need to log in as root. "sudo /sbin/halt" with a properly set up sudo

On the theme of SD - would the FTL (BTW I've been watching too much BSG - I assumed someone had finally sorted out write speed... ) error brick the card's hardware, or could you reformat?

[slacer]

I started this thread because I dislike the idea to simply switch a computer off.

The computer industry had a reason to develop a way to shutdown systems instead off switching it off.

Who is to blame if there is a problem? You guess it - the user!

And if there is a pattern for this problem? Yes, then it is the product!

If switching the device off is not a problem because SD cards don't have this problem

and a journaling filesystem is smart enough to keep a mysql database in a valid state all the time, then everything is great.

But if it is not OK to switch it off - what can we do to make this device robust enough for daily use at school?

Yes, it is important to train teachers and students involved.

And I have seen massive amount of effort to create a way to bring computers in a pc classroom into a valid state automatically after each lesson. This was a school for adults.

Am I too pessimistic?

[rurwin]

Children, and adults, are not what they were in "our day". You could switch off a Commodore 64 or a BBC Micro, even a PC running DOS without a second thought. Nowadays everyone knows that you have to shutdown first. My guess is that it will not be much of a problem, and even if it does bite the odd person from time to time, it will just teach them to keep backups.

There are things you can do to ensure a safe shutdown, those 1 farad super-capacitors may store enough charge to keep the machine up long enough to sync the filesystem if one was really concerned. Such a gadget might make a nice electronics project for a class.

Otherwise all we need to do is to ensure that the issue and the shutdown procedure is clearly documented.

[error404]

I guess this raises a question - does the Pi include a software-controlled power switch, enabling a proper self-shutdown? I'm going to assume no. This is a bit disappointing because it requires a two-step shutdown and precludes an automatic shutdown of e.g. a lab. Since we're talking about relatively large-scale use here, it wouldn't be hard to design an addon that takes care of this issue, providing a way for the system to turn itself off and a button to turn it back on.

@slacer: The correct solution is not to store any (important) state on the local terminal anyway, and this has significant other benefits as well in the classroom environment. Store user files on the network or on removable storage, don't provide write access to the SD card, problem solved.

[rurwin]

"shutdown -h now"

then wait for the message telling you it is OK to switch off. That's the way we were all doing it (even in Windows) in 1995 before the smart switches came in and the PC could switch itself off.

[Canuck]

Did nobody get my pun? I think I'm trying to hard.

[slacer]

error404 said:


@slacer: The correct solution is not to store any (important) state on the local terminal anyway, and this has significant other benefits as well in the classroom environment. Store user files on the network or on removable storage, don't provide write access to the SD card, problem solved.


I assume each student picks a device out of a lockable cupboard, connects it to monitor, mouse, keyboard, usbstick and power. And is happy if he does not have to ask the teacher for an SD card with a valid OS.


don't provide write access to the SD card, problem solved.


You can not avoid write access to the filesystem unless directories like /var /proc /tmp are mounted into a ramdisk or over the network (Model B as a kind of semi diskless client)

[Steady_Bear]

Canuck said:


Did nobody get my pun? I think I'm trying to hard.



No, and yes. Nothing to do with Hans himself was it?

I suspect the system will shutdown exactly the same way all modern PCs do. Issue the shutdown command (the button on front only tells the software to initiate shutdown - unless held in), software makes itself nice and neat and safe, then the hardware goes into standby. It's certainly not off.

If not, then just type sync. Jobs a good'un.

[error404]

slacer said:


I assume each student picks a device out of a lockable cupboard, connects it to monitor, mouse, keyboard, usbstick and power. And is happy if he does not have to ask the teacher for an SD card with a valid OS.


Personally I would think that in a lab environment a static setup makes a lot more sense. If nothing else, because the connectors aren't really designed to be manipulated 4-6 times a day and remain reliable, you'll be constantly replacing cables and repairing/replacing broken power connectors on the Pis, less chance of accidental damage to the Pi moving it around etc. It also makes it much easier to do mass updates and so on if the machines are always hooked up and ready to go.

If you do want to use it in this environment, ask the students to shut the machines down before putting them away. I don't think this is really a big deal, most people, especially young people, understand the need to do this. I'm not sure why a hardware button is preferable to a software one; you're still going to have to make sure they wait for the shutdown process to complete before removing power, which would seem to me the far more difficult thing to accomplish.


You can not avoid write access to the filesystem unless directories like /var /proc /tmp are mounted into a ramdisk or over the network (Model B as a kind of semi diskless client)


Mount all the system partitions read-only, and temporary files can live on an SD card partition that is wiped at each boot.

[slacer]

There are some ideas for possible solutions mentioned in this thread and time will show if it is a real problem at all.

Thank you all

Michael

[Bakul Shah]

My understanding of this issue:


As long as you issue a shutdown command and *wait* until there is positive indication it is safe to shutdown, there is no problem.
If you just pull out the flashcard *while* it is being written to, there is a risk.
If there is a power fail or you unplug the power cord, and the flashcard is being written to, there is a risk.

We don't have any hard data on how small the risk is but it is small. Users should be warned of the risk and how to deal with it. There are various ways you can make it smaller:


Reduce writes where possible. This is a good idea in any case.
Use an "external battery pack" for iPad or some such ($20 or more), to guard against power loss. Some of these have a 5v input and one or two 5v outputs and are a few thousand mAh. Attach this pack to the Raspi to avoid accidental disconnect.

[jwatte]

ukscone said:

I was thinking something like a minimal initramfs concatted (compiled into the kernel) to ensure the ability to boot into a shell, pivot_root to the full

Won't help. Partitions are invisible to the low-level flash controller on the SD card. The problem here is that the SD card has a logical->physical mapping table that is entirelyinvisible to the kernel of the operating system (or any other outside device) and that mapping table is generally not power failure safe. The only way to solve this problem is to provide some amount of time (10ms?) of guaranteed power after the last write transaction has been issued.

In fact, a sufficiently fast "power off" signal from the card might cause the same problem even when doing an orderly shut-down.

[Docteh]

Just stopping by to say that /proc is a separate file system, unrelated to whether or not other file systems are marked read only or not.

[Rubus]

rurwin said:


"shutdown -h now"

then wait for the message telling you it is OK to switch off. That's the way we were all doing it (even in Windows) in 1995 before the smart switches came in and the PC could switch itself off.




That brings back memories!

[Robert_M]

I plan to hire Mr. T to stand by and say, "I pity da fool who neglects a safe shutdown!"

That oughta do the trick!

Math: One plus Seven equals Oney-Seven.

[hyena]

hi,

Many of use that have hacked the plug computers (and liked to swich them off at night)face the same problem .. first problem being people who say dont switch them off .. its particularly a pain if its being used as a server and you have to log on using ssh to issue the shutdown command to shut the thing down.

From my experience of having the plug pulled frequently debian CLI on the plug seems to recover very well, and i havnt had a corrupt os yet .. the applications that were running at the time though invariably end up corrupted and need a reinstall ..

I use now a little windows phone 7 ssh app and just logon over wifi and issue the shutdown command .. someone could probably write a simple two button app for the major platforms just to logon to a running ssh server on the pi (very little overhead) to reboot and shutdown.

I guess one of the first community i/o devices made for the pi could be a little switch and led to power it down and show status

[rrolsbe]

When/if the Puppy Linux developers get it working on the Pi, I hope to run the SDHC card write-locked most of the time.  Let me explain:  I currently boot most of the time into x86 Puppy Linux on my Netbook with the SDHC card write locked and with my Kanguru USB stick write locked on my notebook.  This serves two purposes, even if I did pick up some nasty while browsing/reading Web based Gmail, nothing persistent can be written to any files on these flash devices and I can just turn them off with their power buttons (handy when browsing on a non-encrypted wifi and when traveling by air).  PS-- Before anyone brings it up, I am aware that the write lock on an SDHC card is not a true hardware write-lock as it is on the USB Kanguru stick.  I need to trust SDCH reader is correctly honoring the SDHC write lock tab (which it appears to be doing).  The only time I unlock either of these devices is when I initially create the bootable flash media or when I want to modify something and have it available for future use.