Coburn
Posts: 78
Joined: Tue Sep 06, 2011 2:57 am
Contact: Website

Re: Downloadable SD Card Images

Fri Oct 21, 2011 2:04 am

The thing is, if the SD Card write protection notch is down, then well, Linux will cry because it can't write anything to the SD Card as it's read-only. Although, if you dd directly (forcing a bypass of the switch) to the SD Card device, let's say /dev/sdcard0, you can write anything on it.

What I do on Plug Computers is this:

Before first boot - Plug the SD Card into a Linux PC. Edit the SD Card Linux /etc/fstab file something like:

none tmpfs /var/log defaults,size=16M 0 0


Repeat for other temporary caches that linux might use (/var/run, etc). That way, what happens is that linux boots, it'll write all the boot logs to a RAM Drive and save a few dozen write cycles on the SD Card itself. When the Pi is switched off, the dmesg logs etc are cleared. :)

Another method to do that but make tmpfs areas for all the parts where Linux needs to write to (/var/log, etc) and then mount the root partition read-only. That way, Linux can still read/write, but changes will only be made in memory.

When I get a Raspberry Pi for my tablet PC project, I'll most likely use a big loopback file called userdata.img where I'll mount /home/user on it. If the user botches the image file up, no worries, I'll have a recovery process in there that will recreate a fresh image file.

kme
Posts: 448
Joined: Sun Sep 04, 2011 9:37 am

Re: Downloadable SD Card Images

Fri Oct 21, 2011 2:17 am

Quote from abishur on October 20, 2011, 23:58
I had thought about putting a live CD on the SD card too, but then I thought about the fact that it stores a lot of stuff in RAM, with on 256 MB, do you really want to bog things down by losing a portion of it?Well, if I may jump to the x86 world I'm deeply involved in a thin client project called thinstation. It boots completely diskless in 32 MB (probably 24 MB) RAM with a simple desktop and a few simple tools. No compilers, nothing fancy - just X and a Linux 2.6 kernel.

There is absolutely no reason why Linux would want to write to the boot partition what so ever. Or any disk for that matter. That's another Microsoft illness some people has got into their head as "normal".

SlayingDragons
Posts: 67
Joined: Wed Sep 14, 2011 3:17 am

Re: Downloadable SD Card Images

Fri Oct 21, 2011 2:38 am

Quote from kme on October 21, 2011, 03:17
Quote from abishur on October 20, 2011, 23:58
I had thought about putting a live CD on the SD card too, but then I thought about the fact that it stores a lot of stuff in RAM, with on 256 MB, do you really want to bog things down by losing a portion of it?Well, if I may jump to the x86 world I'm deeply involved in a thin client project called thinstation. It boots completely diskless in 32 MB (probably 24 MB) RAM with a simple desktop and a few simple tools. No compilers, nothing fancy - just X and a Linux 2.6 kernel.

There is absolutely no reason why Linux would want to write to the boot partition what so ever. Or any disk for that matter. That's another Microsoft illness some people has got into their head as "normal".

This, there's no more memory used or speed sacrificed in booting from read-only as opposed to write enabled, unless you have so little ram that you typically have to use swap. The reason live cds and USB drives are slower than booting off an hdd is because the read speeds of those mediums are slower than an hdd, not because it's not allowed write access to the mediums... Where did people get that idea? 0_o

tufty
Posts: 1456
Joined: Sun Sep 11, 2011 2:32 pm

Re: Downloadable SD Card Images

Fri Oct 21, 2011 5:46 am

Quote from Bacan on October 20, 2011, 23:59
I know one client that will want his stuff write protected.
SD also has the concept of a write protection at the software level. By which I mean "defined at the software level, but enforced by the device". You can write protect a group of sectors, and *nothing* can overwrite them without explicitly un-write-protecting them first. Or you can temporarily / permanently write protect the entire device.

robhague
Posts: 12
Joined: Sun Sep 04, 2011 8:14 am
Contact: Website

Re: Downloadable SD Card Images

Fri Oct 21, 2011 7:05 am

I feel that the discussion has wandered somewhat from the point raised yesterday. Our hypothetical African/Inuit boy won't be using a thin client - the whole point is to give him a real, self-contained computer. This requires some method of storage that persists when then power is off (computers have been writing to permanent storage since before Bill Gates was born - I don't know how this could be a "Microsoft illness"). The aims of the RaspPi foundation would also (I think) be served by allowing kids to futz around with the operating system, which carries the risk of rendering it inoperable. Whilst there are uses for the Raspberry Pi where not having permanent storage has sense, the core purpose of providing a computer for teaching programming is not one of them. A recovery partition would allow the user to return a RaspPi to a default state without the need for any additional hardware.

tufty
Posts: 1456
Joined: Sun Sep 11, 2011 2:32 pm

Re: Downloadable SD Card Images

Fri Oct 21, 2011 7:47 am

Quote from robhague on October 21, 2011, 08:05
A recovery partition would allow the user to return a RaspPi to a default state without the need for any additional hardware.
It would, and that would be a highly desirable feature.

However, for it to be useful, it would probably require some soft of "repair boot" button (i.e. power on whilst holding down the button). Probably the easiest way to implement this would be via 3-stage booting, where the GPU fires up an OS booter that detects a power-up hotkey sequence (something similar to the Powermac cmd-opt-o-f "four finger salute" to get into open firmware). On detecting that, it would blow away the user partition and reinitialise, otherwise it would chain the Linux (or other) kernel.

The danger then is that the booter and kernel lie on an easily accessible (and, by necessity for kernel upgrades, writable) FAT32 formatted partition that can be easily buggerated by plugging the card into a PC. Dunno any simple way around that.

Simon

obarthelemy
Posts: 1399
Joined: Tue Aug 09, 2011 10:53 pm

Re: Downloadable SD Card Images

Fri Oct 21, 2011 8:13 am

Quote from kme on October 21, 2011, 03:17
Quote from abishur on October 20, 2011, 23:58
I had thought about putting a live CD on the SD card too, but then I thought about the fact that it stores a lot of stuff in RAM, with on 256 MB, do you really want to bog things down by losing a portion of it?Well, if I may jump to the x86 world I'm deeply involved in a thin client project called thinstation. It boots completely diskless in 32 MB (probably 24 MB) RAM with a simple desktop and a few simple tools. No compilers, nothing fancy - just X and a Linux 2.6 kernel.

There is absolutely no reason why Linux would want to write to the boot partition what so ever. Or any disk for that matter. That's another Microsoft illness some people has got into their head as "normal".

Well, by default Linux does want to write to the / partition (not the one with the bootloader and kernel, but the rootfs one), and the reason is probably logs. I can tests it write (pun !) now on my eCafe: put SD tab on "read-only": no boot.

obarthelemy
Posts: 1399
Joined: Tue Aug 09, 2011 10:53 pm

Re: Downloadable SD Card Images

Fri Oct 21, 2011 8:24 am

If you want to be safe, a separate partition for /home sounds like a good idea.

robhague
Posts: 12
Joined: Sun Sep 04, 2011 8:14 am
Contact: Website

Re: Downloadable SD Card Images

Fri Oct 21, 2011 8:43 am

A separate /home partition would certainly solve a lot of the problems, but my assumption is that kids will eventually get the the stage where they want to modify the system itself.

tufty
Posts: 1456
Joined: Sun Sep 11, 2011 2:32 pm

Re: Downloadable SD Card Images

Fri Oct 21, 2011 9:03 am

Quote from obarthelemy on October 21, 2011, 09:13
Well, by default Linux does want to write to the / partition (not the one with the bootloader and kernel, but the rootfs one), and the reason is probably logs. I can tests it write (pun !) now on my eCafe: put SD tab on "read-only": no boot.
A typical solution for this is to have /var as a separate partition mounted R/W, and (optionally) separate the log messages by severity, with anything below $THRESHOLD dumped to something mounted on tmpfs. Unfortunately, tmpfs is a ram-based file system, and we don't have a great deal of that, so...

For something like Pi, I'd probably configure syslogd to *only* log emergency, alert and critical messages, and dump the rest to /dev/null. A bit like this:

http://www.linuxquestions.org/.....ls-548469/

You could use logrotate to limit the size of log files, and then dump the lower levels to a small tmpfs; that would limit the memory overhead and still keep a bit of recent logging available.

simon

kolin23
Posts: 4
Joined: Tue Nov 01, 2011 7:35 pm

Re: Downloadable SD Card Images

Tue Nov 01, 2011 8:51 pm

Hi,
I have just this moment been thinking about 'DIY' mass duplication of SD cards for friends/community.
I have been thinking about getting;

x1 10 port USB hub
x10 SD cards
x10 USB card readers

In theory using my laptop's built in card reader (as master) I would be able to duplicate that card to the other 10, only issue is I have no idea if this idea will actually work with just copy n paste?

I suppose in the above issue from briefly looking online for hardware the 4gb 15mbps card and SD reader totals to about £5 less delivery so a 'in-case-of-card' wouldn't cost that much to do as long as you could just copy n paste the original image across using the RP.
Would it be possible to lock the SD card in a way that allows read/write but not format?
Take care friends

matt_heys
Posts: 46
Joined: Fri Oct 28, 2011 5:58 pm

Re: Downloadable SD Card Images

Tue Nov 01, 2011 9:25 pm

Quote from Burngate on October 20, 2011, 18:47
Thanx both of you: job done.
I'm glad I'm not the only one thinking down this road!
So RasPi: £15.
SD card: £7.50.
In-case-of SD card: £7.50.
Total £30. BOGO total £60


Why would you Buy One Give One on two SD Cards? R-Pi foundation would only have spare SD cards hanging around then.

I also hope you will be able to buy an R-Pi them without an SD card, environmentally speaking I have stacks of them and it would be more environmentally friendly to use an old ones then to buy new ones.

EDIT
Doh, the paging buttons underneath the quick reply has caught me out again, thought I was at the end of the thread, not replying to an old post.

gbl08ma
Posts: 36
Joined: Thu Oct 27, 2011 6:54 pm
Contact: Website

Re: Downloadable SD Card Images

Tue Nov 01, 2011 9:34 pm

kolin23, copy&pasting files will not properly clone the SD card contents - there are things like symbolic links, files with specific permissions and files that aren't really files (as in Unix everything's a file) e.g. everything behind /dev. Plus, IIRC the cards will have two partitions, one FAT (bootloader entry point) and a ext (main filesystem), and you can't copy those properly with a file browser either.

The best you can do to clone cards is using a 1:1 raw copy, using something like the dd command in Linux. Just make sure that you learn to use it properly or else you might accidentally dd your hard drive/pendrive/any drive with important data. If you're unsure about the use of dd even after you've read every document you could :) , you can always setup a Linux virtual machine (something that will come handy anyways if run Windows as your main OS) and dd from there, just don't mount any file systems other than the SD cards on the VM and you'll be fine even if you incorrectly type the device for dd to raw copy.

As for SD cards that allow normal usage within partitions but no changes in the partition table, or no formatting of partitions, I think that's only possible with very special SD cards that you must develop yourself (they would have to contain some sort of write protection that only allows writing in certain parts of memory), however, I'm not sure about this in any way, so please don't take my word for it.

kolin23
Posts: 4
Joined: Tue Nov 01, 2011 7:35 pm

Re: Downloadable SD Card Images

Tue Nov 01, 2011 10:43 pm

gbl08ma
Annoyingly just found out from a friend that it wouldn't work based on C&P, not tried DD before/yet but can try it out. Wasn't sure if there was a way to do the same process 1: X instead of 1:1 or a simple utility to mount onto multiple devices in one shot. Basically looking at away to do 10+ copies of bootable SD (1 active and 1 in-case-of) for friends/community without needing to look at bulk duplication companies and being lumbered with 1000 cards :/
How about 1 image to X device? would DD handle multiple outputs?
(sorry i'm a noob with linux, few months ago scrapped windows)

obarthelemy
Posts: 1399
Joined: Tue Aug 09, 2011 10:53 pm

Re: Downloadable SD Card Images

Tue Nov 01, 2011 11:39 pm

You can run multiple instances of dd concurrently, even off the same input "file". You can't have a single instance outputting to several devices at once though.
I've found dd horrendously slow though, winimage does the same graphically, in Windows, much faster.

asb
Forum Moderator
Forum Moderator
Posts: 853
Joined: Fri Sep 16, 2011 7:16 pm
Contact: Website

Re: Downloadable SD Card Images

Wed Nov 02, 2011 6:46 am

Quote from obarthelemy on November 1, 2011, 23:39
I've found dd horrendously slow though, winimage does the same graphically, in Windows, much faster.

I expect it's just a matter of default block size (only 512 bytes for dd).

kme
Posts: 448
Joined: Sun Sep 04, 2011 9:37 am

Re: Downloadable SD Card Images

Wed Nov 02, 2011 7:04 am

Quote from asb on November 2, 2011, 06:46
Quote from obarthelemy on November 1, 2011, 23:39
I've found dd horrendously slow though, winimage does the same graphically, in Windows, much faster.

I expect it's just a matter of default block size (only 512 bytes for dd).Yes the default is for floppy disks and makes little sense for modern media. However dd also faithfully copy empty space but that's by design.

tufty
Posts: 1456
Joined: Sun Sep 11, 2011 2:32 pm

Re: Downloadable SD Card Images

Wed Nov 02, 2011 8:47 am

As a side note, ddrescue is a particularly good replacement for dd - it does everything dd does, plus allows for the recovery of "failing" media (particularly those of the "spinny" type), as you can get it to do multiple passes or even stop it in the middle and restart where you left off (if, for example, you have a hard drive that overheats its controller card - I'm looking at you, Western digital - and needs to be unplugged and cooled down on a regular basis)

obarthelemy
Posts: 1399
Joined: Tue Aug 09, 2011 10:53 pm

Re: Downloadable SD Card Images

Wed Nov 02, 2011 9:58 am

It was not only block size, I tweaked that to be as high as RAM would allow (150M out of 512MB, I think), and dd still took more than an hour to backup 8GB from one SD to another.

AlanCox
Posts: 31
Joined: Thu Nov 10, 2011 7:11 pm

Re: Downloadable SD Card Images

Thu Jan 26, 2012 11:18 am

Pedantically speaking the dd default is for hard disk media – which is 512 byte sector until recently. When you are using it for things like rescuing material the fact it uses the real block size is important.

For more normal use dd bs=1M will be better.

SD is slow though, and performance is very card dependant. Some of the brand name products are actually the worst you can get. How you partition the device is also absolutely crucial. There are significant benefits in choice of file system.

See

https://wiki.linaro.org/WorkingGroups/K ... CardSurvey

Clair
Posts: 20
Joined: Mon Jan 02, 2012 11:01 am
Contact: Website

Re: Downloadable SD Card Images

Thu Jan 26, 2012 11:57 am

I've read these three pages and have  to ask for help! Since the SD card OS won't be available at the same time as the R-Pi itself . . .

Could there be, somewhere on the site, clear "dummy" instructions for

1. downloading the image file we need

2. instructions on how to get this onto an SD card so that R-Pi is ready to roll when we slip it into the slot.

I am so excited about this piece of hardware that I do not want to spend hours/days, trying to reinvent the wheel if someone already knows how to do it! (I've already gathered up the cables, the keyboard, mouse, and the power supply–all laid out ready for the "operation".)

Also extremely helpful would be recommendations for several named SD cards (best for speed, size,  and protection).

I, along with others I'm sure, are going nuts waiting for this gem — I want the SD card in hand along with the instructions when the parcel arrives in my mailbox!

Such help would be most welcome — that's actually what I thought I was going to find when I opened this thread.

User avatar
RaTTuS
Posts: 10458
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: Downloadable SD Card Images

Thu Jan 26, 2012 12:16 pm

obarthelemy said:


It was not only block size, I tweaked that to be as high as RAM would allow (150M out of 512MB, I think), and dd still took more than an hour to backup 8GB from one SD to another.


dd bs=1M will be about optimal [YMMV]
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

bredman
Posts: 1415
Joined: Tue Jan 17, 2012 2:38 pm

Re: Downloadable SD Card Images

Thu Jan 26, 2012 1:03 pm

Clair said:


Could there be, somewhere on the site, clear "dummy" instructions for

1. downloading the image file we need

2. instructions on how to get this onto an SD card so that R-Pi is ready to roll when we slip it into the slot.


There are other threads (example http://www.raspberrypi.org/for.....ation-tool) where fantastic work is being done. The focus of this work is to provide an SD card tool for new users.

The current thread has wandered into some technical discussions which are of no relevance to new users. Just sit back and soak it up.

macemoneta
Posts: 59
Joined: Sun Oct 23, 2011 12:47 pm
Contact: Website

Re: Downloadable SD Card Images

Thu Jan 26, 2012 1:10 pm

obarthelemy said:


It was not only block size, I tweaked that to be as high as RAM would allow (150M out of 512MB, I think), and dd still took more than an hour to backup 8GB from one SD to another.


As others have said 1M is about optimal for SD.  Also, if your Linux kernel suffers from the dirty pages problem, you want to add the oflag=dsync option to the dd command.  For example:

dd if=some.img of=/dev/sdX bs=1M oflag=dsync



The oflag=dsync option forces the data out every blocksize, which prevents the kernel from getting into a thrashing situation where writes slow down to a trickle (which can even freeze the entire user interface).  This has been corrected in more recent kernels.

User avatar
ArborealSeer
Posts: 300
Joined: Tue Jan 24, 2012 9:48 am
Location: South West, UK

Re: Downloadable SD Card Images

Thu Jan 26, 2012 8:10 pm

Quick observation.. If distro's are provided as card images won't they have to be in 2 gig, 4 gig and 8 gig sizes.. and so on.. ?

As I think a 2 gig image won't work on a 4 gig card? (well may still act as though its 2 gig)
Pi Status > Farnell, Arrived 24/5- RS, Arrived 1/6

Return to “General discussion”