User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Wed Aug 21, 2019 12:59 pm

This guide describes the installation of vanilla Debian 10.0 (or later) using the official ARM64 ISO, on a Raspberry Model 3B or 3B+.

Vanilla here means that we are going to be installing straight from an untouched ARM64 Debian netinst ISO, with no need for custom kernel modifications or any gymnastics of the original content before installation. In other words, by following this guide, you will get as close to the same experience as the one you have when installing Debian from an x86 ISO on an UEFI PC. You will also end up with everything you'd expect to see from a PC install, including a graphical GRUB prompt as well as a system that'll natively update its kernel and initrd, without the need for further configuration...

ImageActual screenshot of the standard GRUB prompt you'll see after install — Doesn't get any more vanilla than this

Opinionated preamble (skip this part if you're only interested in the guide)

The reason we are able to perform this above is because we finally have a full UEFI firmware for the 3B/3B+, and because the Debian ISOs, be it for x86 or ARM64, are pretty much designed to work right out of the box with UEFI.

At this stage, I feel the need to voice my opinion as to why people who have been interested in getting vanilla distros such as Debian (or Ubuntu, or Suse, or whatever, since it makes things equally easy for those) as well as the Raspberry Pi Foundation haven't been helping with the effort to get a UEFI firmware going, because that is really the one piece of the puzzle anyone interested in running a generic ARM/ARM64 distro should have been concentrating on, instead of trying to alter anything and everything on a case by case basis, to get it to run...

As valid as some of the criticism it receives might be, there is a good reason UEFI exists in the first place, which a lot of people don't seem to appreciate, and that is to ensure that any OS can rely on a unified boot environment to set everything it needs moving forward.

And as this guide hopefully demonstrates, once your platform supports UEFI, everything else becomes incredibly easy!

Plus it also ensures that users can find a familiar environment during boot, which they can control through a User Interface and which can indeed be critical for a platform such as the Pi, which is designed to be an affordable entry point for people of all horizons...

But let's get to the guide, which is mostly a mashup of the already existing succinct guide we have in the Systems readme for the Raspberry Pi 3 UEFI firmware as well as my more detailed blog entry .

Prerequisites

Hardware

  • One micro SD card with sufficient space (at least 16 GB or more).
  • A Raspberry Pi 3 Model B or Model B+. Earlier or later models are not supported yet, since there exists no UEFI firmware for those at the moment.
Note that our goal here is to install the system on a SD card, through netinstall, using a single media for the whole installation process. That same media will also be the target for our installation. In other words, there is no need to use an additional SD or USB media.

Software

Note that if you pick the firmware binary archive linked above, all of the files you need, including the WLAN drivers, are already provided, so you don't need to hunt for them.

Preparation

  • Partition the SD media as MBR and create a ~300 MB partition on it with MBR type 0x0e.
    Note: Make sure that the partition scheme is MBR (not GPT) and the type 0x0e (not 0xef for instance), as the on-CPU Broadcom bootloader supports neither the GPT scheme nor the ESP MBR type.
  • Set the partition as active/bootable. This is needed as the Debian partition manager can not detect it as ESP otherwise, which we need for GRUB installation. If using fdisk on Linux, you can use the a command to set a partition as active. On Windows, you can use diskpart and then type active after selecting the relevant disk and partition.
  • Format the partition as FAT16 (again needed for Debian partition manager to detect it as ESP). If you are using Windows diskpart then format fs=fat quick will format a drive to FAT16. On Linux, the equivalent command would be mkfs.vfat -F 16 /dev/<your_device>. If you need help with the partitioning and formatting of your device, you can find full detailed steps in the Appendixes of the blog entry I linked above.
  • Copy all the UEFI firmware files, including any optional files such as the Wifi firmware drivers, onto the FAT partition.
  • Extract the full content of the ISO onto the FAT partition.
  • Eject your SD card and insert it into your Raspberry Pi 3

Installation

Note: A lot of the steps described below are the same you'd expect if you were installing Debian on a PC.

  • Power up the Pi. It should show the multicoloured screen (indicates that the UEFI firmware is being loaded from the SD card) and then the black and white Raspberry Pi logo (indicates that the UEFI firmware is running)
  • (Optional) Provided that you have proper cooling, it is strongly recommended that you go to the UEFI firmware setup by pressing the <Esc> key once you see the Raspberry logo and then go to Device ManagerRaspberry Pi ConfigurationChipset Configuration and set CPU Clock to Max, as the default for the firmware is to limit the CPU clock to 600 MHz which is slow. Or you can also set your own custom CPU Frequency from the same menu.
  • On the GRUB menu select Install and let the Debian Installer process start. Note: In case anything goes wrong during the install process, you can use <Alt>-<F4> to check the installation log.
  • Select your Language, Country and Keyboard and let the installer proceed until it reports that No Common CD-ROM drive was detected.
  • On Load CD-ROM drivers from removable media select No.
  • On Manually select a CD-ROM module and device select Yes.
  • On Module needed for accessing the CD-ROM select none.
  • On Device file for accessing the CD-ROM type the following exactly:

    Code: Select all

    -t vfat -o rw /dev/mmcblk0p1
    
  • (Optional) If you have the non-free WLAN firmware binaries, and plan to install through wireless, you can let the installer select the firmware files. Please be mindful that you may be asked multiple times as there are multiple files to provide.
  • If requested by the installer, set up your network by choosing the network interface you want to use for installation and (optionally) your access point and credentials.
  • Go through the hostname, user/password set up and customize those as you see fit.
  • Let the installer continue until you get to the Partition disks screen. There, for Partitioning method select Manual. You should see something like this:

    Code: Select all

    MMC/SD card #1 (mmcblk0) - 16.0 GB SD 2WCGO
         #1  primary  314.6 MB  B  K  ESP
             pri/log                  FREE SPACE
    
    In other words, the partition manager should already detect your existing partition as ESP, with the B (bootable) and K (keep data) flags. If that is not the case, (e.g. if it says fat16 or fat32 instead of ESP) then it probably means you either didn't format the partition to FAT16 or you forgot to set the bootable flag. In that case, please refer to the Additional Notes below.
  • Select FREE SPACECreate a new partition and create a 1 GB primary swap partition.
  • Select FREE SPACECreate a new partition and allocate the rest to a primary ext4 root partition (mountpoint = /)
  • After doing the above, your partition report should look like this:

    Code: Select all

    MMC/SD card #1 (mmcblk0) - 16.0 GB SD 2WCGO
         #1  primary  314.6 MB  B  K  ESP
         #2  primary    1.0 GB     f  swap    swap
         #3  primary   14.7 GB     f  ext4    /
    
  • Select Finish partitioning and write changes to disk, then Yes and let the installer continue with the base system installation.
  • After a while, the installer should produce a message that states:

    Code: Select all

    [!!] Configure the package manager
    
    apt-configuration problem
    An attempt to configure apt to install additional packages from the CD failed.
    
    This is a benign message that you can safely ignore by selecting Continue (The reason it is benign is we are running a net install and won't need to access the "CD-ROM" files post install).
  • Once you have dismissed the message above, pick the mirror closest to your geographical location and let the installer proceed with some more software installation.
  • Finally, at the Software selection screen, choose any additional software package you wish to install. Debian desktop environment should work out of the box if you choose to install it.
  • Let the process finalize the software and GRUB bootloader installation and, provided you didn't run into the partition manager issue described above you can reboot your machine when prompted, which, once completed, should bring you to your newly installed vanilla Debian environment. :D

Additional Notes

The reason we use -t vfat -o rw /dev/mmcblk0p1 for the source media (i.e. "CD-ROM" device) is because, whereas the first partition on the SD card is indeed /dev/mmcblk0p1, we also need to provide additional parameters for the mount command that the installer invokes behind the scenes. For instance, if we don't use -t vfat, then ISO-9660 is forced as the file system, and if we don't use -o rw then the partition will be mounted as read-only which then prevents the same partition from being remounted when locating the non-free firmware files or when setting up /efi/boot.

With regards to fixing the partitioning if you don't see B K ESP when entering the partition manager, what you need to do is:
  • Before you create the additional partitions, select the first partition and change its type to ESP. Note however that doing this changes the type of the partition to 0xef which is precisely what we're trying to avoid by having the partition manager already detect it as ESP, as type 0xef is unbootable by the Broadcom CPU.
  • To fix this then, before you choose Continue on the Installation complete prompt you should open a new console with <Alt>-<F2> and type:

    Code: Select all

    chroot /target fdisk /dev/mmcblk0
    
  • From within fdisk press t, 1, e and w, to reset the partition to type 0x0e (FAT16 LBA).
  • Go back to the main installation window (<Alt>-<F1>) and continue with the reboot.

Leeloo
Posts: 46
Joined: Fri Oct 06, 2017 9:53 pm

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Thu Aug 29, 2019 3:13 am

just to give my experience, the guide was relatively easy to follow and everything went pretty smoothly apart from a few issues.

for me at least it seemed letting the installer automatically allocate partitions based on free space is bad because it will complain it can't find the Uefi parition then later grub will fail to install and I guess the only way I could find out of it was to start over :oops: when i manually allocated the partitions it worked just fine.

another thing I noticed, because you have to specify the /boot partition as the cdrom during install it will make a cdrom link on the desktop pointing to boot partition that right clicking on will not delete :<

for some reason even though I installed the none free wifi drivers wireless would not conenct to my router during setup but it worked just fine installing via Ethernet and switching to wireless after the install was completed.

also it didn't add the default username to the list of"sudoers" I didn't even know that was a thing, guess I will have to look up how todo that.

other than that running fine, kind of nice to be running a proper full Linux distro on the pi!

Thanks

ps:wondering if any other distros offer arm64 images / support that can be installed using the same method ?

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Thu Aug 29, 2019 1:06 pm

Leeloo wrote:
Thu Aug 29, 2019 3:13 am
the guide was relatively easy to follow and everything went pretty smoothly apart from a few issues.

Thanks for testing it and reporting!
letting the installer automatically allocate partitions based on free space is bad because it will complain it can't find the Uefi parition then later grub will fail to install and I guess the only way I could find out of it was to start over :oops: when i manually allocated the partitions it worked just fine.

Yes, that is why you shouldn't try to deviate too much from the guide, and why the steps above do guide you through manual partitioning specifically.

The problem here is that, if you let the Debian installer do its automatic partitioning, and especially if it's for a separate drive (e.g. if you boot the installer from USB, instead of SD, which many may think looks simpler because then you don't have to fiddle with telling Debian where your "source CDROM" is), because it detected that it booted in UEFI mode, it will of course use UEFI defaults, and either create a new GPT drive, which sadly is incompatible with the Broadcom bootloader or, if has to work with a drive that is already set to use MBR, will try to use 0xEF type for the EFI System Partition, which also won't work with the Broadcom SoC...

One can only wish that Broadcom had added GPT support, or, at the very least, added type 0xEF to the type of MBR partitions their SoCs can boot from (with the latter literally being a <4 bytes change for their internal bootloader, that they could have added a long time ago), as we wouldn't be in this mess. But alas, in the many years they've been pushing silicon, Broadcom doesn't seem to have registered that, yes, UEFI partition support is more than a gimmick and that end users will be let down if you continue to treat it as a secondary item.

I am hoping that, perhaps, the EEPROM bootcode.bin might be able to work around this blatant SoC limitation for the Pi 4, but I'm not holding my breath...
another thing I noticed, because you have to specify the /boot partition as the cdrom during install it will make a cdrom link on the desktop pointing to boot partition that right clicking on will not delete :<

This is covered in the longer guide that I have linked to (Post install fixes). Because I didn't want to make the guide above too long, I chose to remove this extra information.

In short, these are the 2 small post install issues that I am aware of at the moment, and how to fix them:

  • You may find a cdrom0 drive on your desktop, which can't seem to be accessible. This is a leftover from the installer process not knowing how to handle the installation media device. You should edit /etc/fstab as root to remove it.
  • If you installed the cups package, you may get an error during boot while loading modules (systemctl --failed will report that systemd-modules-load.service is in failed state). This is all due to the current cups package trying to load IBM PC kernel modules... on a non PC device. To fix this, simply delete /etc/modules-load.d/cups-filters.conf and reboot.
for some reason even though I installed the none free wifi drivers wireless would not conenct to my router during setup

That's strange. I did test Wireless install on a B+ and had no issue connecting. We'll have to see if other users report the same issue...
also it didn't add the default username to the list of"sudoers" I didn't even know that was a thing, guess I will have to look up how todo that.

I don't think this has anything to do with the guide above or the fact that a Pi3 was used (i.e. I expect you'll find the same issue when installing Debian in UEFI mode on a PC).
ps:wondering if any other distros offer arm64 images / support that can be installed using the same method ?

You can find some information about that in the Systems.md guide for the EDK2 repository.

I've had good success installing Ubuntu 18.04 using netinst on a Pi 3B. Last time I checked, the 3B+ had some issues with Ubuntu on account of the ethernet network driver not having been updated yet but maybe this has changed with the refreshed updates for 18.04. The guide mentions that OpenSUSE Leap 42.3 has also been successfully tested (but not by me), therefore we expect most ARM64 generic distros, that use a kernel recent enough to support the Rpi3 hardware, and can handle UEFI boot, to install and work relatively well.

Oh, and you can also get a full Windows 10 working (with the full GUI and everything, not just in IoT mode), but that's a different story.

As I said, things do become incredibly easy when you have a proper UEFI firmware for your platform... :D

geev03
Posts: 122
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Thu Aug 29, 2019 6:06 pm

Akeo wrote:
Thu Aug 29, 2019 1:06 pm
...

Thanks for testing it and reporting!
....
Oh, and you can also get a full Windows 10 working (with the full GUI and everything, not just in IoT mode), but that's a different story.

As I said, things do become incredibly easy when you have a proper UEFI firmware for your platform... :D
On a RPi3B , the installation was successful, thanks to the details, ( https://pete.akeo.ie/2019/07/installing ... ry-pi.html )
I have a Windows 10 (ARM) ,Pi3B but stuck in version 1803. So how can one install the latest WoA-1903 using this UEFI ?

Code: Select all

login as: geev03
geev03@192.168.1.187's password:
Linux pi64deb 4.19.0-5-arm64 #1 SMP Debian 4.19.37-5+deb10u2 (2019-08-08) aarch64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Could not chdir to home directory /home/geev03: No such file or directory
$ uname -a
Linux pi64deb 4.19.0-5-arm64 #1 SMP Debian 4.19.37-5+deb10u2 (2019-08-08) aarch64 GNU/Linux
$ free -m
              total        used        free      shared  buff/cache   available
Mem:            908         313         108           5         486         507
Swap:           953           0         953
$

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Thu Aug 29, 2019 6:53 pm

geev03 wrote:
Thu Aug 29, 2019 6:06 pm
On a RPi3B , the installation was successful, thanks to the details, ( https://pete.akeo.ie/2019/07/installing ... ry-pi.html )

Great!
So how can one install the latest WoA-1903 using this UEFI ?

This is outside of the scope of this thread, which is for Debian, but you'll find a couple of links that should help with Windows installation in the Systems.md from the EDK2 UEFI source, that I already linked above.

Be mindful that, as you may expect, running a full Windows on a Raspberry Pi 3 is incredibly slow, due hardware bottlenecks in terms of CPU, RAM and SD card performance. So please don't go installing Windows on a Pi 3 in the hope that it can be used as cheap Windows hardware. On the other hand, if all you want is test the ARM64 version of Windows 10, it's a great way to achieve just that.

Leeloo
Posts: 46
Joined: Fri Oct 06, 2017 9:53 pm

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 3:27 am

Sorry the first issue was totally my own fault having some experience with I possibly skimmed parts of the guide more than I should have :oops:
That's strange. I did test Wireless install on a B+ and had no issue connecting. We'll have to see if other users report the same issue...
mine is the 3B+ too, tried it 3 times because I thought maybe I typed the key wrong, each time took a while requesting keys or similar then failed, sorry I didn't get a more specific error message. when/if I get around to installing it again I will report back.
This is covered in the longer guide that I have linked to
I will check it thank you!

I am curious of a few things, when using the Uefi the frequency is set by the loader, won't the OS also scale / throttle the cpu speed as needed? and secondly is it possible the loader would support other versions of the pi eventually ? (zero is my favorite :D )

Thank you!

geev03
Posts: 122
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 9:03 am


...
So please don't go installing Windows on a Pi 3 in the hope that it can be used as cheap Windows hardware. On the other hand, if all you want is test the ARM64 version of Windows 10, it's a great way to achieve just that.
Yes, it is just to test the 64 bit Pi3B hardware with another 64 bit OS. WoA (Windows-10, 1803) has WSL and so it can be tested for endurance under WsL Debian etc.
Attachments
woa_ARMv8_RPi3b_WSL_Debian.jpg
woa_ARMv8_RPi3b_WSL_Debian.jpg (139.34 KiB) Viewed 2097 times

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 9:33 am

Leeloo wrote:
Fri Aug 30, 2019 3:27 am
won't the OS also scale / throttle the cpu speed as needed?

Yes. The custom frequency you set is the maximum frequency you want, but as with all modern CPUs, the OS kernel will scale / throttle down that frequency when the load is low.
is it possible the loader would support other versions of the pi eventually ? (zero is my favorite :D )

The next Pi that should receive UEFI support is the Pi 4, as there are quite a few people interested in seeing that happens sooner rather than later considering that it's more or less solving 2 of the main issues we have regarding to using a Pi as a general computing platform which are: Lack of RAM (the 4 GB model finally offers a bit more breathing space there) and, the main bottleneck so far, especially if you're trying to run Windows, slow transfer rates when accessing the system disk (since, with the Pi 4, it has finally becomes realistic to try to run a system at true USB 3.0 speeds, without contention about NIC access).

On the other hand, because this is a volunteer project, I doubt there will be much traction for the Zero, especially as it uses a different CPU, but who knows...

And again, I'm going to use this opportunity to point out that it would be nice to have involvement of the Raspberry Pi Foundation with regards to the development of the UEFI firmware, because, if I were in charge of the project, I would certainly be looking into boosting the size of the Pi 4 eeprom to 2 MB (how about doing that with the next revision, that fixes those USB-C resistors? :D) and having a resident UEFI firmware there for all future models, so that users do get a complete UEFI environment when they plug a Pi (even when they don't have any SD or USB media), with a shell and everything, instead of a dumb screen. You could even customize this to provide some friendly UI messages, or add stuff like a BASIC interpreter into the UEFI shell, which would make for a much friendlier experience for both beginners as well as experienced users...

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5964
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 10:21 am

The cost of a larger EEPROM makes built-in UEFI impractical for the pi. I'd recommend using the sd card like it's the EEPROM and chainloading tianocore's implementation from there. As your guide and others point out, that's already possible.

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 12:43 pm

I appreciate the reply, but can you give a an estimate for how prohibitive the cost of going from a 4MBit to a 16MBit EEPROM would be for a Pi4?

The thing is, we're not even asking for a redesign here (since it's SPI), just going from an existing 512KB EEPROM to a model with 4x the capacity (since, even with bootcode.bin, what remains of 2 MB should be enough for the UEFI firmware).

With volume pricing, I strongly doubt the costs being incurred are going to be that dramatic. Especially, since the cost of the new 512KB EEPROM has been factored in the base price for the 1GB Pi 4 model, I doubt it is that high to start with. I could of course look up the pricing myself, but I'd rather have a person more familiar with the process provide figures on how prohibitive multiplying the size of the EEPROM by 4 (or 8, as I'm not sure if we can do away with start.elf, and this might be the biggest roadblock) would actually be.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5964
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 1:19 pm

I don't have any numbers, but given the price point of the pi, any cost increase at all is dramatic and needs to be weighed against the benefits it offers.

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 1:31 pm

Okay. I still appreciate your willingness to engage in this kind of discussions. 8-)

Hopefully, there will come a stage in the future of the Pi platform, even if it's long term, where the benefits will be seen as outweighing the cost increases.

For the time being, I guess it'll up to the people involved in its development to demonstrate that having a UEFI firmware for the platform is something that benefits users, so we'll continue to try to do just that!

trejan
Posts: 574
Joined: Tue Jul 02, 2019 2:28 pm

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 1:48 pm

Akeo wrote:
Fri Aug 30, 2019 12:43 pm
The thing is, we're not even asking for a redesign here (since it's SPI), just going from an existing 512KB EEPROM to a model with 4x the capacity (since, even with bootcode.bin, what remains of 2 MB should be enough for the UEFI firmware).
How are you going to fit everything into 2MB? start4x.elf + fixup4x.dat + bcm2838-rpi-4-b.dtb is ~3.5MB alone. If you use the debug firmware then it is ~4.5MB. That isn't including the bootcode which is currently padded to 512KB or your UEFI firmware.

You're looking at an 8MB EEPROM to comfortably fit everything and have space for future expansion.

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 2:15 pm

The .dtb is included in the UEFI firmware so there's no need to provide it separately. And my current understanding is that start.elf probably duplicates a lot of things that the firmware does and, ideally, we should be able to combine them into a single EEPROM binary that merges the two, but which is much smaller than the size of the separate files combined. As a matter of fact, the current requirement for users to manually provide so many separate files for boot is one more reason why combining/hiding them into a firmware EEPROM would be a lot nicer.

But then again, I touched on the start.elf issue in my previous post and I have reasons to think (since the Pi3 firmware is far from occupying its 2 MB payload) that, even if we have to include a verbatim start.elf a 4 MB EEPROM should be fine.

The thing is, if you look at the RPi3 UEFI firmware downloads from here, which contain all of the files needed to boot a Pi 3 (including both the Model B and Model B+ Device Trees), you'll find that all this data is only slightly above 3 MB compressed. And I see little reason why the EEPROM should not be able to handle compressed elements, when most of its (current) process is: load stage n, execute stage n, for which, since we should have control of some of this staging code (or at least, I hope the people from the Raspberry Pi Foundation who liaise with Broadcom do), we should be able to add a decompression process as needed.

Hence my conclusion that, in the best case scenario (removal of duplicate/unneeded features between start.elf and UEFI.fd), we should be able to fit in 2MB, and if not, 4MB should still be enough.

But of course, without having a real Pi 4 UEFI firmware, or feedback from Broadcom/The Pi Foundation on how difficult it might be to add decompression in the various stages, this remains an educated estimate and we may indeed have to go for 8 MB (which, not being involved in hardware procurement process and considering the tendency of EEPROM chips to steadily go down in price I still think should be okay. But of course, I am not the one who has to try to balance production costs).

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Aug 30, 2019 2:37 pm

...and of course, the minute I posted the above, I remembered that, if I'm not mistaken, start.elf is for the VideoCore so it's not really something that we will be able to combine. But what I mentioned about decompressing each stage still stands, and even with a standalone start.elf, and early bootcode that of course can not be compressed, I do think 4 MB should be enough.

User avatar
ehem
Posts: 6
Joined: Mon Sep 09, 2019 1:17 am

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Thu Sep 12, 2019 8:58 pm

Akeo wrote:
Wed Aug 21, 2019 12:59 pm
Opinionated preamble (skip this part if you're only interested in the guide)

The reason we are able to perform this above is because we finally have a full UEFI firmware for the 3B/3B+, and because the Debian ISOs, be it for x86 or ARM64, are pretty much designed to work right out of the box with UEFI.

At this stage, I feel the need to voice my opinion as to why people who have been interested in getting vanilla distros such as Debian (or Ubuntu, or Suse, or whatever, since it makes things equally easy for those) as well as the Raspberry Pi Foundation haven't been helping with the effort to get a UEFI firmware going, because that is really the one piece of the puzzle anyone interested in running a generic ARM/ARM64 distro should have been concentrating on, instead of trying to alter anything and everything on a case by case basis, to get it to run...

As valid as some of the criticism it receives might be, there is a good reason UEFI exists in the first place, which a lot of people don't seem to appreciate, and that is to ensure that any OS can rely on a unified boot environment to set everything it needs moving forward.
I would like to second this opinion. UEFI is a really awful standard, but it is a well-known one and thus doesn't require specialty support. Being able to boot standard OS images is an extremely valuable feature.

If the effort to port every Linux distribution individually was put into getting EFI support working on the RP4 all of them would almost certainly be available on the RP4 sooner.

ShiftPlusOne wrote:
Fri Aug 30, 2019 10:21 am
The cost of a larger EEPROM makes built-in UEFI impractical for the pi. I'd recommend using the sd card like it's the EEPROM and chainloading tianocore's implementation from there. As your guide and others point out, that's already possible.
I also agree with this opinion. Raising the cost of the Pi is a Bad Idea(tm). Further, allowing access to the boot process before something EFI gets loaded lets students get closer to the raw hardware.

I would suggest though that the Raspberry Pi Foundation should spend some time in trying to get EFI onto the RP4. It would be quite valuable for a future Raspberry Pi 5 to have a chain-loadable EFI implementation on the day it becomes available.


Akeo wrote:
Wed Aug 21, 2019 12:59 pm
  • Partition the SD media as MBR and create a ~300 MB partition on it with MBR type 0x0e.
    Note: Make sure that the partition scheme is MBR (not GPT) and the type 0x0e (not 0xef for instance), as the on-CPU Broadcom bootloader supports neither the GPT scheme nor the ESP MBR type.
  • Set the partition as active/bootable. This is needed as the Debian partition manager can not detect it as ESP otherwise, which we need for GRUB installation. If using fdisk on Linux, you can use the a command to set a partition as active. On Windows, you can use diskpart and then type active after selecting the relevant disk and partition.
  • Format the partition as FAT16 (again needed for Debian partition manager to detect it as ESP). If you are using Windows diskpart then format fs=fat quick will format a drive to FAT16. On Linux, the equivalent command would be mkfs.vfat -F 16 /dev/<your_device>. If you need help with the partitioning and formatting of your device, you can find full detailed steps in the Appendixes of the blog entry I linked above.
  • Copy all the UEFI firmware files, including any optional files such as the Wifi firmware drivers, onto the FAT partition.
  • Extract the full content of the ISO onto the FAT partition.
  • Eject your SD card and insert it into your Raspberry Pi 3
Before doing anything else, I would suggest running `blkdiscard /dev/<sdcard device>`. You're not going to want any of the old data from the card, so doing UNMAP/TRIM first is best.

Why are you suggesting "~300 MB"? Is this due to needing enough space for the ISO image? This due to the 256MB being the "conventional" size of the bootloader area? For the case I'm thinking of I would either want to match the 256MB recommendation or to make it much smaller so I didn't need to waste all that storage space. I would be tempted to create a temporary diskslice^Wpartition to store the ISO image and then free it later.

There is a rather nicer, but tricky, solution for this situation a "hybrid MBR". Create a GPT table on the SD card and create the boot area. Create a temporary file which equals the number of sectors of the SD card multiplied by 512 (if the SD card uses 512B sectors match the size, if the SD card uses 4KB sectors divide the size by 8). This can be accomplished with `truncate -s 17179869184` (512B sectors) or `truncate -s 2147483648` (4KB sectors).

Now run your favorite `fdisk` program on this temporary file and tell it to create an entry matching the boot area in the GPT and type 0x0E. Then create 1-2 entries which cover the rest of the SD card, but with type 0xEE. Then do `dd count=1 if=disk.image of=/dev/mmcblk0` (likely /dev/sd? with a SD to USB adapter). I would also recommend saving a copy of this header for later use (`dd count=1 if=disk.image of=<somewhere on the boot area where you can find it readily>`).

The theory is the boot firmware will read the MBR and load the files it needs. Once GRUB or the proper OS is up and running, it will operate according to the GPT. This way you can have lots of slices^Wpartitions while still being able to boot. Beware that whenever you modify the table your favorite `fdisk` program may nuke the MBR and it will need to be restored.

Be careful with this technique as it isn't exactly kosher by the standards!

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Fri Sep 13, 2019 11:04 am

ehem wrote:
Thu Sep 12, 2019 8:58 pm
I would suggest running `blkdiscard /dev/<sdcard device>`.

That's fine, and probably a good suggestion if you are on Linux... but the guide is meant to cater to Windows users as well.

Also, from what I can read, there aren't many SD card readers that support TRIM and apparently the one well known hardware that supports this... is the Raspberry Pi. But of course that makes it a bit difficult to use blkdiscard since you'd first have to have another Linux install running from USB to be able to use blkdiscard on the SD card.

At any rate, concerns that have to do with the underlying technology that is currently being used by flash based media (who knows if TRIM is still going to be relevant on SD cards that are manufactured 10 years from now) are IMO better left out of an installation guide.
Why are you suggesting "~300 MB"? Is this due to needing enough space for the ISO image?

Yup. Being able to copy the full ISO content into the ESP greatly simplifies the installation process.

For the record, there is no UEFI recommendation that I know of regarding the size of the ESP. Rather each OS manufacture seems to have their preferred minimum size. For instance, whereas Microsoft used to recommend 100 MB as minimum for ESP (or 300 MB if you have a disk with 4K sectors), I know that the disk manager on Mac OS has issues recognizing an ESP that is less than 200 MB, because that's what Apple uses as their minimum size. So I would gather that the 256MB "standard" ESP size that everybody seems to want to use these days probably comes out of that. But I'd wager that, once people start to have to contend with 4K drives (that actually report a 4K sector size, as a lot of HDD manufacturers currently seem to use 4K internally, but still report 512 bytes externally), we're going to start recommendations for ESPs being larger than 300 MB as the new de-facto size...

And again, I will point out that, if you want to deviate from the guide, and create a smaller ESP and/or a separate partition to store the ISO content, or install from USB, it's 100% your choice, as the guide was designed to make it as straightforward as possible for all users to complete the installation, by avoiding possible alternatives that add steps that would make it easy to screw things up. Besides, when using a 16 or 32 GB SD card, I don't see "wasting" an extra 50 MB for the ESP as that big a deal. If anything, you never know when having a little extra space on the ESP might come handy, as opposed to being hell bent on saving a measly 50 MB off your multi-GB media...

Likewise for the Hybrid MBR proposal. You're of course free to add complexity to your setup if you want GPT + MBR. But that's definitely not something I'd recommend for first time users, especially as it may make maintenance of the system more difficult in the long run.

Simplicity has the advantage of being better supported out of the box... ;)

User avatar
ehem
Posts: 6
Joined: Mon Sep 09, 2019 1:17 am

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Sat Sep 14, 2019 12:17 am

Akeo wrote:
Fri Sep 13, 2019 11:04 am
For the record, there is no UEFI recommendation that I know of regarding the size of the ESP. Rather each OS manufacture seems to have their preferred minimum size. For instance, whereas Microsoft used to recommend 100 MB as minimum for ESP (or 300 MB if you have a disk with 4K sectors), I know that the disk manager on Mac OS has issues recognizing an ESP that is less than 200 MB, because that's what Apple uses as their minimum size. So I would gather that the 256MB "standard" ESP size that everybody seems to want to use these days probably comes out of that. But I'd wager that, once people start to have to contend with 4K drives (that actually report a 4K sector size, as a lot of HDD manufacturers currently seem to use 4K internally, but still report 512 bytes externally), we're going to start recommendations for ESPs being larger than 300 MB as the new de-facto size...
Sounds like the default Raspbian installation is recommending 256MB for the boot area. I'm curious about why they're suggesting making it that large.

There is a trickier issue though. Some of the larger (>128GB) SD cards have a rather large erase size. A 200GB SanDisk card I have handy gives a sector size of 512B, but a discard_granularity (look in /sys/block/mmcblk0/queue) of 24MB. I'm unsure whether a given erase block can hold blocks written all over the device, versus explicitly holding a given 24MB range. If it is a fixed range then 24MB alignment could be a dramatic performance increase on the particular card. I'm suspecting larger SD cards are getting higher performance by using large blocks internally which means their lifetime may not be nearly as good as expected.

Meeting the requirements of the "A1" or "A2" benchmarks may encourage use of smaller erase blocks, but that isn't required. I'm a bit concerned at this situation.

Akeo wrote:
Fri Sep 13, 2019 11:04 am
Likewise for the Hybrid MBR proposal. You're of course free to add complexity to your setup if you want GPT + MBR. But that's definitely not something I'd recommend for first time users, especially as it may make maintenance of the system more difficult in the long run.

Simplicity has the advantage of being better supported out of the box... ;)
That is true, but someone wanting to have more than 3 diskslices^Wpartitions available to them might appreciate a mention that something can be done to allow that. I don't think someone needs to be very far beyond beginner to want that many.



Right now I'm still in the planning stage for my project, as such I don't even have all of the hardware. Since it is something my project requires, would you be willing to try installing the "xen-hypervisor" package on your Debian RP3?

If yes, could you install it and then try rebooting and selecting the "Debian GNU/Linux, with Xen hypervisor" option in GRUB. Does anything come up on the HDMI output? Does anything come up on the serial port?

In theory Xen is supposed to work on A53s, but the reports on another thread were no one had gotten anywhere on a PI 3. Your setup is one worth testing since having GRUB load Xen is the standard configuration. There are indications Raspbian has removed all support for running with Xen from their kernel, so vanilla Debian is worthy of trying. (FYI the thread is here)

User avatar
Akeo
Posts: 11
Joined: Wed Aug 14, 2019 7:17 pm
Contact: Website

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Sun Sep 15, 2019 6:10 pm

ehem wrote:
Sat Sep 14, 2019 12:17 am
Does anything come up on the HDMI output? Does anything come up on the serial port?

On both outputs (since I have GRUB configured to output to serial as well as EFI console. Note that you'll need yet to be released UEFI Firmware v1.6 for that to work), here's what I get after selecting Debian GNU/Linux, with Xen hypervisor:
Loading Xen 4.11-arm64.efi ...
error: can't find command `multiboot'.
Loading Linux 4.19.0-6-arm64 ...
error: can't find command `module'.
Loading initial ramdisk ...
error: can't find command `module'.

Press any key to continue...

If you need more testing, then you'll have to get a Pi 3 and run it yourself, as I'm afraid my time is not unlimited...

User avatar
ehem
Posts: 6
Joined: Mon Sep 09, 2019 1:17 am

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Wed Sep 18, 2019 1:53 am

Akeo wrote:
Sun Sep 15, 2019 6:10 pm
If you need more testing, then you'll have to get a Pi 3 and run it yourself, as I'm afraid my time is not unlimited...
I thought asking you to try the single package was borderline. You saying "no" to that would have been fine. Since you were willing to try the single package though, I've already filed a bug with Debian about multiboot.mod is missing from grub-efi-arm64-bin. This does answer the question, running Xen on a Raspberry PI is going to be interesting.

Since I'm going to need a PI 4 for my project though I'm going to wait a bit. I think the Raspberry PI Foundation investing a bit of engineer time on UEFI for the Raspberry PI 4 would be time well-spent. As you've noted being able to run completely unmodified Linux distributions is a rather nice feature...

robbo007
Posts: 17
Joined: Thu Aug 23, 2012 8:28 am

Re: Guide: Installation of *VANILLA* Debian 10.0 or later on Pi 3B/3B+

Thu Oct 17, 2019 9:45 pm

Without reading 1000 of line of comments. Does Vanilla Debian work on the Raspberry Pi 4? Cant seem to get a straight answer?

Return to “Debian”