Page 1 of 11

Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Tue Sep 03, 2019 11:56 pm
by dom
As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.
This is for bleeding edge testers. Things may not work. Backing up, or using a fresh sdcard is a wise precaution.

To get the kernel run the usual

Code: Select all

sudo rpi-update
to get the latest kernel. If you want to switch to 64-bit kernel add to config.txt

Code: Select all

arm_64bit=1
You can still the usual 32-bit raspbian buster image just fine, but the kernel will be 64-bit.
Installing and running 64-bit packages should now be possible.

The 64-bit kernel (even with 32-bit raspbian) should have some performance advantages with kernel heavy loads (e.g. usb/ethernet) as we avoid some of the page mapping costs of LPAE. There may also be some (lesser) performance boost from the 64-bit kernel code.

Feel free to try it. Report any tests that run faster (or slower) and anything that does or doesn't work.

Note: We're focusing on Pi4 here. In theory the kernel does run on Pi3, but that's not our priority.
The lack of FIQ on 64-bit kernel will reduce USB performance on Pi3 (Pi4 doesn't require the FIQ).

Code: Select all

pi@pi4:~ $ uname -a
Linux domnfs 4.19.69-v8+ #1261 SMP PREEMPT Tue Sep 3 20:27:16 BST 2019 aarch64 GNU/Linux
Known issues:
vcsm driver doesn't work with 64-bit (only the cma version) which will break software decode with kodi
Mathematica and Wolfram fail to run (detection of raspberry pi os fails) Workaround here
Java 8 doesn't work. OpenJDK 11 does work.

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 12:41 am
by jdonald
This is awesome.
dom wrote:
Tue Sep 03, 2019 11:56 pm
performance advantages with kernel heavy loads (e.g. usb/ethernet)
Is the FIQ issue with dwc_otg fixed? Not sure which GitHub tickets track that issue but last I saw detailed mention of it is in rst's post.

If this is unresolved, my impression is that it breaks plenty of USB webcams and other devices transferring in bulk are slower than their 32-bit kernel behavior? If so that would make it hard to properly benchmark kernel-heavy USB loads.

First thought for anyone doing ethernet benchmarking: you must use unencrypted rsync/ftp/rsh/http or else the cipher will bottleneck on the CPU in userland.

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 8:06 am
by 6by9
jdonald wrote:
Wed Sep 04, 2019 12:41 am
This is awesome.
dom wrote:
Tue Sep 03, 2019 11:56 pm
performance advantages with kernel heavy loads (e.g. usb/ethernet)
Is the FIQ issue with dwc_otg fixed? Not sure which GitHub tickets track that issue but last I saw detailed mention of it is in rst's post.
Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).
As noted in the title, this is mainly aimed at pi4.

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 9:00 am
by epoch1970
dom wrote:
Tue Sep 03, 2019 11:56 pm
As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.
Great news. Well done!

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 9:27 am
by jamesh
epoch1970 wrote:
Wed Sep 04, 2019 9:00 am
dom wrote:
Tue Sep 03, 2019 11:56 pm
As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.
Great news. Well done!
A lot of work has been done by the community and upstream devs, so very much a combined effort.

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 10:24 am
by DougieLawson
Can you remind me, which Raspberries does 64 bit work on? I know 3B+ and 4Bs are OK, as I've run Sakaki-san's Gentoo on my 3B+ (back in February).

What's the revision numbers for the 64-bit capable boards?

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 10:41 am
by DougieLawson
This doesn't work on a 3A+ with

Code: Select all

arm_64bit=1
but does work OK with

Code: Select all

kernel=kernel8.img
.

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 10:44 am
by epoch1970
DougieLawson wrote:
Wed Sep 04, 2019 10:24 am
Can you remind me, which Raspberries does 64 bit work on? I know 3B+ and 4Bs are OK, as I've run Sakaki-san's Gentoo on my 3B+ (back in February).
Pi 3B is ok, I suppose Pi 2 rev. 1.2 (with the same SOC as 3B) runs in 64bit mode as well.
A good year (or is it more...) ago I had a 64b buildroot OS working just fine on my Pi 3Bs (linux tree off RPF's Github repo), but I switched back to 32 because goodies like vcgen (?) or some GPIO libs were not available. The full Pi "experience" was not there yet.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 10:56 am
by jamesh
As Dom said, this focuses on Pi4, although MAY work on other Pi's. I've changed the title to reflect that.

For revision codes that say which processor you have, see here...https://www.raspberrypi.org/documentati ... /README.md

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 11:07 am
by geev03
dom wrote:
Tue Sep 03, 2019 11:56 pm
....
You can still the usual 32-bit raspbian buster image just fine, but the kernel will be 64-bit.
Installing and running 64-bit packages should now be possible.
......
Feel free to try it. Report any tests that run faster (or slower) and anything that does or doesn't work.

.....

Code: Select all

pi@pi4:~ $ uname -a
Linux domnfs 4.19.69-v8+ #1261 SMP PREEMPT Tue Sep 3 20:27:16 BST 2019 aarch64 GNU/Linux
Known issues:
vcsm driver doesn't work with 64-bit (only the cma version) which will break software decode with kodi

The 64 bit kernel shows up and remote access is working.
- Web browser starts and works
- Mathematica and Wolfram are not showing up :(

Code: Select all

pi@raspberrypi:/boot $ sudo rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** We're running for the first time
 *** Backing up files (this will take a few minutes)
 *** Remove old firmware backup
 *** Backing up firmware
 *** Remove old modules backup
 *** Backing up modules 4.19.66-v7l+
PARTSIZE:268435968
#############################################################
WARNING: 'rpi-update' updates to pre-releases of the linux
kernel tree and Videocore firmware.

'rpi-update' should only be used if there is a specific
reason to do so - for example, a request by a Raspberry Pi
engineer.

DO NOT use 'rpi-update' as part of a regular update process.

##############################################################
Would you like to proceed? (y/N)
 *** Downloading specific firmware revision (this will take a few minutes)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168    0   168    0     0    800      0 --:--:-- --:--:-- --:--:--   800
100  113M  100  113M    0     0  1292k      0  0:01:29  0:01:29 --:--:-- 2859k
 *** Updating firmware
 *** Updating kernel modules
 *** depmod 4.19.69-v8+
 *** depmod 4.19.69-v7+
 *** depmod 4.19.69-v7l+
 *** depmod 4.19.69+
 *** Updating VideoCore libraries
 *** Using HardFP libraries
 *** Updating SDK
 *** Running ldconfig
 *** Storing current firmware revision
 *** Deleting downloaded files
 *** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to 1b797509b8c28cba7834684986b97feb1a265e9c
 *** A reboot is needed to activate the new firmware
pi@raspberrypi:/boot $

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 11:18 am
by DougieLawson
jamesh wrote:
Wed Sep 04, 2019 10:56 am
For revision codes that say which processor you have, see here...https://www.raspberrypi.org/documentati ... /README.md
I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.

I still don't have a RPi4B 4GB because the household finance director doesn't leave for her three months in Australia until Oct 7th.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 11:28 am
by Gavinmc42
Just threw some free pascal opengl at it, works fine :D
Lazarus and Ultibo IDEs working.
I still don't have a RPi4B 4GB because the household finance director doesn't leave for her three months in Australia until Oct 7th.
Oh, thanks for the reminder, Pi4B4's are due in Sep here in Oz :D
Oh no, that first pre order batch must have sold out, next lot 14th Oct :(

Ok, now for the serious stuff, see if it works with the stuff I threw at Gentoo64.
Overnight compiles etc

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 11:36 am
by dom
DougieLawson wrote:
Wed Sep 04, 2019 10:41 am
This doesn't work on a 3A+ with

Code: Select all

arm_64bit=1
but does work OK with

Code: Select all

kernel=kernel8.img
.
The requirement for kernel=kernel8.img was mentioned in first post.
I'm just building a firmware that will go out later that make that no longer necessary.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 11:41 am
by dom
DougieLawson wrote:
Wed Sep 04, 2019 11:18 am
I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.
You need BCM2837 (PI3/Pi3+) or BCM2838 (Pi4).
Later models of Pi2 switched to using BCM2837, but the majority sold will be BCM2836 (so no 64-bit capability)

The revision code is /proc/cpuinfo is the definitive information.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 11:46 am
by DougieLawson
dom wrote:
Wed Sep 04, 2019 11:41 am
DougieLawson wrote:
Wed Sep 04, 2019 11:18 am
I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.
You need BCM2837 (PI3/Pi3+) or BCM2838 (Pi4).
Later models of Pi2 switched to using BCM2837, but the majority sold will be BCM2836 (so no 64-bit capability)

The revision code is /proc/cpuinfo is the definitive information.
Thank you. One 3A+ is running already, the 3Bs/3B+ are just waiting for the rpi-update to finish.

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 12:20 pm
by dom
dom wrote:
Wed Sep 04, 2019 11:36 am
The requirement for kernel=kernel8.img was mentioned in first post.
I'm just building a firmware that will go out later that make that no longer necessary.
Firmware pushed. arm_64bit=1 is all you need to enable 64-bit kernel on Pi3 too now.

Re: Pi4 64-bit raspbian kernel for testing

Posted: Wed Sep 04, 2019 1:01 pm
by geev03
dom wrote:
Wed Sep 04, 2019 12:20 pm
....

Firmware pushed. arm_64bit=1 is all you need to enable 64-bit kernel on Pi3 too now.

Code: Select all

*** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to c472ab43407d3bf1ed162a9f7c3f5060e6f99d14
 *** A reboot is needed to activate the new firmware
OK, the same Pi4B sd card works on Pi3B, ( with the same problems with some of the programs that worked fine on 32 bit buster)

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 1:18 pm
by dom
geev03 wrote:
Wed Sep 04, 2019 11:07 am
- Mathematica and Wolfram are not showing up :(
If you are daring, edit /opt/Wolfram/WolframEngine/12.0/Executables/mathematica
find lines:

Code: Select all

				armv?l)
					SystemIDList="Linux-ARM";;
and add after:

Code: Select all

				aarch64)
					SystemIDList="Linux-ARM";;
You can also make a comparable change to /opt/Wolfram/WolframEngine/12.0/Executables/wolfram

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 1:48 pm
by geev03
dom wrote:
Wed Sep 04, 2019 1:18 pm
..

If you are daring, edit /opt/Wolfram/WolframEngine/12.0/Executables/mathematica
find lines:

Code: Select all

				armv?l)
					SystemIDList="Linux-ARM";;
and add after:

Code: Select all

				aarch64)
					SystemIDList="Linux-ARM";;
You can also make a comparable change to /opt/Wolfram/WolframEngine/12.0/Executables/wolfram
Thanks a lot, perfect.

Code: Select all

pi@raspberrypi:~ $ sudo wolfram
Mathematica 12.0.1 Kernel for Linux ARM (32-bit)
Copyright 1988-2019 Wolfram Research, Inc.

In[1]:=
Mathematica is also working as expected, again many thanks.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 2:20 pm
by DougieLawson
Interestingly running rpi-update in a chroot on a V7 system gets

Code: Select all

You appear to be using a custom kernel file.
Skipping installation of new kernel, as bundled dtb files may be incompatible with your kernel.
and blows up doing nothing useful.

I've sharpened my hacking pencil.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 2:24 pm
by jdonald
6by9 wrote:
Wed Sep 04, 2019 8:06 am
Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).
Got it. In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 2:27 pm
by trejan
jdonald wrote:
Wed Sep 04, 2019 2:24 pm
In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?
The Pi 4 uses the xhci driver for the 4 main USB ports as it is a USB 3 controller connected via PCIe.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 2:30 pm
by 6by9
jdonald wrote:
Wed Sep 04, 2019 2:24 pm
6by9 wrote:
Wed Sep 04, 2019 8:06 am
Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).
Got it. In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?
The Pi4 has a VLI, PCI-e based USB interface, which supports (amongst other things) USB3. USB3 will be significantly faster than the USB2 interface available via dwc_otg or dwc2.

The old dwc interface is still present on the SoC and is presented via the USB-C (power) connector as an option for supporting device mode (the VLI chip doesn't support that at all), but it is not intended to be in use on 99.99% of systems which only need USB host mode.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Wed Sep 04, 2019 2:32 pm
by dom
DougieLawson wrote:
Wed Sep 04, 2019 2:20 pm
Interestingly running rpi-update in a chroot on a V7 system gets

Code: Select all

You appear to be using a custom kernel file.
Skipping installation of new kernel, as bundled dtb files may be incompatible with your kernel.
Do you still have "kernel=xxx" set? That is what triggers this. It shouldn't be required now.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Posted: Thu Sep 05, 2019 6:18 pm
by 180lifer
Since Raspbian is being converted to 64-bit (woohoo!!) does that mean Raspbian Desktop for PCs will become 64-bit as well? I have had trouble getting it to run in a virtual machine using VMware Workstation Player because (if I remember right) the 32-bit binaries confused the VM since the underlying Debian OS was 64-bit.

Also, I sincerely hope that PiServer works with the Pi 4 in the not too distant future. PiServer really has a lot of promise and I’d love to see how the fast Pi 4 does with it.