dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5370
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Tue Sep 03, 2019 11:56 pm

As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.
This is for bleeding edge testers. Things may not work. Backing up, or using a fresh sdcard is a wise precaution.

To get the kernel run the usual

Code: Select all

sudo rpi-update
to get the latest kernel. If you want to switch to 64-bit kernel add to config.txt

Code: Select all

arm_64bit=1
You can still the usual 32-bit raspbian buster image just fine, but the kernel will be 64-bit.
Installing and running 64-bit packages should now be possible.

The 64-bit kernel (even with 32-bit raspbian) should have some performance advantages with kernel heavy loads (e.g. usb/ethernet) as we avoid some of the page mapping costs of LPAE. There may also be some (lesser) performance boost from the 64-bit kernel code.

Feel free to try it. Report any tests that run faster (or slower) and anything that does or doesn't work.

Note: We're focusing on Pi4 here. In theory the kernel does run on Pi3, but that's not our priority.
The lack of FIQ on 64-bit kernel will reduce USB performance on Pi3 (Pi4 doesn't require the FIQ).

Code: Select all

pi@pi4:~ $ uname -a
Linux domnfs 4.19.69-v8+ #1261 SMP PREEMPT Tue Sep 3 20:27:16 BST 2019 aarch64 GNU/Linux
Known issues:
vcsm driver doesn't work with 64-bit (only the cma version) which will break software decode with kodi
Mathematica and Wolfram fail to run (detection of raspberry pi os fails) Workaround here
Java 8 doesn't work. OpenJDK 11 does work.

jdonald
Posts: 417
Joined: Fri Nov 03, 2017 4:36 pm

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 12:41 am

This is awesome.
dom wrote:
Tue Sep 03, 2019 11:56 pm
performance advantages with kernel heavy loads (e.g. usb/ethernet)
Is the FIQ issue with dwc_otg fixed? Not sure which GitHub tickets track that issue but last I saw detailed mention of it is in rst's post.

If this is unresolved, my impression is that it breaks plenty of USB webcams and other devices transferring in bulk are slower than their 32-bit kernel behavior? If so that would make it hard to properly benchmark kernel-heavy USB loads.

First thought for anyone doing ethernet benchmarking: you must use unencrypted rsync/ftp/rsh/http or else the cipher will bottleneck on the CPU in userland.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7555
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 8:06 am

jdonald wrote:
Wed Sep 04, 2019 12:41 am
This is awesome.
dom wrote:
Tue Sep 03, 2019 11:56 pm
performance advantages with kernel heavy loads (e.g. usb/ethernet)
Is the FIQ issue with dwc_otg fixed? Not sure which GitHub tickets track that issue but last I saw detailed mention of it is in rst's post.
Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).
As noted in the title, this is mainly aimed at pi4.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

epoch1970
Posts: 3873
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 9:00 am

dom wrote:
Tue Sep 03, 2019 11:56 pm
As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.
Great news. Well done!
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24149
Joined: Sat Jul 30, 2011 7:41 pm

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 9:27 am

epoch1970 wrote:
Wed Sep 04, 2019 9:00 am
dom wrote:
Tue Sep 03, 2019 11:56 pm
As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.
Great news. Well done!
A lot of work has been done by the community and upstream devs, so very much a combined effort.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
DougieLawson
Posts: 36567
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 10:24 am

Can you remind me, which Raspberries does 64 bit work on? I know 3B+ and 4Bs are OK, as I've run Sakaki-san's Gentoo on my 3B+ (back in February).

What's the revision numbers for the 64-bit capable boards?
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
DougieLawson
Posts: 36567
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 10:41 am

This doesn't work on a 3A+ with

Code: Select all

arm_64bit=1
but does work OK with

Code: Select all

kernel=kernel8.img
.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

epoch1970
Posts: 3873
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 10:44 am

DougieLawson wrote:
Wed Sep 04, 2019 10:24 am
Can you remind me, which Raspberries does 64 bit work on? I know 3B+ and 4Bs are OK, as I've run Sakaki-san's Gentoo on my 3B+ (back in February).
Pi 3B is ok, I suppose Pi 2 rev. 1.2 (with the same SOC as 3B) runs in 64bit mode as well.
A good year (or is it more...) ago I had a 64b buildroot OS working just fine on my Pi 3Bs (linux tree off RPF's Github repo), but I switched back to 32 because goodies like vcgen (?) or some GPIO libs were not available. The full Pi "experience" was not there yet.
Last edited by epoch1970 on Wed Sep 04, 2019 11:12 am, edited 1 time in total.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24149
Joined: Sat Jul 30, 2011 7:41 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 10:56 am

As Dom said, this focuses on Pi4, although MAY work on other Pi's. I've changed the title to reflect that.

For revision codes that say which processor you have, see here...https://www.raspberrypi.org/documentati ... /README.md
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

geev03
Posts: 135
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 11:07 am

dom wrote:
Tue Sep 03, 2019 11:56 pm
....
You can still the usual 32-bit raspbian buster image just fine, but the kernel will be 64-bit.
Installing and running 64-bit packages should now be possible.
......
Feel free to try it. Report any tests that run faster (or slower) and anything that does or doesn't work.

.....

Code: Select all

pi@pi4:~ $ uname -a
Linux domnfs 4.19.69-v8+ #1261 SMP PREEMPT Tue Sep 3 20:27:16 BST 2019 aarch64 GNU/Linux
Known issues:
vcsm driver doesn't work with 64-bit (only the cma version) which will break software decode with kodi

The 64 bit kernel shows up and remote access is working.
- Web browser starts and works
- Mathematica and Wolfram are not showing up :(

Code: Select all

pi@raspberrypi:/boot $ sudo rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** We're running for the first time
 *** Backing up files (this will take a few minutes)
 *** Remove old firmware backup
 *** Backing up firmware
 *** Remove old modules backup
 *** Backing up modules 4.19.66-v7l+
PARTSIZE:268435968
#############################################################
WARNING: 'rpi-update' updates to pre-releases of the linux
kernel tree and Videocore firmware.

'rpi-update' should only be used if there is a specific
reason to do so - for example, a request by a Raspberry Pi
engineer.

DO NOT use 'rpi-update' as part of a regular update process.

##############################################################
Would you like to proceed? (y/N)
 *** Downloading specific firmware revision (this will take a few minutes)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168    0   168    0     0    800      0 --:--:-- --:--:-- --:--:--   800
100  113M  100  113M    0     0  1292k      0  0:01:29  0:01:29 --:--:-- 2859k
 *** Updating firmware
 *** Updating kernel modules
 *** depmod 4.19.69-v8+
 *** depmod 4.19.69-v7+
 *** depmod 4.19.69-v7l+
 *** depmod 4.19.69+
 *** Updating VideoCore libraries
 *** Using HardFP libraries
 *** Updating SDK
 *** Running ldconfig
 *** Storing current firmware revision
 *** Deleting downloaded files
 *** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to 1b797509b8c28cba7834684986b97feb1a265e9c
 *** A reboot is needed to activate the new firmware
pi@raspberrypi:/boot $
Attachments
raspbian_64bit_no_mathematica.jpg
raspbian_64bit_no_mathematica.jpg (139.28 KiB) Viewed 21763 times

User avatar
DougieLawson
Posts: 36567
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 11:18 am

jamesh wrote:
Wed Sep 04, 2019 10:56 am
For revision codes that say which processor you have, see here...https://www.raspberrypi.org/documentati ... /README.md
I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.

I still don't have a RPi4B 4GB because the household finance director doesn't leave for her three months in Australia until Oct 7th.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
Gavinmc42
Posts: 4042
Joined: Wed Aug 28, 2013 3:31 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 11:28 am

Just threw some free pascal opengl at it, works fine :D
Lazarus and Ultibo IDEs working.
I still don't have a RPi4B 4GB because the household finance director doesn't leave for her three months in Australia until Oct 7th.
Oh, thanks for the reminder, Pi4B4's are due in Sep here in Oz :D
Oh no, that first pre order batch must have sold out, next lot 14th Oct :(

Ok, now for the serious stuff, see if it works with the stuff I threw at Gentoo64.
Overnight compiles etc
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5370
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 11:36 am

DougieLawson wrote:
Wed Sep 04, 2019 10:41 am
This doesn't work on a 3A+ with

Code: Select all

arm_64bit=1
but does work OK with

Code: Select all

kernel=kernel8.img
.
The requirement for kernel=kernel8.img was mentioned in first post.
I'm just building a firmware that will go out later that make that no longer necessary.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5370
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 11:41 am

DougieLawson wrote:
Wed Sep 04, 2019 11:18 am
I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.
You need BCM2837 (PI3/Pi3+) or BCM2838 (Pi4).
Later models of Pi2 switched to using BCM2837, but the majority sold will be BCM2836 (so no 64-bit capability)

The revision code is /proc/cpuinfo is the definitive information.

User avatar
DougieLawson
Posts: 36567
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 11:46 am

dom wrote:
Wed Sep 04, 2019 11:41 am
DougieLawson wrote:
Wed Sep 04, 2019 11:18 am
I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.
You need BCM2837 (PI3/Pi3+) or BCM2838 (Pi4).
Later models of Pi2 switched to using BCM2837, but the majority sold will be BCM2836 (so no 64-bit capability)

The revision code is /proc/cpuinfo is the definitive information.
Thank you. One 3A+ is running already, the 3Bs/3B+ are just waiting for the rpi-update to finish.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5370
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 12:20 pm

dom wrote:
Wed Sep 04, 2019 11:36 am
The requirement for kernel=kernel8.img was mentioned in first post.
I'm just building a firmware that will go out later that make that no longer necessary.
Firmware pushed. arm_64bit=1 is all you need to enable 64-bit kernel on Pi3 too now.

geev03
Posts: 135
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: Pi4 64-bit raspbian kernel for testing

Wed Sep 04, 2019 1:01 pm

dom wrote:
Wed Sep 04, 2019 12:20 pm
....

Firmware pushed. arm_64bit=1 is all you need to enable 64-bit kernel on Pi3 too now.

Code: Select all

*** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to c472ab43407d3bf1ed162a9f7c3f5060e6f99d14
 *** A reboot is needed to activate the new firmware
OK, the same Pi4B sd card works on Pi3B, ( with the same problems with some of the programs that worked fine on 32 bit buster)
Attachments
pi3_buster_64bit_kernel.jpg
pi3_buster_64bit_kernel.jpg (126.93 KiB) Viewed 21606 times

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5370
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 1:18 pm

geev03 wrote:
Wed Sep 04, 2019 11:07 am
- Mathematica and Wolfram are not showing up :(
If you are daring, edit /opt/Wolfram/WolframEngine/12.0/Executables/mathematica
find lines:

Code: Select all

				armv?l)
					SystemIDList="Linux-ARM";;
and add after:

Code: Select all

				aarch64)
					SystemIDList="Linux-ARM";;
You can also make a comparable change to /opt/Wolfram/WolframEngine/12.0/Executables/wolfram

geev03
Posts: 135
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 1:48 pm

dom wrote:
Wed Sep 04, 2019 1:18 pm
..

If you are daring, edit /opt/Wolfram/WolframEngine/12.0/Executables/mathematica
find lines:

Code: Select all

				armv?l)
					SystemIDList="Linux-ARM";;
and add after:

Code: Select all

				aarch64)
					SystemIDList="Linux-ARM";;
You can also make a comparable change to /opt/Wolfram/WolframEngine/12.0/Executables/wolfram
Thanks a lot, perfect.

Code: Select all

pi@raspberrypi:~ $ sudo wolfram
Mathematica 12.0.1 Kernel for Linux ARM (32-bit)
Copyright 1988-2019 Wolfram Research, Inc.

In[1]:=
Mathematica is also working as expected, again many thanks.

User avatar
DougieLawson
Posts: 36567
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 2:20 pm

Interestingly running rpi-update in a chroot on a V7 system gets

Code: Select all

You appear to be using a custom kernel file.
Skipping installation of new kernel, as bundled dtb files may be incompatible with your kernel.
and blows up doing nothing useful.

I've sharpened my hacking pencil.
Last edited by DougieLawson on Wed Sep 04, 2019 2:25 pm, edited 1 time in total.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

jdonald
Posts: 417
Joined: Fri Nov 03, 2017 4:36 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 2:24 pm

6by9 wrote:
Wed Sep 04, 2019 8:06 am
Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).
Got it. In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?

trejan
Posts: 880
Joined: Tue Jul 02, 2019 2:28 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 2:27 pm

jdonald wrote:
Wed Sep 04, 2019 2:24 pm
In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?
The Pi 4 uses the xhci driver for the 4 main USB ports as it is a USB 3 controller connected via PCIe.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7555
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 2:30 pm

jdonald wrote:
Wed Sep 04, 2019 2:24 pm
6by9 wrote:
Wed Sep 04, 2019 8:06 am
Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).
Got it. In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?
The Pi4 has a VLI, PCI-e based USB interface, which supports (amongst other things) USB3. USB3 will be significantly faster than the USB2 interface available via dwc_otg or dwc2.

The old dwc interface is still present on the SoC and is presented via the USB-C (power) connector as an option for supporting device mode (the VLI chip doesn't support that at all), but it is not intended to be in use on 99.99% of systems which only need USB host mode.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5370
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Wed Sep 04, 2019 2:32 pm

DougieLawson wrote:
Wed Sep 04, 2019 2:20 pm
Interestingly running rpi-update in a chroot on a V7 system gets

Code: Select all

You appear to be using a custom kernel file.
Skipping installation of new kernel, as bundled dtb files may be incompatible with your kernel.
Do you still have "kernel=xxx" set? That is what triggers this. It shouldn't be required now.

180lifer
Posts: 5
Joined: Sun Feb 04, 2018 9:13 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Sep 05, 2019 6:18 pm

Since Raspbian is being converted to 64-bit (woohoo!!) does that mean Raspbian Desktop for PCs will become 64-bit as well? I have had trouble getting it to run in a virtual machine using VMware Workstation Player because (if I remember right) the 32-bit binaries confused the VM since the underlying Debian OS was 64-bit.

Also, I sincerely hope that PiServer works with the Pi 4 in the not too distant future. PiServer really has a lot of promise and I’d love to see how the fast Pi 4 does with it.

Return to “Advanced users”