dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5372
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 01, 2019 4:42 pm

f0rd0prefect wrote:
Fri Nov 01, 2019 12:38 pm
Is 'uname -a' the only method to detect the 64-Bit-Kernel?is.
arch?

ralphrmartin
Posts: 12
Joined: Sat May 06, 2017 3:42 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sat Nov 02, 2019 1:23 pm

Doesn't work for me. I tried adding arm_64bit=1 to config.txt, but the RPi4 just hangs waiting for /dev/sda2 during boot.
My current config boots off an extrnal SSD connected via a USB3 hub, using the cmdline.txt of
console=serial0,115200 console=tty1 root=/dev/sda2 rootfstype=ext4 fsck.repair=yes elevator=deadline rootwait net.ifnames=0 genet.skip_umac_reset=n

Using 4.19.75.

TenMM
Posts: 1
Joined: Thu Oct 24, 2019 11:59 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sat Nov 02, 2019 5:46 pm

I have 2 pi4's, one runs the 64 bit kernel the other won't. Both configured at appox. the same time, updated rpi one then the other. Added the arm_64bit=1 to the config.txt file then rebooted it. It rebooted then did the same to the other and it would not reboot. Used notepad++ to delete the arm_64bit=1 line off using my laptop and then it booted up no problem. I don't have any particular reason for upgrading to the 64 bit kernel just doing it for kicks and giggles.

Tichun
Posts: 70
Joined: Wed Jul 06, 2016 1:36 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sun Nov 03, 2019 8:24 pm

Is there an ETA for stable release?

Will we be able to run aarch64 instructions with 32 bit userland? (e.g. to learn armv8 without much hassle)
I can see there is no toolchain in repository, but would 64bit assembly work with say this 3rd party toolchain https://github.com/abhiTronix/raspberry ... s/releases?

Somebody mentioned above we would still need chroot to run 64bit userland code, is that true?
There is a blog post that doesn't really imply that - https://daltonmatos.wordpress.com/2012/ ... it-kernel/

Kendek
Posts: 128
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sun Nov 03, 2019 9:16 pm

Tichun wrote:
Sun Nov 03, 2019 8:24 pm
Is there an ETA for stable release?

Will we be able to run aarch64 instructions with 32 bit userland? (e.g. to learn armv8 without much hassle)
I can see there is no toolchain in repository, but would 64bit assembly work with say this 3rd party toolchain https://github.com/abhiTronix/raspberry ... s/releases?

Somebody mentioned above we would still need chroot to run 64bit userland code, is that true?
There is a blog post that doesn't really imply that - https://daltonmatos.wordpress.com/2012/ ... it-kernel/
The stable 64-bit kernel is already available in Raspbian, installed by default.
There are no 64-bit packages in the repository, so no libc6:arm64. The chroot method is practical for now.

jdonald
Posts: 417
Joined: Fri Nov 03, 2017 4:36 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sun Nov 03, 2019 9:34 pm

Tichun wrote:
Sun Nov 03, 2019 8:24 pm
There is a blog post that doesn't really imply [that a chroot is required] - https://daltonmatos.wordpress.com/2012/ ... it-kernel/
Thank you for the link. The examples there are specific to x86_64 (after all it's from early 2012) but probably can be made to work on the Pi and other armhf/aarch64 systems.

He goes over two methods: The second one (static linking) is mentioned somewhere in this thread, while the first one (running the non-default loader directly) is new to me and good to know. However, neither make for a convenient 64-bit development environment.

Somewhere in this thread I've mentioned a couple other methods like LXC or installing multiarch to make a FrankenDebian/Raspbian, but chroot or sakaki's raspbian-nspawn-64 remain the most straightforward.

Tichun
Posts: 70
Joined: Wed Jul 06, 2016 1:36 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Mon Nov 04, 2019 9:06 am

YouTube windowed 1080p playback is moderately slower, it used to drop 25-30% frames but without video stuttering or antyhing like that.
Now it hangs every few seconds and after few minutes pops up high temperature indicator.
That's on a pi3b.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24168
Joined: Sat Jul 30, 2011 7:41 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Mon Nov 04, 2019 9:24 am

Kendek wrote:
Sun Nov 03, 2019 9:16 pm
Tichun wrote:
Sun Nov 03, 2019 8:24 pm
Is there an ETA for stable release?

Will we be able to run aarch64 instructions with 32 bit userland? (e.g. to learn armv8 without much hassle)
I can see there is no toolchain in repository, but would 64bit assembly work with say this 3rd party toolchain https://github.com/abhiTronix/raspberry ... s/releases?

Somebody mentioned above we would still need chroot to run 64bit userland code, is that true?
There is a blog post that doesn't really imply that - https://daltonmatos.wordpress.com/2012/ ... it-kernel/
The stable 64-bit kernel is already available in Raspbian, installed by default.
There are no 64-bit packages in the repository, so no libc6:arm64. The chroot method is practical for now.
I'd regard it as a stable Beta.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

tasansal
Posts: 1
Joined: Fri Nov 08, 2019 3:52 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 08, 2019 3:58 pm

Kendek wrote:
Mon Oct 14, 2019 8:53 am
MikeDB wrote:
Sun Oct 13, 2019 9:57 pm
Seems rather convoluted (and probably needing a larger memory Pi4). I was hoping for something more along the lines of special command lines in Geany.
Basic steps for a simple 64-bit compilation:

Prepare the Ubuntu chroot environment:

Code: Select all

mkdir ubuntu
cd ubuntu
wget http://cdimage.ubuntu.com/ubuntu-base/daily/current/eoan-base-arm64.tar.gz
tar -xzf *.tar.gz
cat /etc/resolv.conf >etc/resolv.conf
chroot .
apt update
apt install locales-all build-essential
Then compile your program:

Code: Select all

export CFLAGS="-march=armv8-a+crc -mtune=cortex-a72"
export CXXFLAGS=$CFLAGS
export MAKEFLAGS="-j4"

make
This was very helpful, I successfully compiled OpenBLAS with OpenMP support using this method. However, if you are compiling anything that is multi-process or multi-threaded you need to let the chroot environment know by adding this before you go get into chroot, or else it compiles single core/threaded version:

Code: Select all

cat /proc/cpuinfo > proc/cpuinfo

dickon
Posts: 568
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 08, 2019 8:31 pm

Gods...

No.

Do:

Code: Select all

chroot $foo
mount -t proc proc /proc
mount -t sysfs sys /sys
mount -t devtmpfs udev /dev
mount -t devpts devpts /dev/pts
or similar. There may be some I've missed.

Some people prefer to bind mount from the root filesystem. It's personal preference.

acidtech
Posts: 1
Joined: Sat Nov 09, 2019 2:36 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sat Nov 09, 2019 2:39 am

I need to build a kernel module for the 64bit kernel. Is there an easier solution than building a cross compiler and completely building the kernel? I've got a chroot for pi64 with aarch64-gcc but I dont have kernel headers and the normal method of getting them doesnt apparently work.

Kendek
Posts: 128
Joined: Thu Jul 25, 2019 4:39 pm
Location: Kaposvár, Hungary

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sat Nov 09, 2019 2:59 pm

acidtech wrote:
Sat Nov 09, 2019 2:39 am
I need to build a kernel module for the 64bit kernel. Is there an easier solution than building a cross compiler and completely building the kernel? I've got a chroot for pi64 with aarch64-gcc but I dont have kernel headers and the normal method of getting them doesnt apparently work.
You can use the compatible kernel source instead of the kernel headers.
Just prepare before module compilation:

Code: Select all

make bcm2711_defconfig
make prepare scripts

RandyOo
Posts: 19
Joined: Sat Mar 12, 2016 8:16 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Nov 14, 2019 10:35 pm

Reporting in... I bought a 4GB Pi4 with the sole intent of having it perform the role of a small ZFS-based NAS, now that it has USB 3.0, gigabit ethernet, and lots more RAM. I had already managed to get ZFS up and running on Pi3, so I knew it could be done.

I managed to get ZFS compiled and running on the 32 bit kernel, but kept encountering killed tasks due to OOM (out-of-memory) errors while running a simple (but large) rsync between two ZFS pools, and even though swap was set up and present, it was unused. Seems to be some kind of manifestation of ZFS not being happy on a 32-bit kernel.

After conquering the learning curve of getting a chroot environment set up to compile arm64, and kernel source downloaded from git and built, I finally managed to get the ZFS module compiled for arm64, and subsequently installed.

Now, with the 64-bit kernel running, the memory issue is gone, and throughput on the rsync file copy between two USB 3.0 drives appears to have doubled! :D

Although I do enjoy learning, I'll look forward to the day when ZFS is just an apt-get install away... :lol:

dickon
Posts: 568
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Nov 14, 2019 10:52 pm

ZFS came from Solaris, and 64b Solaris at that. The Solaris kernel uses 64b virtual addressing extensively, which the Linux kernel does not, and as such, ZFS isn't really suited for 32b operation. You probably ran out of address space, which is easily done these days on a 32b system.

RossDv8
Posts: 243
Joined: Thu Jan 21, 2016 12:35 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Nov 14, 2019 11:35 pm

I just noticed this post again and realised I had forgotten I was running the 64-bit Colonel. Since I began using him I have changed to booting from microSD / running on SSD, and done a number of updates/upgrades.
All without issue, except that for a few weeks the Pi 4 suddenly began doing a normal reboot (something it had never been able to do from new on 32-bit) then a couple of full-upgrades ago, it reverted to having to power cycle it to complete a reboot.
So now, whenever I choose reboot from the menu, or do a sudo reboot, I have to pull the power and reapply it once the screen goes blue.

Because it had been doing that from new on a 32 bit kernel, I can;t blame it on the 64-bit. I did notice a 'firmware update' or two on the day it reverted to its old behaviour.
But for everything else the Pi 4 has been completely stable and running smoothly on the 64 bit kernel. No problems with any of the programs, which I imagine are all still 32-bit.
Remember, nobody is listening to you
until you fart ...

trejan
Posts: 891
Joined: Tue Jul 02, 2019 2:28 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 15, 2019 12:19 am

RossDv8 wrote:
Thu Nov 14, 2019 11:35 pm
All without issue, except that for a few weeks the Pi 4 suddenly began doing a normal reboot (something it had never been able to do from new on 32-bit) then a couple of full-upgrades ago, it reverted to having to power cycle it to complete a reboot.
So now, whenever I choose reboot from the menu, or do a sudo reboot, I have to pull the power and reapply it once the screen goes blue.

Because it had been doing that from new on a 32 bit kernel, I can;t blame it on the 64-bit. I did notice a 'firmware update' or two on the day it reverted to its old behaviour.
Have you recently changed USB devices or HATs? If one of them is backfeeding power then you get problems rebooting. The bootloader configuration defaults were changed to help with the HATs backfeeding 3.3V but USB devices will still cause problems AFAIK.

RossDv8
Posts: 243
Joined: Thu Jan 21, 2016 12:35 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 15, 2019 12:37 am

Thanks for that suggestion. Yes, at one time I had changed external USB drives. And I had actually suggested the backfeed power problem and a reply suggested I might be right. But after a firmware update, it all stared working. Then, as I mentioned in this post, after another update reboot stopped working again.
And during those events, no, none of the connected components have changed, The 3TB drive with the USB Splitter to power it is still plugged into the same unpowered USB 3 hub on one USB 3 port, and the SSD is still plugged into the other USB 3 port. The wifi keyboard donger is still plugged into a USB 2 port.

It is weird that suddenly reboot would start working and work perfectly for a few weeks, then just as suddenly revert to not working.
I'm used to it now, so to reboot I usually just do a shutdown, or if I accidentally choose reboot, I check the LEDs for activity then cycle the power.
The only time it is a nuisance is my auto update / upgrade script, which does a reboot. I now have to remember to manually upgrade software/OS.

EDIT
Since 'my' Pi has reverted to needing a power cycle to reboot, I unplugged the positive side of the splitter cable that powers the 3TB external drive and plugged it into the spare USB 2 port. There is still a 'head click' as it fires up, but not the click click click that usually happens when it does not have external power. So USB 3 is powering the data side of the splitter and into the external 3TB drive and the SSD, and USB 2 is powering the wifi donger and the positive side of the splitter cable.
I preferred it when it worked using external power to the splitter, but I can live with the data loss if the drive dies. For now, my shell scripts that execute a reboot are back to working again.

As for the 64-bit kernel? Very happy. For ordinary day to day stuff it seems great.
Last edited by RossDv8 on Fri Nov 15, 2019 11:13 pm, edited 1 time in total.
Remember, nobody is listening to you
until you fart ...

trejan
Posts: 891
Joined: Tue Jul 02, 2019 2:28 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 15, 2019 12:49 am

RossDv8 wrote:
Fri Nov 15, 2019 12:37 am
But after a firmware update, it all stared working. Then, as I mentioned in this post, after another update reboot stopped working again.
What does "sudo vcgencmd bootloader_config" say? If you changed or overwrote your bootloader configuration then that might explain why it was working then stopped.

If you do have something backfeeding 3.3V pin and the bootloader is set to turn off the PMIC entirely then it would hang at restart.
WAKE_ON_GPIO

If 1 then 'sudo halt' will run in a lower power mode until either GPIO3 or GLOBAL_EN are shorted to ground.

Default: 0 in original version of bootloader (2019-05-10). Newer bootloaders have this set to 1.
Version: All

POWER_OFF_ON_HALT

If 1 and WAKE_ON_GPIO=0 then switch off all PMIC outputs in halt. This is lowest possible power state for halt but may cause problems with some HATs because 5V will still be on. GLOBAL_EN must be shorted to ground to boot.

Default: 0
Version: 2019-07-15

RossDv8
Posts: 243
Joined: Thu Jan 21, 2016 12:35 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 15, 2019 1:07 am

This is the output of mine as Default

pi@raspberrypi:~ $ sudo vcgencmd bootloader_config
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
FREEZE_VERSION=0

I don;t use, nor am I likely to use a HAT. I wonder if perhaps I should have
WAKE_ON_GPIO=0
POWER_OFF_ON_HALT=1

And whether the Default is the reason my PI 4 never actually shuts down unless I switch the power OFF.
I always suspected it might be to do with drawing power for the fan from GPIO, and the Default setting more or less suggests it might be..
Remember, nobody is listening to you
until you fart ...

geoffr
Posts: 89
Joined: Wed Aug 22, 2012 11:25 am
Location: Melbourne, VIC

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Nov 15, 2019 6:38 am

Kendek wrote:
Sun Nov 03, 2019 9:16 pm
Tichun wrote:
Sun Nov 03, 2019 8:24 pm
Is there an ETA for stable release?

Will we be able to run aarch64 instructions with 32 bit userland? (e.g. to learn armv8 without much hassle)
I can see there is no toolchain in repository, but would 64bit assembly work with say this 3rd party toolchain https://github.com/abhiTronix/raspberry ... s/releases?

Somebody mentioned above we would still need chroot to run 64bit userland code, is that true?
There is a blog post that doesn't really imply that - https://daltonmatos.wordpress.com/2012/ ... it-kernel/
The stable 64-bit kernel is already available in Raspbian, installed by default.
There are no 64-bit packages in the repository, so no libc6:arm64. The chroot method is practical for now.
Possibly one of the simplest chroot methods would be to install lxc, and then create a container which uses the arm64 architecture. I realise that would be Debian, not Raspbian, but it may fit the bill for a lot of people. I haven't tried any development tools in an arm64 lxc container, but I have successfully created and run a few.

User avatar
MikeDB
Posts: 163
Joined: Sun Oct 12, 2014 8:27 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sun Nov 24, 2019 11:04 pm

Thought I'd give the 64 bit kernel another try so loaded it up on a fresh SD card and also did rpi-update to get the very latest version.

Noticeable that Chromium runs faster and doesn't crash (yet?).

However the Pi4 is running 8 degrees hotter than it does with the 32bit kernel. This is with the latest bootloader with the USB fix which reduced the temperature of the 32 bit version by about 2 degrees.

Anybody else seeing it run hotter ?

RossDv8
Posts: 243
Joined: Thu Jan 21, 2016 12:35 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Mon Nov 25, 2019 3:00 am

What sort of temps are you getting? Mine ran hot on a 128GB Samsing EVO Plus microSD, but is back to averaging 45-48 (by the display in the panel next to the clock) when running on SSD. If I switch back to a microSD, my temps are up closer to 60 again at times.

This of course, is with a fan. Of course, all freezes and Chromium problems stopped as soon as I began running Raspbian from SSD.

E D I T for MikeD8 info

Running off the SSD using VLC, streaming https://www.youtube.com/watch?v=sUd4PTIzm6s, on a UHD TV, mine is curently sneaking up to 50-52 deg Celsius.
But for most other stuff, it is sitting below 50.

Ambient room temp is only 32 deg Cels. So fairly cool today.
Last edited by RossDv8 on Tue Nov 26, 2019 1:28 am, edited 1 time in total.
Remember, nobody is listening to you
until you fart ...

User avatar
MikeDB
Posts: 163
Joined: Sun Oct 12, 2014 8:27 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Mon Nov 25, 2019 11:42 am

RossDv8 wrote:
Mon Nov 25, 2019 3:00 am
What sort of temps are you getting? Mine ran hot on a 128GB Samsing EVO Plus microSD, but is back to averaging 45-48 (by the display in the panel next to the clock) when running on SSD. If I switch back to a microSD, my temps are up closer to 60 again at times.

This of course, is with a fan. Of course, all freezes and Chromium problems stopped as soon as I began running Raspbian from SSD.
Yes mine is with a fan as well. My first job was with HP and Bill Hewlett had set a rule years before that we never went 20 deg above ambient on any component so I still try to keep to that even though components have of course got much much more reliable.

With the 32bit kernel I'm running at 42degC but with 64bit it's at 50degC doing the same amount of work which is three of the four cores running flat out producing audio and the fourth doing Linux housekeeping. I'll try with an SSD as you suggest and see if that is also different from 32 to 64 bits.

oicfar
Posts: 23
Joined: Tue Sep 17, 2019 7:39 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Mon Nov 25, 2019 4:37 pm

Hi,

which max. memory size is correct? 3.86G (raspbian lite 32 bit) or 3.77G (raspbian lite 64 bit)?
memory_32_vs_64.jpg
memory_32_vs_64.jpg (17.1 KiB) Viewed 535 times

User avatar
MikeDB
Posts: 163
Joined: Sun Oct 12, 2014 8:27 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Mon Nov 25, 2019 4:41 pm

oicfar wrote:
Mon Nov 25, 2019 4:37 pm
Hi,

which max. memory size is correct? 3.86G (raspbian lite 32 bit) or 3.77G (raspbian lite 64 bit)?

memory_32_vs_64.jpg
Surely they both are. Anything 64bits tends to use a little more memory

Return to “Advanced users”