User avatar
rpdom
Posts: 15199
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Sep 05, 2019 6:42 pm

180lifer wrote:
Thu Sep 05, 2019 6:18 pm
Since Raspbian is being converted to 64-bit (woohoo!!) does that mean Raspbian Desktop for PCs will become 64-bit as well?
The Raspberry Pi Desktop for PCs already uses a 64-bit kernel. It just uses 32-bit userland code, exactly the same as Raspbian will with the new kernel.

We'll have to wait for a full 64-bit release.

pica200
Posts: 138
Joined: Tue Aug 06, 2019 10:27 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Sep 05, 2019 7:55 pm

Ah, finally some 64 bit love! Thank you :) Technically i have been using this kernel already since weeks as part of a different distro (they are just building it from the linux fork on Github) and did not notice any kernel bug yet but the drivers are still pretty rough at the edges.

andrum99
Posts: 807
Joined: Fri Jul 20, 2012 2:41 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Sep 05, 2019 8:05 pm

180lifer wrote:
Thu Sep 05, 2019 6:18 pm
Since Raspbian is being converted to 64-bit (woohoo!!) does that mean Raspbian Desktop for PCs will become 64-bit as well? I have had trouble getting it to run in a virtual machine using VMware Workstation Player because (if I remember right) the 32-bit binaries confused the VM since the underlying Debian OS was 64-bit.
What are you using as the host OS for VMware Workstation Player? I've tested VMware Workstation Pro using Windows 10 Pro 64-bit as the host OS, and Raspberry Pi Desktop for PCs runs fine on that. I've not tested Vmware Workstation Player but I would expect it to be the same on that.

smist08
Posts: 1
Joined: Thu Sep 05, 2019 8:20 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Sep 05, 2019 8:24 pm

Doesn't work for me. Raspberry Pi 4. If I set either arm_64bit=1 or kernel=kernel8.img then the screen goes black after the daemon load screen and the desktop never appears. The green light is out on the Pi. Only the red light on with no other sign of activity. I think an error message flashed quickly, but I couldn't read it.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23671
Joined: Sat Jul 30, 2011 7:41 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Sep 05, 2019 9:49 pm

smist08 wrote:
Thu Sep 05, 2019 8:24 pm
Doesn't work for me. Raspberry Pi 4. If I set either arm_64bit=1 or kernel=kernel8.img then the screen goes black after the daemon load screen and the desktop never appears. The green light is out on the Pi. Only the red light on with no other sign of activity. I think an error message flashed quickly, but I couldn't read it.
What do you have plugged in?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23671
Joined: Sat Jul 30, 2011 7:41 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Thu Sep 05, 2019 9:50 pm

pica200 wrote:
Thu Sep 05, 2019 7:55 pm
Ah, finally some 64 bit love! Thank you :) Technically i have been using this kernel already since weeks as part of a different distro (they are just building it from the linux fork on Github) and did not notice any kernel bug yet but the drivers are still pretty rough at the edges.
Technically, commits for 64bit support have been going in for years...
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5331
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 11:00 am

smist08 wrote:
Thu Sep 05, 2019 8:24 pm
Doesn't work for me. Raspberry Pi 4. If I set either arm_64bit=1 or kernel=kernel8.img then the screen goes black after the daemon load screen and the desktop never appears. The green light is out on the Pi. Only the red light on with no other sign of activity. I think an error message flashed quickly, but I couldn't read it.
Without arm_64bit=1 what does "vcgencmd version", "uname -a", "vcgencmd get_config int" report?
To catch a brief kernel log message you can record a video from a phone and then pause it on playback.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5963
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 11:18 am

I left chromium compiling for a few days to see if there are any stability issues using the pi for building large packages. No issues so far and it's finally able to build without OOMing while linking.

I was worried the 4GB per process limit may still be there with a 32 bit userland, but no that's gone.

User avatar
Gavinmc42
Posts: 3736
Joined: Wed Aug 28, 2013 3:31 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 11:35 am

No issues so far and it's finally able to build without OOMing while linking.
No problem compiling Chromium on a Pi4B4?
I suppose Firefox has a chance too?
Pi development on Pi's, that's desktop replacement :lol:

DistCC compile it on a Pi4 cluster?
Did you use a 64bit compiler and optimize for Chromium A72?
Do you need to recompile the Compiler for A72?
How much faster would it compile Chromium then?

Just copy the Gentoo GCC binaries across to 64bit buster?
I need to take time off to try all this new stuff Pi4's bring to the mix.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

geev03
Posts: 121
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 11:50 am

Finding it difficult to run Docker :(
"WARN[2019-09-06T12:41:11.718042745+01:00] Your kernel does not support swap memory limit
WARN[2019-09-06T12:41:11.718103577+01:00] Your kernel does not support cgroup cfs period
WARN[2019-09-06T12:41:11.718179780+01:00] Your kernel does not support cgroup cfs quotas
WARN[2019-09-06T12:41:11.718201428+01:00] Your kernel does not support cgroup rt period
WARN[2019-09-06T12:41:11.718221576+01:00] Your kernel does not support cgroup rt runtime"

Code: Select all

login as: pi
pi@192.168.1.171's password:
Linux raspberrypi 4.19.69-v8+ #1261 SMP PREEMPT Tue Sep 3 20:27:16 BST 2019 aarch64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Sep  6 12:07:50 2019
pi@raspberrypi:~ $ neofetch
  `.::///+:/-.        --///+//-:``    pi@raspberrypi
 `+oooooooooooo:   `+oooooooooooo:    --------------
  /oooo++//ooooo:  ooooo+//+ooooo.    OS: Raspbian GNU/Linux 10 (buster) aarch6
  `+ooooooo:-:oo-  +o+::/ooooooo:     Host: Raspberry Pi 4 Model B Rev 1.1
   `:oooooooo+``    `.oooooooo+-      Kernel: 4.19.69-v8+
     `:++ooo/.        :+ooo+/.`       Uptime: 31 mins
        ...`  `.----.` ``..           Packages: 2135 (dpkg)
     .::::-``:::::::::.`-:::-`        Shell: bash 5.0.3
    -:::-`   .:::::::-`  `-:::-       Terminal: /dev/pts/1
   `::.  `.--.`  `` `.---.``.::`      CPU: (4) @ 1.500GHz
       .::::::::`  -::::::::` `       Memory: 300MiB / 3808MiB
 .::` .:::::::::- `::::::::::``::.
-:::` ::::::::::.  ::::::::::.`:::-
::::  -::::::::.   `-::::::::  ::::
-::-   .-:::-.``....``.-::-.   -::-
 .. ``       .::::::::.     `..`..
   -:::-`   -::::::::::`  .:::::`
   :::::::` -::::::::::` :::::::.
   .:::::::  -::::::::. ::::::::
    `-:::::`   ..--.`   ::::::.
      `...`  `...--..`  `...`
            .::::::::::
             `.-::::-`

pi@raspberrypi:~ $ docker -v
Docker version 19.03.2, build 6a30dfc
pi@raspberrypi:~ $ docker run -it alpine /bin/sh
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
pi@raspberrypi:~ $ sudo -i
root@raspberrypi:~# /etc/init.d/docker start
-bash: /etc/init.d/docker: No such file or directory
root@raspberrypi:~# cd /home/pi/
root@raspberrypi:/home/pi# cd docker/
root@raspberrypi:/home/pi/docker# ls
containerd       ctr     dockerd      docker-proxy
containerd-shim  docker  docker-init  runc
root@raspberrypi:/home/pi/docker# dockerd
INFO[2019-09-06T12:41:10.727860351+01:00] Starting up                           
INFO[2019-09-06T12:41:10.780312236+01:00] libcontainerd: started new containerd process  pid=2070
INFO[2019-09-06T12:41:10.780489104+01:00] parsed scheme: "unix"                         module=grpc
INFO[2019-09-06T12:41:10.780527437+01:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2019-09-06T12:41:10.780623047+01:00] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] }  module=grpc
INFO[2019-09-06T12:41:10.780657194+01:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2019-09-06T12:41:10.803008062+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x4000720600, CONNECTING  module=grpc
INFO[2019-09-06T12:41:11.493908944+01:00] starting containerd                           revision=894b81a4b802e4eb2a91d1ce216b8817763c29fb version=v1.2.6
INFO[2019-09-06T12:41:11.494844912+01:00] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
INFO[2019-09-06T12:41:11.495630901+01:00] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
WARN[2019-09-06T12:41:11.496083024+01:00] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
INFO[2019-09-06T12:41:11.496132505+01:00] loading plugin "io.containerd.snapshotter.v1.aufs"...  type=io.containerd.snapshotter.v1
WARN[2019-09-06T12:41:11.509403909+01:00] failed to load plugin io.containerd.snapshotter.v1.aufs  error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.19.69-v8+\n": exit status 1"
INFO[2019-09-06T12:41:11.509561536+01:00] loading plugin "io.containerd.snapshotter.v1.native"...  type=io.containerd.snapshotter.v1
INFO[2019-09-06T12:41:11.510652262+01:00] loading plugin "io.containerd.snapshotter.v1.overlayfs"...  type=io.containerd.snapshotter.v1
INFO[2019-09-06T12:41:11.511563730+01:00] loading plugin "io.containerd.snapshotter.v1.zfs"...  type=io.containerd.snapshotter.v1
WARN[2019-09-06T12:41:11.512127685+01:00] failed to load plugin io.containerd.snapshotter.v1.zfs  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
INFO[2019-09-06T12:41:11.512211332+01:00] loading plugin "io.containerd.metadata.v1.bolt"...  type=io.containerd.metadata.v1
WARN[2019-09-06T12:41:11.512325034+01:00] could not use snapshotter zfs in metadata plugin  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
WARN[2019-09-06T12:41:11.512375719+01:00] could not use snapshotter btrfs in metadata plugin  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
WARN[2019-09-06T12:41:11.512420811+01:00] could not use snapshotter aufs in metadata plugin  error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.19.69-v8+\n": exit status 1"
INFO[2019-09-06T12:41:11.516935024+01:00] loading plugin "io.containerd.differ.v1.walking"...  type=io.containerd.differ.v1
INFO[2019-09-06T12:41:11.517084652+01:00] loading plugin "io.containerd.gc.v1.scheduler"...  type=io.containerd.gc.v1
INFO[2019-09-06T12:41:11.517209687+01:00] loading plugin "io.containerd.service.v1.containers-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.517316500+01:00] loading plugin "io.containerd.service.v1.content-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.517417499+01:00] loading plugin "io.containerd.service.v1.diff-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.517487757+01:00] loading plugin "io.containerd.service.v1.images-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.517568219+01:00] loading plugin "io.containerd.service.v1.leases-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.517631126+01:00] loading plugin "io.containerd.service.v1.namespaces-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.517693810+01:00] loading plugin "io.containerd.service.v1.snapshots-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.517753087+01:00] loading plugin "io.containerd.runtime.v1.linux"...  type=io.containerd.runtime.v1
INFO[2019-09-06T12:41:11.518089045+01:00] loading plugin "io.containerd.runtime.v2.task"...  type=io.containerd.runtime.v2
INFO[2019-09-06T12:41:11.518266728+01:00] loading plugin "io.containerd.monitor.v1.cgroups"...  type=io.containerd.monitor.v1
INFO[2019-09-06T12:41:11.527491597+01:00] loading plugin "io.containerd.service.v1.tasks-service"...  type=io.containerd.service.v1
INFO[2019-09-06T12:41:11.527639558+01:00] loading plugin "io.containerd.internal.v1.restart"...  type=io.containerd.internal.v1
INFO[2019-09-06T12:41:11.527851240+01:00] loading plugin "io.containerd.grpc.v1.containers"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.527900813+01:00] loading plugin "io.containerd.grpc.v1.content"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.527976960+01:00] loading plugin "io.containerd.grpc.v1.diff"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.528017386+01:00] loading plugin "io.containerd.grpc.v1.events"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.528062681+01:00] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.528102551+01:00] loading plugin "io.containerd.grpc.v1.images"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.528144458+01:00] loading plugin "io.containerd.grpc.v1.leases"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.528183661+01:00] loading plugin "io.containerd.grpc.v1.namespaces"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.528221327+01:00] loading plugin "io.containerd.internal.v1.opt"...  type=io.containerd.internal.v1
INFO[2019-09-06T12:41:11.529996709+01:00] loading plugin "io.containerd.grpc.v1.snapshots"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.530059727+01:00] loading plugin "io.containerd.grpc.v1.tasks"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.530103967+01:00] loading plugin "io.containerd.grpc.v1.version"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.530157855+01:00] loading plugin "io.containerd.grpc.v1.introspection"...  type=io.containerd.grpc.v1
INFO[2019-09-06T12:41:11.532319880+01:00] serving...                                    address="/var/run/docker/containerd/containerd-debug.sock"
INFO[2019-09-06T12:41:11.532476378+01:00] serving...                                    address="/var/run/docker/containerd/containerd.sock"
INFO[2019-09-06T12:41:11.532523525+01:00] containerd successfully booted in 0.050716s
INFO[2019-09-06T12:41:11.562227289+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x4000720600, READY  module=grpc
INFO[2019-09-06T12:41:11.587592837+01:00] parsed scheme: "unix"                         module=grpc
INFO[2019-09-06T12:41:11.587663613+01:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2019-09-06T12:41:11.587715853+01:00] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] }  module=grpc
INFO[2019-09-06T12:41:11.587763149+01:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2019-09-06T12:41:11.587914999+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x40007210e0, CONNECTING  module=grpc
INFO[2019-09-06T12:41:11.589117408+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x40007210e0, READY  module=grpc
INFO[2019-09-06T12:41:11.596055587+01:00] parsed scheme: "unix"                         module=grpc
INFO[2019-09-06T12:41:11.596128660+01:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2019-09-06T12:41:11.596220548+01:00] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] }  module=grpc
INFO[2019-09-06T12:41:11.596255343+01:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2019-09-06T12:41:11.596452766+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x40008ac050, CONNECTING  module=grpc
INFO[2019-09-06T12:41:11.596507877+01:00] blockingPicker: the picked transport is not ready, loop back to repick  module=grpc
INFO[2019-09-06T12:41:11.597214719+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x40008ac050, READY  module=grpc
INFO[2019-09-06T12:41:11.625481225+01:00] [graphdriver] using prior storage driver: overlay2
WARN[2019-09-06T12:41:11.718042745+01:00] Your kernel does not support swap memory limit
WARN[2019-09-06T12:41:11.718103577+01:00] Your kernel does not support cgroup cfs period
WARN[2019-09-06T12:41:11.718179780+01:00] Your kernel does not support cgroup cfs quotas
WARN[2019-09-06T12:41:11.718201428+01:00] Your kernel does not support cgroup rt period
WARN[2019-09-06T12:41:11.718221576+01:00] Your kernel does not support cgroup rt runtime
WARN[2019-09-06T12:41:11.718241742+01:00] mountpoint for pids not found         
INFO[2019-09-06T12:41:11.720720596+01:00] Loading containers: start.            
INFO[2019-09-06T12:41:12.125942402+01:00] stopping healthcheck following graceful shutdown  module=libcontainerd
INFO[2019-09-06T12:41:12.125983697+01:00] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
INFO[2019-09-06T12:41:12.127601878+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x40008ac050, TRANSIENT_FAILURE  module=grpc
INFO[2019-09-06T12:41:12.127736450+01:00] pickfirstBalancer: HandleSubConnStateChange: 0x40008ac050, CONNECTING  module=grpc
INFO[2019-09-06T12:41:12.128077075+01:00] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=moby
failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables: Operation not supported.
 (exit status 1)
root@raspberrypi:/home/pi/docker#

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5963
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 11:59 am

Gavinmc42 wrote:
No issues so far and it's finally able to build without OOMing while linking.
No problem compiling Chromium on a Pi4B4?
I suppose Firefox has a chance too?
Pi development on Pi's, that's desktop replacement :lol:

DistCC compile it on a Pi4 cluster?
Did you use a 64bit compiler and optimize for Chromium A72?
Do you need to recompile the Compiler for A72?
How much faster would it compile Chromium then?

Just copy the Gentoo GCC binaries across to 64bit buster?
I need to take time off to try all this new stuff Pi4's bring to the mix.
Not sure which questions need answering and which are rhetorical, but I'll give it a go.

This is a normal build of our chromium package we ship. It builds an armv6 version and an armv7 version. Optimisations for specific CPUs don't make a difference. We just build two versions for the hand written NEON routines, which make a huge difference.

In general, we get performance improvements by looking exactly where the clock cycles are going and optimising that, find where data is copied or processed unnecessarily , use hw acceleration where possible and so on. The compiler can only do so much and new instruction sets don't help.
https://github.com/bavison/skia/commits ... d_chrome72

You wouldn't need to build a new compiler, just pass the appropriate -march option.

No, I wouldn't expect to see any worthwhile performance improvements in gcc or llvm built specifically for a72, but I wouldn't rule it out without testing it.

No, you can't just copy binaries from a 64 bit system to a 32bit system. This is just the kernel, not userland. You'd need to copy all of the dependencies down to libc. At that point just use Debian arm64.

I don't think this affects firefox in any way. IIRC the issue with firefox were down to problems with Rust, but I never looked at it.

DistCC is not worth the hassle. Most packages take a few minutes to build and most of the time is spent by sbuild setting up the build environment, installing dependencies, running configure and so on, not the build itself. For large packages, ccache is enough.

User avatar
Gavinmc42
Posts: 3736
Joined: Wed Aug 28, 2013 3:31 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 12:07 pm

Not sure which questions need answering and which are rhetorical, but I'll give it a go.
Good answers :D
I keep forgetting you guys still do 32bit versions :lol:
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23671
Joined: Sat Jul 30, 2011 7:41 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 12:26 pm

Gavinmc42 wrote:
Fri Sep 06, 2019 12:07 pm
Not sure which questions need answering and which are rhetorical, but I'll give it a go.
Good answers :D
I keep forgetting you guys still do 32bit versions :lol:
Not funny. 27M users out there using the 32bit version. You are an outlier - don't think of yourself all the time, think of the user base. That's what we do.

And, please try and keep the brain dumps within reason.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
DougieLawson
Posts: 36124
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 12:49 pm

What's the -mcpu=? -mtune=? -march=? -mfp=? options for gcc on RPi4B that you're using?
Same question for RPI3B, RPi3B+ & RPi3A+.

Because https://gcc.gnu.org/onlinedocs/gcc-8.3. ... RM-Options is just a weeny bit confusing.

Do I need anything special in the way of libraries (or do I need to rebuild any) to get things running in 64-bit? [In which case I may need to switch to Sakaki-san's Gentoo64.]
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

MrQ
Posts: 6
Joined: Fri Sep 06, 2019 1:32 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 1:35 pm

I want to build my own 64 bit kernel. What are the differences in building procedure between 32 and 64 bit ? https://www.raspberrypi.org/documentati ... uilding.md

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5331
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 1:47 pm

MrQ wrote:
Fri Sep 06, 2019 1:35 pm
I want to build my own 64 bit kernel. What are the differences in building procedure between 32 and 64 bit ? https://www.raspberrypi.org/documentati ... uilding.md
Main one is replace ARCH=arm with ARCH=arm64.
CROSS_COMPILE=aarch64-linux-gnu- rather than CROSS_COMPILE=arm-linux-gnueabihf-
The kernel built will be arch/arm64/boot/Image (no zImage for 64-bit).

MrQ
Posts: 6
Joined: Fri Sep 06, 2019 1:32 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 2:39 pm

dom wrote:
Fri Sep 06, 2019 1:47 pm
MrQ wrote:
Fri Sep 06, 2019 1:35 pm
I want to build my own 64 bit kernel. What are the differences in building procedure between 32 and 64 bit ? https://www.raspberrypi.org/documentati ... uilding.md
Main one is replace ARCH=arm with ARCH=arm64.
CROSS_COMPILE=aarch64-linux-gnu- rather than CROSS_COMPILE=arm-linux-gnueabihf-
The kernel built will be arch/arm64/boot/Image (no zImage for 64-bit).
I don't cross compile. Just building directly on my RPI4.

jdonald
Posts: 413
Joined: Fri Nov 03, 2017 4:36 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 3:47 pm

ShiftPlusOne wrote:
Fri Sep 06, 2019 11:59 am
This is a normal build of our chromium package we ship. It builds an armv6 version and an armv7 version.
Awesome, I didn't realize you were doing this since the Pi 2B v1.1.
On Pi, I thought Skia was only used by default on Firefox now. If you go to chrome://gpu on Buster it says Skia Renderer: Disabled
In general, we get performance improvements by looking exactly where the clock cycles are going and optimising that, find where data is copied or processed unnecessarily , use hw acceleration where possible and so on. The compiler can only do so much and new instruction sets don't help.
...
Optimisations for specific CPUs don't make a difference.
Profiling and optimizing is the right way to go for double-digit gains. However, I'm skeptical as to outputting instructions specific to higher-end CPU cores not making a difference.

See this lzma benchmarking which covers tuning in both 32-bit and 64-bit. Going from Cortex-A9 to Cortex-A7/A53/A72 instructions consistently adds 2% even in the 32-bit case. I don't know if it's due to integer division, IFTTT instructions, or other features lacking in Cortex-A9, but it's measurable. (Edit: Misremembered my old benchmarks. It's ARMv6 -> ARMv7 (Cortex-A9) that gives 2%. Cortex-A9 -> Cortex-A72 boosts another 7%!)

Benchmarks vary a lot but I believe this is closer to the median improvement rather than an outlier. Recall that sysbench-cpu gets a 10x speedup in 32-bit with -march=armv8-a+crc+simd -mtune=cortex-a72 (integer division).

I have a hard time telling exactly what flags chromium-browser-v7 was built with (doesn't seem to have an about:buildconfig option like Firefox). From reading v74.0.3729.182_mmal_3_22.patch I suspect that it's doing -march=armv7-a, which would tune for Cortex-A9 and miss out on features starting with Cortex-A7. Ideally the chromium-browser-armv7 build should be using -march=armv8-a+crc+simd -mtune=cortex-a7 to match the lowest-end ARMv7 processor used in the Pi 2.

(Agreed with DougieLawson on how the GCC tuning args are confusing, such as -march=armv8-a+crc+simd -mtune=cortex-a7 required in 32-bit while -march=armv7-a -mtune=cortex-a7 will not do what one expects.)

I recognize I'm a bit off-topic here, but this is plenty interesting.

andrum99
Posts: 807
Joined: Fri Jul 20, 2012 2:41 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 4:11 pm

jamesh wrote:
Fri Sep 06, 2019 12:26 pm
Gavinmc42 wrote:
Fri Sep 06, 2019 12:07 pm
I keep forgetting you guys still do 32bit versions :lol:
Not funny.
Seconded. 32-bit has served Raspberry Pi rather well up to this point - don't knock it.

MrQ
Posts: 6
Joined: Fri Sep 06, 2019 1:32 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 6:17 pm

dom wrote:
Fri Sep 06, 2019 1:47 pm
CROSS_COMPILE=aarch64-linux-gnu- rather
BTW., there's no such command inside tools folder...

pica200
Posts: 138
Joined: Tue Aug 06, 2019 10:27 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 9:06 pm

DougieLawson wrote:
Fri Sep 06, 2019 12:49 pm
What's the -mcpu=? -mtune=? -march=? -mfp=? options for gcc on RPi4B that you're using?
Same question for RPI3B, RPi3B+ & RPi3A+.

Because https://gcc.gnu.org/onlinedocs/gcc-8.3. ... RM-Options is just a weeny bit confusing.

Do I need anything special in the way of libraries (or do I need to rebuild any) to get things running in 64-bit? [In which case I may need to switch to Sakaki-san's Gentoo64.]
Ideally you want:
32 bit: -march=armv8-a+crc+simd -mtune=cortex-a72
64 bit: -march=armv8-a+crc -mtune=cortex-a72
The difference boils down to which options gcc uses by default. See end of this page: https://gcc.gnu.org/onlinedocs/gcc-9.1. ... 64-Options

-march specifies the minimum the code should run under (can go down all the way to ARMv6 for Pi 1 support for example).
-mtune tells gcc to optimize code generation for the specified CPU (this may be limited by -march since older CPUs/archs lack features).

Together -mtune and -march replace -mcpu which is getting deprecated.

jdonald
Posts: 413
Joined: Fri Nov 03, 2017 4:36 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Fri Sep 06, 2019 10:05 pm

DougieLawson wrote:
Fri Sep 06, 2019 12:49 pm
Do I need anything special in the way of libraries (or do I need to rebuild any) to get things running in 64-bit?
You don't need additional libraries to get the kernel running 64-bit, but I would guess you're asking about running 64-bit userland programs.

There are several ways to do that, but a Debian arm64 chroot is probably easiest. Follow the debootstrap instructions sakaki posted here. It usually takes less than 10 minutes to build a 64-bit chroot on a Pi 4, but for your older devices it may be longer so I advise omitting the "&>/dev/null" parts as she mentions so that you can monitor the progress.

Once you have the 64-bit chroot, in there you can apt install gcc-aarch64-linux-gnu and compile stuff with -march=armv8-a+crc -mtune=cortex-a72 as pica200 described. Except if you're on a Pi 3B+ or 3A+ you probably want -mtune=cortex-a53 instead.

User avatar
Gavinmc42
Posts: 3736
Joined: Wed Aug 28, 2013 3:31 am

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sat Sep 07, 2019 9:04 am

Anyone noticed audio issues with the 64bit kernel?
Warzone2100 and 0AD don't have sound in 64bit mode.
Tested YT videos on Chromium, sound works.

Switched back to 32bit and the games sound tracks are fine and YT vids still work.
Seconded. 32-bit has served Raspberry Pi rather well up to this point - don't knock it.
I did not, was not, never intended to knock Raspbian.
32bit Raspbian and other 32bit OS's just work, I don't think about it or worry about it anymore.
But I have been spending lots of time in 64bit land and was preoccupied with that.
My apologies to anyone who got upset by an inappropriate emoji usage.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

mitchejj
Posts: 2
Joined: Fri Sep 06, 2019 5:20 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sat Sep 07, 2019 8:23 pm

Since we are talking about `rpi-update` and the 64-bit kernel... I suppose I should ask something I’ve yet to find an answer for with all my searching... how does one get the kernel headers for the kernel install via rpi-update? I would assume those people willing to something closer to bleeding edge might also be the type of person who might make use of the headers. So why not also install the headers?

Just wondering.

jdonald
Posts: 413
Joined: Fri Nov 03, 2017 4:36 pm

Re: Pi4 64-bit raspbian kernel for testing - Focus on Pi4

Sat Sep 07, 2019 9:45 pm

MrQ wrote:
Fri Sep 06, 2019 2:39 pm
I don't cross compile. Just building directly on my RPI4.
That can work in theory. Raspbian doesn't provide a gcc-aarch64-linux-gnu package, so you would likely compile under a 64-bit userland (e.g. Gentoo) or chroot (see above).

And I think you should set KERNEL=kernel8 (not kernel7 or kernel7l).
MrQ wrote:
Fri Sep 06, 2019 6:17 pm
dom wrote:
Fri Sep 06, 2019 1:47 pm
CROSS_COMPILE=aarch64-linux-gnu- rather
BTW., there's no such command inside tools folder...
Sounds like you decided to cross-compile. Highly recommended if you might have to debug things and iterate. Get a toolchain via sudo apt install gcc-aarch64-linux-gnu on your Debian or Ubuntu x86_64 system. The toolchains under https://github.com/raspberrypi/tools are all rather old anyhow.

Return to “Advanced users”