Pi4 64-bit raspbian kernel for testing - Focus on Pi4

[andrum99]

Hi,

which max. memory size is correct? 3.86G (raspbian lite 32 bit) or 3.77G (raspbian lite 64 bit)?

memory_32_vs_64.jpg

Surely they both are. Anything 64bits tends to use a little more memory

But max memor should be same for 32 and 64.

Both figures are correct - there will be a reason why the 64-bit kernel can see less usable memory. I will leave someone who knows to comment on why. It is obviously something to do with some memory being needed by the GPU. I also seem to remember there's also 16MB of memory that needed some special sauce applied for Linux to be able to use. Perhaps this sauce is yet to be applied, or cannot be applied, to 64-bit.

[dom]

Both figures are correct - there will be a reason why the 64-bit kernel can see less usable memory. I will leave someone who knows to comment on why. It is obviously something to do with some memory being needed by the GPU. I also seem to remember there's also 16MB of memory that needed some special sauce applied for Linux to be able to use. Perhaps this sauce is yet to be applied, or cannot be applied, to 64-bit.

The GPU uses exactly the same amount of memory in 32 and 64 bit modes.
The kernel itself is larger with a 64-bit build and so there is less free memory remaining.

[andrum99]

Both figures are correct - there will be a reason why the 64-bit kernel can see less usable memory. I will leave someone who knows to comment on why. It is obviously something to do with some memory being needed by the GPU. I also seem to remember there's also 16MB of memory that needed some special sauce applied for Linux to be able to use. Perhaps this sauce is yet to be applied, or cannot be applied, to 64-bit.

The GPU uses exactly the same amount of memory in 32 and 64 bit modes.
The kernel itself is larger with a 64-bit build and so there is less free memory remaining.

I stand corrected.

[andrum99]

I'm getting the unknown event type 37 messages again, this time with the 64-bit kernel under heavy USB load from 4 UAS hard drives:

Code: Select all

Nov 27 20:31:07 pi4b2 kernel: [86935.640640] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:33:42 pi4b2 kernel: [87090.769079] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:42:48 pi4b2 kernel: [87636.268550] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:43:12 pi4b2 kernel: [87660.585951] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:44:10 pi4b2 kernel: [87718.588640] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:44:55 pi4b2 kernel: [87763.349847] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:46:06 pi4b2 kernel: [87834.596155] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:48:17 pi4b2 kernel: [87965.731894] xhci_hcd 0000:01:00.0: ERROR unknown event type 37
Nov 27 20:50:52 pi4b2 kernel: [88120.284449] xhci_hcd 0000:01:00.0: ERROR unknown event type 37

This is apparently due to the event ring filling before the USB interrupt handler can empty it.

I was told the last time (in the description of https://github.com/raspberrypi/linux/pull/3147) that these are "benign, but throughput
will suffer", but it's just to let you know they have put in another appearance. I was not able to see any difference in speed when these errors were happening - it was less than 1 per second previously, and you can see above that it is less than that for this new occurrence.

[RandyOo]

I've just created a post that describes some problems I've been experiencing with USB hubs.

I can't really say with any certainty if it's the 64 bit kernel, since I haven't really done any testing with the 32 bit kernel yet. If it's warranted, I'd be happy to oblige, as time permits.

[RossDv8]

RandyOo, Somewhere in another post is a discussion about the Pi 4 and some issues a few of us were having with USB 3 hubs and rebooting the Pi 4.
I was using a non-powered hub with a powered USB 3 external drive, and whenever I had it plugged in and tried to reboot, the Pi 4 shut downb and refuse to boot back up. Pulling the powered external drive from the non-powered hub and using a non-powered drive and all was normal.
Using some powered USB 3 hubs caused problems for some people also.

It might not be causing the type of problems you are experiencing, but it also just might. There is a possibility that some power supplied to the powered hub, or powered drives via a non-powered hub from your NAS are feeding power back into the USB ports.

It shouldn't happen, because one would think there would be some sort of diode or transistor protection to prevent back feeding. But I have no idea how the small computers are designed. I'm purely guessing.

Anyway, I was having the reboot problem, and a couple of other dramas, and others reported problems with powered USB 3 hubs. It might have some bearing on it.

And relevant info is - I had the problem both before and after going to the 64 bit Colonel.

[RandyOo]

RandyOo, Somewhere in another post is a discussion about the Pi 4 and some issues a few of us were having with USB 3 hubs and rebooting the Pi 4.
(snip)
And relevant info is - I had the problem both before and after going to the 64 bit Colonel.

@RossDv8, thanks for that. I had actually already stumbled across that thread. It seems that powered hubs that backfeed power will even cause trouble booting on PCs.

This is most certainly a different issue, and very unlikely to be power-related. The biggest question is whether it's 64-bit specific (and related to this thread), or even Raspberry Pi-related, since it's possible that these hubs are unreliable on other hardware...

At this point, I'm (personally) satisfied to have found a hub that doesn't induce speed resets or disconnects on the USB bus. I'm only posting here to offer up information for the sake of the community/development.

[RossDv8]

I updated to this version of the kernel:

4.19.86-v8+ #1283 SMP PREEMPT Fri Nov 29 18:43:33 GMT 2019 aarch64

and I don;t know if it was coincidental or not, but my old problem with changing stuff via obconf is making the Titlebar inactive again.

Specifically changing the mouse double click on Titlebar action to Shade, and changing the Theme.
Previously I had a very quick reset by clicking any of the Defaults on Desktop Settings., but now that doesn;t happen.

HOWEVER !!!!

I am back to overclock CPU at 1850 Voltage at 3 GPU at 600

While I was running those settings again for some weeks, it was ony after updating the kernel yesterday that the Title bar began playing up. So there's a possibility that the overclock settings and the new kernel don;t play nicely.

But I can't blame the kernel for the problem. I'll have to restore the standard clock settings, run for a week or so, and see if it happens again.
Just at the moment though, I have it all working nicely.

[RandyOo]

While digging through /var/log/messages to collect error messages related to the USB hub/SCSI errors I've been experiencing, I've coming across many (hundreds of) instances of the following:

Code: Select all

[Mon Dec  2 13:00:42 2019] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Dec  2 13:00:42 2019] [drm]                           dumb:  54852kb BOs (5)
[Mon Dec  2 13:00:44 2019] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Dec  2 13:00:44 2019] [drm]                           dumb:  54852kb BOs (5)

I haven't experienced any obvious issues with video rendering.

[andrum99]

While digging through /var/log/messages to collect error messages related to the USB hub/SCSI errors I've been experiencing, I've coming across many (hundreds of) instances of the following:

Code: Select all

[Mon Dec  2 13:00:42 2019] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Dec  2 13:00:42 2019] [drm]                           dumb:  54852kb BOs (5)
[Mon Dec  2 13:00:44 2019] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Dec  2 13:00:44 2019] [drm]                           dumb:  54852kb BOs (5)

I haven't experienced any obvious issues with video rendering.

What were you doing when the errors occurred? Are there any other symptoms?

[RandyOo]

While digging through /var/log/messages to collect error messages related to the USB hub/SCSI errors I've been experiencing, I've coming across many (hundreds of) instances of the following:

Code: Select all

[Mon Dec  2 13:00:42 2019] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Dec  2 13:00:42 2019] [drm]                           dumb:  54852kb BOs (5)
[Mon Dec  2 13:00:44 2019] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[Mon Dec  2 13:00:44 2019] [drm]                           dumb:  54852kb BOs (5)

I haven't experienced any obvious issues with video rendering.

What were you doing when the errors occurred? Are there any other symptoms?

The GUI has mainly been used for running a web browser and Kodi. Since I didn't notice the errors in dmesg until later, I'm not sure exactly what was happening when they showed up previously. However, the error just showed up in connection with toggling Chromium between full screen mode and windowed mode. There a black bar across the top of the display, as well.

I can't seem to replicate the issue at will, although it has happened on 9 different days, each time repeating about 20 lines in the log in each instance.

[andrum99]

Just to add another issue - the (now official) rascas Raspberry Pi-specific build of Kodi is non-functional on the 64-bit kernel - it displays its bootsplash then crashes. Log available if anyone is interested.

[RossDv8]

Just to add another issue - the (now official) rascas Raspberry Pi-specific build of Kodi is non-functional on the 64-bit kernel - it displays its bootsplash then crashes. Log available if anyone is interested.

Does the rascas build do anything specific, for example is it built to play 4k video?
Just curious, because a quick search didn't give me much about it really.

In the mean tiime, Kodi 18.4.6 (in the suppository) still works very nicely on the Pi 4 with the 64 bit kernel, until the @rastas build can be sorted..

[echmain]

...
In the mean tiime, Kodi 18.4.6 (in the suppository)

So THAT’S where they keep it.

[Rascas]

Just to add another issue - the (now official) rascas Raspberry Pi-specific build of Kodi is non-functional on the 64-bit kernel - it displays its bootsplash then crashes. Log available if anyone is interested.

Does the rascas build do anything specific, for example is it built to play 4k video?
Just curious, because a quick search didn't give me much about it really.

In the mean tiime, Kodi 18.4.6 (in the suppository) still works very nicely on the Pi 4 with the 64 bit kernel, until the @rastas build can be sorted..

About Kodi, what people are calling here as "Rascas build", it is currently the packages available on the RPi Foundation repo. Basically what you get when you install Kodi on Raspbian through apt or the graphical package manager. The latest available version is Kodi 18.4-6, it will be updated soon to 18.5. You have more information about that here: https://www.raspberrypi.org/forums/view ... 6&t=251645

And as of today, Kodi only works with a 64 bit kernel on Raspbian if you update the kernel/firmware with rpi-update tool, it won't work with the current kernel/firmware available on the repos.

And yes, Kodi on Raspbian supports 4K HEVC/h265 videos which is the only codec that the RPi 4 has a 4K hardware video decoder.

[RossDv8]

And as of today, Kodi only works with a 64 bit kernel on Raspbian if you update the kernel/firmware with rpi-update tool, it won't work with the current kernel/firmware available on the repos.

Answers my question perfectly, thanks.
And yes, it is working perfectly for me so far on the

Linux raspberrypi 4.19.86-v8+ #1283 SMP PREEMPT Fri Nov 29 18:43:33 GMT 2019 aarch64 GNU/Linux

kernel (the most recent update I have).

Now I know what the rastas thing is about. Thanks for the good work..

[andrum99]

Just to add another issue - the (now official) rascas Raspberry Pi-specific build of Kodi is non-functional on the 64-bit kernel - it displays its bootsplash then crashes. Log available if anyone is interested.

Does the rascas build do anything specific, for example is it built to play 4k video?
Just curious, because a quick search didn't give me much about it really.

In the mean tiime, Kodi 18.4.6 (in the suppository) still works very nicely on the Pi 4 with the 64 bit kernel, until the @rastas build can be sorted..

That's the one I'm using. I was unable to get it to work on a clean build of Raspbian. It was previously available from a separate repo, but has now been integrated into Raspberry Pi's repo (archive.raspberrypi.org).

[RossDv8]

I don;t know if this is particularly relevant or interesting, but I just opened the desktop wallpaper image of the Millenium Falcon in GIMP while omxplayerGUI was playing an .m4a file and Chromium was doing stuff in 4 tabs.

I ran the Filters > Edge detect > Neon on the image and watched the temperature display and the CPU frequency monitor in the top right corner of the panel.

CPU Frequency went from 950MHz with just Chromium running its 4 tabs, and omxplayerGUI and a File manager running, and a fairly constant 48 decrees Celsius in the ambient room temp of 32 deg Cels (its a cool day here today), to 1900 MHz once Gimp was doing its thing, and the temp started swapping 48 - 50 - 52 at the beginning, to 52 - 55 - 58 at the end.

The maximum temp was 58 and it regularly cycled back to 52 while loaded.
The fan is still only running on the 3.3V pins of the GPIO, so it is still running about half speed.
When I ran this test a few months ago I managed to get the temperature up quite a bit higher initially without the system being overclocked. I'm quite impressed that now, even with the CPU and GPU overclocked, the Pi 4 is running relatively cool.

[RandyOo]

I've just created a post that describes some problems I've been experiencing with USB hubs.

I can't really say with any certainty if it's the 64 bit kernel, since I haven't really done any testing with the 32 bit kernel yet. If it's warranted, I'd be happy to oblige, as time permits.

I've now seen this issue with USB device resets (and subsequent I/O errors, data loss/corruption) across all 4 hard drives that I've attempted to connect to a USB 3.0 hub. I've tried 4 different hubs, 2 of them with and without external power, and have seen the behavior with all of them, though some seem worse than others. It also occurred after disabling UAS with the storage quirks option in cmdline.txt.

I'm reasonably confident that it isn't a power issue, based on the fact that:

• the drives are self-powered
• externally powered hubs don't help
• device resets only happen on one drive at a time (ruling out a problem with grid power)

It seems to point towards an issue with the USB driver, or perhaps a hardware issue?

I'm currently testing a theory that the smartmontools package is triggering the error, but I can't declare it with any confidence until I've been able to test for several days, at least. I will also attempt to test the 32-bit kernel to see if this issue is unique to 64-bit, but it could possibly be a few weeks before I can get to it.

[RandyOo]

I'm currently testing a theory that the smartmontools package is triggering the error, but I can't declare it with any confidence until I've been able to test for several days, at least. I will also attempt to test the 32-bit kernel to see if this issue is unique to 64-bit, but it could possibly be a few weeks before I can get to it.

I've just duplicated the issue even without smartmoontools installed or smartd running, meaning that my theory was wrong, and so I've moved on to testing with the 32-bit kernel.

@jamesh, if you have some suggestions as to how I can capture some more data that would be useful for debugging, I'll be happy to contribute any information I can. I've tried checking my logs from before I switched to the 32 bit kernel, but alas, it was over the default 4 week retention level. I've put "loglevel=6" in my cmdline.txt, but it doesn't seem to generate any more detailed info in dmesg or /var/log/messages.

I considered sending a message on the LKML, but don't even know if this is a USB issue, or perhaps a SCSI issue manifesting as an USB problem. Pi-specific, or ARM8-specific? Hopefully, after a few days of testing back on the 32-bit kernel, it'll be narrowed down a bit.

[RandyOo]

I will also attempt to test the 32-bit kernel to see if this issue is unique to 64-bit, but it could possibly be a few weeks before I can get to it.

After four days of testing the USB port with the hub attached (by looping dd reads of disks), I've replicated the issue on the 32 bit kernel, so I guess it's off-topic for this thread, since it's not unique to the 64-bit kernel. Sorry for the distraction.

[ulph0]

hi,
that's great. but if i try to execute 64bit oracle java like this /opt/jdk/jdk1.8.0_221/bin/java
i get an error thet the file or directory was not found. i think we need a dynamical loader for 64bit binaries. is there a way to get this work?
regards.

[akschu]

Reporting in... I bought a 4GB Pi4 with the sole intent of having it perform the role of a small ZFS-based NAS, now that it has USB 3.0, gigabit ethernet, and lots more RAM. I had already managed to get ZFS up and running on Pi3, so I knew it could be done.

I managed to get ZFS compiled and running on the 32 bit kernel, but kept encountering killed tasks due to OOM (out-of-memory) errors while running a simple (but large) rsync between two ZFS pools, and even though swap was set up and present, it was unused. Seems to be some kind of manifestation of ZFS not being happy on a 32-bit kernel.

After conquering the learning curve of getting a chroot environment set up to compile arm64, and kernel source downloaded from git and built, I finally managed to get the ZFS module compiled for arm64, and subsequently installed.

Now, with the 64-bit kernel running, the memory issue is gone, and throughput on the rsync file copy between two USB 3.0 drives appears to have doubled!

Although I do enjoy learning, I'll look forward to the day when ZFS is just an apt-get install away...

How were you able to compile the ZFS kernel module? I'm trying to do the same, but there isn't a kernel-headers or kernel-source package I can find, and downloading the official raspberrypi kernel source and building it results in "gcc: error: unrecognized command line option ‘-mgeneral-regs-only’"

I can't figure out how to build modules for this kernel. Any help would be greatly appreciated.

schu

[RandyOo]

How were you able to compile the ZFS kernel module? I'm trying to do the same, but there isn't a kernel-headers or kernel-source package I can find, and downloading the official raspberrypi kernel source and building it results in "gcc: error: unrecognized command line option ‘-mgeneral-regs-only’"

I can't figure out how to build modules for this kernel. Any help would be greatly appreciated.

Have a look at this post, and the script he posted on github in that first post of the thread. You may find that the kernel version needs to be updated within the script, but it ought to be a good start for you, at least. Afraid I can't help, since I haven't documented a step-by-step procedure. It took quite a bit of trial-and-error for me. Good luck!

[sshadow]

I note that the 64-bit kernel keeps my Pi4 significantly cooler than the 32-bit one.
64-bit temp: ~125 F, or ~51.5 C
32-bit temp: ~150 F, or ~65.5 C

This is without cooling or a heat sink. When I turn my fan on, both numbers plunge to more reasonable values.