theoldwizard1
Posts: 17
Joined: Thu Apr 28, 2016 8:23 pm

FAST RAM

Thu Nov 22, 2018 2:15 am

None of the Pi board (or more properly none of their MCUs) have any on chip RAM other than L1 and L2 caches.

If I have a process that is "locked" to one core (using sched_setaffinity) that is simply reading GPIO pins and writing them to a circular buffer, is there anyway to "lock" the L2 cache control system so that that data never gets "flushed". a second process will be reading that data and doing "something" with it, but this process will run at normal priority and will not be locked.

Alternatively, how difficult is it to interface a fast RAM to the processor and how many cycles would it take to actually write data to this RAM.

Clearly I am looking for a "deep dive" into to how the L1 and L2 caches work.

User avatar
DavidS
Posts: 3794
Joined: Thu Dec 15, 2011 6:39 am
Location: USA
Contact: Website

Re: FAST RAM

Thu Nov 22, 2018 5:14 am

theoldwizard1 wrote:
Thu Nov 22, 2018 2:15 am
None of the Pi board (or more properly none of their MCUs) have any on chip RAM other than L1 and L2 caches.

If I have a process that is "locked" to one core (using sched_setaffinity) that is simply reading GPIO pins and writing them to a circular buffer, is there anyway to "lock" the L2 cache control system so that that data never gets "flushed". a second process will be reading that data and doing "something" with it, but this process will run at normal priority and will not be locked.

Alternatively, how difficult is it to interface a fast RAM to the processor and how many cycles would it take to actually write data to this RAM.

Clearly I am looking for a "deep dive" into to how the L1 and L2 caches work.
No need for a "deep dive".

Obviously you already got to the point of successful boot, successful enabling of the CPU Caches, and successfully turned on the MMU with a valid page table. As such you need little else, as long as your code and data is small enough it will stay in the caches, nothing more to it, so long as that really is all your system does.
RPi = Way for me to have fun and save power.
100% Off Grid.
Household TTL Electricity Usage = 1.4KW/h per day.
500W Solar System, produces 2.8KW/h per day average.

LdB
Posts: 1004
Joined: Wed Dec 07, 2016 2:29 pm

Re: FAST RAM

Thu Nov 22, 2018 6:16 am

I still don't believe it will change the fact the GPIO bus is slow and you still are limited. The max gpio speed is 71Mhz the core is doing 1400Mhz that is a 20 to 1 ratio and you can loop it on 3 cycle loop. If it was capable of going faster it already would.

theoldwizard1
Posts: 17
Joined: Thu Apr 28, 2016 8:23 pm

Re: FAST RAM

Thu Nov 22, 2018 7:41 am

DavidS wrote:
Thu Nov 22, 2018 5:14 am
Obviously you already got to the point of successful boot, successful enabling of the CPU Caches, and successfully turned on the MMU with a valid page table. As such you need little else, as long as your code and data is small enough it will stay in the caches, nothing more to it, so long as that really is all your system does.
Of course that is not the ONLY thing it is doing ! I am concerned about other processes "flushing" the L2 cache (which is shared amongst all of the cores, correct ?) and/or what ever the "write back" strategy is that would stall the dedicated processor.
LdB wrote:
Thu Nov 22, 2018 6:16 am
I still don't believe it will change the fact the GPIO bus is slow and you still are limited. The max gpio speed is 71Mhz the core is doing 1400Mhz that is a 20 to 1 ratio and you can loop it on 3 cycle loop. If it was capable of going faster it already would.
I am an old "hard real time" guy. Never really dealt with caches and how they change the determinism of a process or a GPIO pin could not be read within 1 or 2 cycles. These are "new concept" to me ! We did not clock very fast, but these issues just did not exist !

theoldwizard1
Posts: 17
Joined: Thu Apr 28, 2016 8:23 pm

Re: FAST RAM

Thu Nov 22, 2018 1:25 pm

If I can not read the GPIO pins and write them to shared RAM every 100nsec (50nsec better) then I should abandon this approach and would have to use a FPGA.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 21513
Joined: Sat Jul 30, 2011 7:41 pm

Re: FAST RAM

Thu Nov 22, 2018 2:19 pm

theoldwizard1 wrote:
Thu Nov 22, 2018 1:25 pm
If I can not read the GPIO pins and write them to shared RAM every 100nsec (50nsec better) then I should abandon this approach and would have to use a FPGA.
Just checked with a guy in the office, 400ns would appear to be achievable, but probably not much lower than that as there's a lot of bus fluff between the ARMs and the GPIO's.

You could probably sample faster on the VPU's as they connect direct to the GPIO's, but that's not particularly accessible.

This is worth a read viewtopic.php?t=26907
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

LdB
Posts: 1004
Joined: Wed Dec 07, 2016 2:29 pm

Re: FAST RAM

Thu Nov 22, 2018 4:27 pm

We spent a fair bit of time messing around looking at making a cheap high speed digital storage scope but physically there are limits

The output mode is documented on the net by Henner Zeller
https://github.com/hzeller/rpi-gpio-dma-demo

The read was not significantly different and there is definitely some bus stuff that goes on as you try to push it which is why I said I think you are up against the wall.

BTW I did the same thing I bought a small FPGA kickstarter kit to do my storage scope.

User avatar
DavidS
Posts: 3794
Joined: Thu Dec 15, 2011 6:39 am
Location: USA
Contact: Website

Re: FAST RAM

Thu Nov 22, 2018 4:59 pm

theoldwizard1 wrote:
Thu Nov 22, 2018 7:41 am
DavidS wrote:
Thu Nov 22, 2018 5:14 am
Obviously you already got to the point of successful boot, successful enabling of the CPU Caches, and successfully turned on the MMU with a valid page table. As such you need little else, as long as your code and data is small enough it will stay in the caches, nothing more to it, so long as that really is all your system does.
Of course that is not the ONLY thing it is doing ! I am concerned about other processes "flushing" the L2 cache (which is shared amongst all of the cores, correct ?) and/or what ever the "write back" strategy is that would stall the dedicated processor.
LdB wrote:
Thu Nov 22, 2018 6:16 am
I still don't believe it will change the fact the GPIO bus is slow and you still are limited. The max gpio speed is 71Mhz the core is doing 1400Mhz that is a 20 to 1 ratio and you can loop it on 3 cycle loop. If it was capable of going faster it already would.
I am an old "hard real time" guy. Never really dealt with caches and how they change the determinism of a process or a GPIO pin could not be read within 1 or 2 cycles. These are "new concept" to me ! We did not clock very fast, but these issues just did not exist !
As GPIO is your bottle neck not sure that it matters, though how much code do you have running? Is there anythign that needs to explicitly flush the cache in your system? And do you allign your code to 16 word blocks?

These are all things that will effect how well your code stays in cache. There is rarely a need to manually flush the cache, most things can be done by allocating pages that corrispond with hardware registers, framebuffer, mailboxes, or other things that may require not remaining in cache as write through.
RPi = Way for me to have fun and save power.
100% Off Grid.
Household TTL Electricity Usage = 1.4KW/h per day.
500W Solar System, produces 2.8KW/h per day average.

theoldwizard1
Posts: 17
Joined: Thu Apr 28, 2016 8:23 pm

Re: FAST RAM

Thu Nov 22, 2018 7:15 pm

jamesh wrote:
Thu Nov 22, 2018 2:19 pm
theoldwizard1 wrote:
Thu Nov 22, 2018 1:25 pm
If I can not read the GPIO pins and write them to shared RAM every 100nsec (50nsec better) then I should abandon this approach and would have to use a FPGA.
Just checked with a guy in the office, 400ns would appear to be achievable, but probably not much lower than that as there's a lot of bus fluff between the ARMs and the GPIO's.
40 years ago, we could read (the equivalent) of a GPIO and place it in RAM in 2 msec. That was 2 cycles so, yes, we were only clocking at 1MHz !

Clearly, none of the modern chips in SBCs are designed for "hard real time". Then you have a "general purpose" OS in between the programmer and the "metal". Don't get me wrong these are AMAZING processors, but clearly they are designed for interfacing with human's and not high speed devices.

theoldwizard1
Posts: 17
Joined: Thu Apr 28, 2016 8:23 pm

Re: FAST RAM

Thu Nov 22, 2018 7:27 pm

LdB wrote:
Thu Nov 22, 2018 4:27 pm
BTW I did the same thing I bought a small FPGA kickstarter kit to do my storage scope.
How far have you gotten on the project ? Would you care to share ?

The Rigol and other inexpensive bench scopes are truly amazing ! The portable 'scope market needs a serious "kick in the butt" ! PicoScopes are using ancient technology. Their software (and the automotive software is very, VERY good) is what is selling them. AESWave is selling their uScope Master kit for $400 and it is based on the miniDSO with some additional bells and whistles (AESWave uScope vs miniDSO scope kit) that you can put together yourself for around $325.

LdB
Posts: 1004
Joined: Wed Dec 07, 2016 2:29 pm

Re: FAST RAM

Fri Nov 23, 2018 3:14 am

I can do that .. it's not very exciting .. it uses a PC for display so not aimed at portable market so not sure it is terribly useful :-)

There is an old xilinx design example (xapp154.pdf) which gives you the code and the simple R/C setup you need to build the A/D into a fpga. Some of the newer Xilinx stuff has the A/D inbuilt as the can synthesize the circuit in silicon. There are also a couple of other techniques to build the A/D into the FPGA. From there it's digital filter and store the sucker in onboard ram. The most complex part by far is the communication of the data out to the PC.

If you search around the net around that keyword "xapp154" i feel sure someone will have used the design in more suitable start point if you want to do mobile design.

Return to “Bare metal, Assembly language”