kmuffsfpv
Posts: 2
Joined: Mon Mar 05, 2018 7:38 am

High bandwidth interface to FPGA?

Fri Mar 16, 2018 6:03 am

Hi, I was hoping someone can point me in the right direction.. I haven't been able to find whether or not it would be possible to use the SMI, SDIO, or another interface on the CM3 to get decent data rates between the CM3 and an FPGA (ideally >300Mb/s).. Considering that the broadcom chip is fairly closed as far as documentation, it would be very nice to know if someone has tried this before.. :P

Thanks
- Kmuffs

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Fri Mar 16, 2018 8:27 am

Gert posted some SMI-IDE code.
viewtopic.php?f=45&t=197875

SMI 16bit wide is only about 19MHz for 300Mbs.
I think SMI will do 40+MHz? But Gert is an expert ;)

A search has shown 25MHz has been done.
Get a bit more speed going 18bit wide?

Use a Pi 3B+ Ethernet?
Use one core for the DMA?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Mon Mar 19, 2018 11:13 pm

Hi Gavinmc,

I've read the topic you linked, and there was a discussion about to make the SMI documentation public. I checked the Github repo, but could not find it. Maybe I did something wrong.
Do you know it was/still is available somewhere?
I am also interesed in this peripheral. ;)

Thanks,
Imre
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 1:14 pm

The code is the documentation?
https://github.com/fenlogic/IDE_trial
These is a simple FPGA in the schematic too.

Been a while, not sure if there was more docs.
The actual SMI registers and how to set them up is the main trick.

I have not tried it, I don't use C these days.
But it looks to be word read and write and block read and write code.

Code: Select all

 // TODO: switch to higher clock freq for fast PIO modes 
smi_clock_freq = 125000000;
Not sure if that smi_clk is a real 125MHz data clock or what fast PIO mode is?

Recent GPIO bit banging on the new Pi3B+ show software toggling at 60MHz compared to 40MHz on the BCM835 chips.
But software toggling is not DMA based word data dumping.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7329
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 2:40 pm

There's also a kernel driver for it - https://github.com/raspberrypi/linux/bl ... _smi_dev.c
I haven't checked on the apis for it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 5:01 pm

Thanks for the answers! These seem to be useful. :)
But I am affraid, I have to dig quite deep into linux. :roll:
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 11:53 pm

But I am afraid, I have to dig into linux quite deep
That scares me too ;)

So I was very happy to find something that is not Linux.
https://ultibo.org/

You have not said what you are using the FPGA for, do you need Linux?
If so, then that kernel driver will give you clues.
If you make your FPGA look like a NAND flash it might be one way.

Gert's IDE/SMI interface is similar to NAND flash/compact flash...
The Motorola 6800 interface or Intel 8080 interface are both old, well known to grumpy old guys like me.
These were common to some things like LCDs but at slower rates.

I really need to dig into that 125MHz clock option for pio, that sounds way too fast.
There must be wait and divide by clock cycles or something.

From the manual, the GPCLKs can go to 125MHz but at 1.2V, but I don't think GPIO can go that fast.
Two or three wait cycles?
62.5 and 41.67MHz do seem close to the bitbang results.
125MHz at 1.2V is consistent with 62.5MHz at 3.3V swing.
As far as I can tell the GPCLK pins are just normal GPIO with ALT functions.

New silicon chip in the Pi3B+, it is a flip chip with a mirrored die, what this means it there are no internal bond wires.
Shorter wires in RF terms means higher freqs.
It would be interesting to compare GPCLK0 output on new 3B+ to old 3B on a cro to see what the signal levels and waveshape looks like.
At 40um and flipped die the 3B+ should have the fastest I/O until 28um or less versions come online in 20xx?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Mon Apr 23, 2018 10:45 pm

Thank you for linking ultibo. It seems to be a definitely useful thing. Especially if all else fails. :)
You have not said what you are using the FPGA for, do you need Linux?

I have not said that I use FPGA. It was said by the author of this thread. ;)

I am just also interested in the potential of SMI. It would be a very nice project for me, if I can make two or more RPIs working together, as one system (not in a cluster configuration). Some kind of SMP.

For this, I will need to use FPGA(s) for sure. So, I haven't told yet, but I will use FPGA(s) though. ;)

But at this moment, I am reading about Linux SMP in details. Not an easy topic for sure. At least for me. :)
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 1:53 am

But at this moment, I am reading about Linux SMP in details. Not an easy topic for sure. At least for me. :)
Not easy for me either else I would not be posting on a beginners computer forum :lol:

I have been thinking about a model on the Cray 1/2 made from Zero's.
Using the header in SMI mode as a high speed interface.

It might be faster if CM3's used, perhaps CM3+ when/if it comes out.
A flexible PCB with sockets could connect them all.

Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?

Now that Gentoo Aarch64 is out the full ARM8vA speed can be used.
So many things to do with Pi's.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 11:33 am

I have been thinking about a model on the Cray 1/2 made from Zero's.
Using the header in SMI mode as a high speed interface.

It sounds an also very interesting project. ;)
Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?

I have also thought about them. I am not exactly sure, but I found that these peripherals are accessible directly by the VideoCore only, not the ARM cores. In the other hand, I don't know is it possilbe or not to use them for data transfer somehow.
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7329
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 12:03 pm

Imre wrote:
Tue Apr 24, 2018 11:33 am
Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?
I have also thought about them. I am not exactly sure, but I found that these peripherals are accessible directly by the VideoCore only, not the ARM cores. In the other hand, I don't know is it possilbe or not to use them for data transfer somehow.
DSI https://github.com/raspberrypi/linux/bl ... /vc4_dsi.c
CSI https://github.com/raspberrypi/linux/pull/2513

Both are very much designed around transferring image frames, ie big buffers. Small buffers will mean lots of interrupts and may not be practical.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 12:37 pm

Both are very much designed around transferring image frames, ie big buffers. Small buffers will mean lots of interrupts and may not be practical.
That could be perfect.
Not sure if a DSI display output can talk to a CSI camera input, both are LVDS.
But if they could, then a ring type high speed serial channel could be done.

Actually for CV/NN, a 32bit RGBA image format would be interesting to try.
I think it was Pete Warden who said 8bit data was fine for AI/ML/CV.
It is something I want to try and that is using RGBA image blending methods as NN filters.
Basically a stack of 2D array filters to get a mono (yes/no) output.

As the VC4 will be handling the DSI/CSI data moving, the Aarch64/NEON cores do the image processing.
Near total ignorance suggests it may be worth investigating further.

Some FPGA can do LVDS too :D
How many channels do they have?
Is there FPGA CSI/DSI libraries?
The FPGA could act as CSI/DSI converter?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 1:56 am

Today's blog has an interesting device. HDMI to CSI.
https://www.raspberrypi.org/blog/tinker ... streaming/

I suppose if you organise your data in image format this is a fast way to move data round too.
That's 3 LVDS high speed serial data methods, HDMI, DSI, CSI.
Do they have lower layers for raw data transfer?

FPGA hat that uses those DSI/CSI connectors.

viewtopic.php?p=786997
https://www.cnx-software.com/2017/05/10 ... rm-factor/
https://www.parallella.org/2015/06/01/t ... ra-module/

Pack bandaids, this looks to be bleeding edge stuff :lol:

Raspiraw just grabs CSI input without going through bayer etc
viewtopic.php?f=43&t=109137

DSI is probably going to be easier as I think there is a Linux framebuffer driver.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 2:36 am

// TODO: switch to higher clock freq for fast PIO modes
smi_clock_freq = 125000000;

Not sure if that smi_clk is a real 125MHz data clock or what fast PIO mode is?
Quoting myself, but pio??? is that a reference to parallel I/O
https://en.wikipedia.org/wiki/Parallel_I/O

Around the time this was designed-8 years ago?
Compact flash was still around so perhaps this is reference to CF interfaces?
https://en.wikipedia.org/wiki/CompactFlash
The iPod mini, Nokia N91, iriver H10 (5 or 6 GB model), PalmOne LifeDrive, and Rio Carbon used a Microdrive to store data.
Pi + IBM microdrive? Run FreeDOS on it, very retro :lol:
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7329
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 7:41 am

Gavinmc42 wrote:
Sat Apr 28, 2018 1:56 am
Today's blog has an interesting device. HDMI to CSI.
https://www.raspberrypi.org/blog/tinker ... streaming/
It's the same Auvidea board that has been discussed so many times before.
Auvidea cloned the prototype developed at Pi Towers and shown at Embedded World in Nuremberg in 2014 http://blog.pi3g.com/2014/03/embedded-w ... bc-market/. The drivers using the firmware have always been a bit flaky, and are not officially supported (I wish our comms team would talk to us sometimes before publicising things!).
Lots of discussion at viewtopic.php?f=38&t=120702.

No, it doesn't add anything over doing an FPGA direct to CSI2, except making life more complicated (you have to worry about EDIDs and the like).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 8:06 am

I thought that Toshiba part number looked familiar.

Might have to get some FPGAs to play with.
Wonder if a FPGA could emulate a CSI camera?

Stereo sensors to FPGA to CSI?

Hangon, does not the Google AIY Vision sensor use CSI?
https://blog.google/topics/machine-lear ... vices-see/
https://aiyprojects.withgoogle.com/visi ... e-hardware
Any source code for this?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1440
Joined: Sat Sep 10, 2011 11:43 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 12:49 pm

Yes, you can use an FPGA to do CSI, with just a couple of resistors, but it won't be able to run full Gbit speed. Instead you can use a CSI phy and connect to the FPGA

Not easy though
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sun Apr 29, 2018 1:01 am

Yes, you can use an FPGA to do CSI, with just a couple of resistors, but it won't be able to run full Gbit speed. Instead you can use a CSI phy and connect to the FPGA
Not easy though
Serious investment in tools and time to do it.
But is there any faster way to get data into a Pi?
As a thought experiment, it should work, just lots of hard work.
You should have said it is impossible, someone would be out to prove you wrong :lol:

It would be nice to a have a small CPLD board that had LVDS IO and there was free software for designing/programming it.
Most of the good CPLD/FPGA come in packages with lots of pins.

SATA is LVDS? Some sort of converter? SATA has TX/RX so need DSI/CSI.
Starting now, might get it going about the time Pi5 comes out :lol:
So many ideas, so much lack of skill and knowledge :oops:

I would try SMI first, might not be as fast but it would probably work and could be tested in days ;)
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

andrea.varesio
Posts: 8
Joined: Wed May 29, 2013 11:06 am

Re: High bandwidth interface to FPGA?

Sun May 12, 2019 8:46 pm

can anyone provide SMI (secondary mamory interface) hardware specification?

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Mon May 13, 2019 12:45 am

can anyone provide SMI (secondary mamory interface) hardware specification?
That's really old, try a search for ATA, PCMCIA.
Not really a spec, more an old Standard.

Somewhat related for the LCD DPI visa GPIO is the 8080/6800 interface modes.
You will find some source code in the Kernel SMI Nand driver.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

BEB
Posts: 3
Joined: Thu Aug 22, 2019 7:10 pm

Re: High bandwidth interface to FPGA?

Thu Oct 10, 2019 1:40 pm

andrea.varesio wrote:
Sun May 12, 2019 8:46 pm
can anyone provide SMI (secondary mamory interface) hardware specification?
The problem is not to have access to the hardware specification. The SMI bus is nothing else than 68xx/808x style interface and there are plenty of references available everywhere on the web giving the description of the signals.

The biggest issue is that Raspberry team did not publish any datasheet describing exact configuration expected on the SMI. The only information available is the source code of the SMI driver and the excellent work from Gert Van Loo to make the IDE interface. There is simply nothing explained in the BCM2837 GPIO datasheet for the SMI configuration.

I have developed an FPGA based interface for the RPi (I called it RasPiBus) and I was planning originally to use SMI for high bandwidth parallel bus. For now, using the SMI is such a pain that I have created my own protocol based on GPIO. Even if it has lower performances than SMI, I am able to get more than 3MB/s on the parallel bus to the FPGA.

gizmomouse
Posts: 17
Joined: Mon Jun 10, 2019 2:45 am

Re: High bandwidth interface to FPGA?

Mon Oct 21, 2019 12:46 am

@BEB Why was it challenging to use the SMI interface?

I'm curious because I designed a board with the SMI interface with an FPGA in mind. I'm a ways away from exercising it but one thing that concerned me was that there doesn't seem to be any dedicated SMI clock, perhaps I missed it. This would imply that the FPGA interface would need to be asynchronous, and we would need to over-sample the signals on the FPGA side to read the SMI data... Or some other scheme that I haven't thought of??

I did some research on this when I started working on this board last year and found this within the kernel documentation:

<kernel>/Documentation/devicetree/bindings/misc/brcm/bcm2835-smi.txt

The part that interested me was the 'brcm,smi-clock-source' and the 'brcm,smi-clock-divisor'

It allows the user to select the clock source and clock divider. I was thinking that if I create a GPCLK with the same settings I would then have a 'clocked' SMI interface (there would probably a phase delay but that can be solved with delay taps on an FPGA). Perhaps I'm just naive and this is what was intended or there is another way of doing this. If so please let me know.

As a side note I am hopeful that a throughput of 125MB/s is possible because the settings in the the above documentation are

Code: Select all

...
  brcm,smi-clock-source = <6>;
  brcm,smi-clock-divisor = <4>;
...
based on this page: Raspberry Pi GPCLK Page these settings are for a 125MHz clock.

Dave

User avatar
Gavinmc42
Posts: 3761
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Mon Oct 21, 2019 2:15 am

I think BCM2835 can access GPIO at about 40MHz and the 3's at 60MHz.
Pi4's are faster again?
If you use a 16bit wide bus then 80MB/s would be more realistic?

It is hard to switch cmos 3V3 logic faster than 100-200MHz.
That's why LVDS is "low voltage" and DDRAMs have lower voltage I/O etc.

I have stuck a cro probe on that 125MHz GPCLK0, it was not a full voltage rail to rail swing.
It was only about 1V peak to peak.

You might get 125MB/s or close on a Pi4?
If you read up on 68/80 type bus interface there are CS, RD, WR lines.
RD/WR are clock type data latch signals.
Not sure about sychnnoous mode that has a clock?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Return to “Compute Module”