Page 1 of 2

Programming languages support

Posted: Wed Oct 09, 2019 2:16 pm
by allampanato
Hi to all. What is the state of various programming languages on the RPi?

I am currently planning a project, where I will need to access the GPIO, the camera module and use Websockets. So I need a language which can access all the features of the RPi. I will use a RPi4 and a RPi Zero W.

I know Python fully supports the RPi family. I have used it for years. But Python is a bit slow for my needs now.

I have a list of potential candidates: Java/Kotlin/JVM in general, C#, C++, Go, Javascript/Node.js. Node.js should have a decent support ( from what I have read ). C#/.Net core too.

Also, what about Ultibo? It should support the RPi hardware fully. But websockets?

Re: Programming languages support

Posted: Wed Oct 09, 2019 2:24 pm
by jamesh
I think any computer language you can think of will be supported on the Pi.

However, the range is narrowed if you want particular libraries.

For your use case, requiring speed , I'd go with C or C++. All your requirements would be covered by easily available libraries.

No idea about Ultibo, isn't that Pascal based?

Re: Programming languages support

Posted: Wed Oct 09, 2019 3:40 pm
by allampanato
Yes, I know that practically every programming language can run on the RPi/Linux. But I need to access the GPIO pins and the camera module. And not every language has bindings to the appropriate libraries.

I know that Python has them all. Nodejs should have them ( not 100% sure ). All others, don't know.

And yes, Ultibo is Freepascal-based.

Re: Programming languages support

Posted: Wed Oct 09, 2019 3:46 pm
by rpiMike
I would try Python first. I've not had any speed issues - its coped with everything I've used it for.

Re: Programming languages support

Posted: Wed Oct 09, 2019 4:36 pm
by jamesh
allampanato wrote:
Wed Oct 09, 2019 3:40 pm
Yes, I know that practically every programming language can run on the RPi/Linux. But I need to access the GPIO pins and the camera module. And not every language has bindings to the appropriate libraries.

I know that Python has them all. Nodejs should have them ( not 100% sure ). All others, don't know.

And yes, Ultibo is Freepascal-based.
I'll repeat that C/C++ has libraries for all you need, and the ONLY other camera library I know about is in Python.

Re: Programming languages support

Posted: Wed Oct 09, 2019 4:40 pm
by ejolson
rpiMike wrote:
Wed Oct 09, 2019 3:46 pm
I would try Python first. I've not had any speed issues - its coped with everything I've used it for.
It sounds like the person making the original post has been using Python and already run into limitations with regards to execution speed.

One thought would be to profile your Python code and write the part that is slow in C and and call it from Python. While a multiple-language approach tends to be more complicated than writing everything in one language, there are many people who know Python and C well enough to work on such projects.

If I were writing code for a complicated multi-threaded server, I would try Go.

Although Go was designed to make it easy to code a multi-threaded server, it is not particularly popular on this forum. Go also runs best in 64-bit environments with AES hardware support and may need some custom interfacing code for the GPIO and camera on the Pi. As none of these problems are likely insurmountable, I think it might be the better choice.

Re: Programming languages support

Posted: Wed Oct 09, 2019 6:04 pm
by timrowledge
Smalltalk is considerably faster than python and offers easy access to gpio, web services, client-server, OS access, excellent developer tools, actual, real, portability, a sensible language and generally a lot of good things. There has been a Smalltalk available for ARM since before it was originally announced.
www.squeak.org

Re: Programming languages support

Posted: Thu Oct 10, 2019 12:43 am
by ejolson
timrowledge wrote:
Wed Oct 09, 2019 6:04 pm
Smalltalk is considerably faster than python and offers easy access to gpio, web services, client-server, OS access, excellent developer tools, actual, real, portability, a sensible language and generally a lot of good things. There has been a Smalltalk available for ARM since before it was originally announced.
www.squeak.org
The link is broken. Has squeak been discontinued?

Re: Programming languages support

Posted: Thu Oct 10, 2019 1:00 am
by Gavinmc42
https://squeak.org/
Wonder if there is a an Aarch64 version?

Re: Programming languages support

Posted: Thu Oct 10, 2019 11:44 am
by Heater
allampanato,
What is the state of various programming languages on the RPi?
Very good. Pretty much every language you will ever need is supported. Frankly if a language does not work on Linux/Unix it's better to be looking at something else.

For your project I suggest using node.js because:

1) Dealing with web sockets in node.js is trivially easy.

The requirement for websockets suggests using JSON as the data transfer format. Which again is trivially easy in node.js. Do not even think about C/C++, dealing with JSON in C/C++ is the road to madness.

2) Dealing with multiple asynchronous events is trivially easy in node.js

The requirement for juggling inputs from GPIO, the camera and the network at the same time suggests using threads in most programming languages. The event driven programming model of Javascript in node.js makes responding to all these events trivially easy. Using treaads is not necessarily difficult but it's a complexity one would rather not deal with.

3) Performance will be better than Python.

If you really need more performance than you can get from node.js then it's probably better to skip the sloth and bloat of Java or any JVM based language. Normally I would suggest C++ at that point but today Rust is a far better alternative https://www.rust-lang.org/

Re: Programming languages support

Posted: Thu Oct 10, 2019 11:47 am
by B.Goode
ejolson wrote:
Thu Oct 10, 2019 12:43 am
timrowledge wrote:
Wed Oct 09, 2019 6:04 pm
Smalltalk is considerably faster than python and offers easy access to gpio, web services, client-server, OS access, excellent developer tools, actual, real, portability, a sensible language and generally a lot of good things. There has been a Smalltalk available for ARM since before it was originally announced.
www.squeak.org
The link is broken. Has squeak been discontinued?

Just a trivial slip in composing the forum post. The site is there.

Re: Programming languages support

Posted: Thu Oct 10, 2019 11:59 am
by jahboater
Heater wrote:
Thu Oct 10, 2019 11:44 am
Using threads is not necessarily difficult but it's a complexity one would rather not deal with.
On Linux (Raspbian) or UNIX, select() makes that sort of thing fairly easy - without using threads.

Code: Select all

man 2 select
It is actually a system call.

Re: Programming languages support

Posted: Thu Oct 10, 2019 12:13 pm
by Heater
select() is great and all but you end up having to write code like this: https://www.tutorialspoint.com/unix_sys ... select.htm

Most of which is nothing to do with the problem you want to solve. Rather you are having to think about the bigger problem of how to do that kind of thing in C.

Re: Programming languages support

Posted: Thu Oct 10, 2019 12:28 pm
by jahboater

Code: Select all

    /* Watch stdin (fd 0) to see when it has input. */
    FD_ZERO(&rfds);
    FD_SET(0, &rfds);
    /* Wait up to five seconds. */
    tv.tv_sec = 5;
    tv.tv_usec = 0;
    retval = select(1, &rfds, NULL, NULL, &tv);
Seems simple enough to me.
The same code works for thousands of devices or sockets.
Much easier and safer than threads IMHO.
Its also very fast and light weight.

Re: Programming languages support

Posted: Thu Oct 10, 2019 12:37 pm
by jamesh
Heater wrote:
Thu Oct 10, 2019 11:44 am
allampanato,
What is the state of various programming languages on the RPi?
Very good. Pretty much every language you will ever need is supported. Frankly if a language does not work on Linux/Unix it's better to be looking at something else.

For your project I suggest using node.js because:

1) Dealing with web sockets in node.js is trivially easy.

The requirement for websockets suggests using JSON as the data transfer format. Which again is trivially easy in node.js. Do not even think about C/C++, dealing with JSON in C/C++ is the road to madness.

2) Dealing with multiple asynchronous events is trivially easy in node.js

The requirement for juggling inputs from GPIO, the camera and the network at the same time suggests using threads in most programming languages. The event driven programming model of Javascript in node.js makes responding to all these events trivially easy. Using treaads is not necessarily difficult but it's a complexity one would rather not deal with.

3) Performance will be better than Python.

If you really need more performance than you can get from node.js then it's probably better to skip the sloth and bloat of Java or any JVM based language. Normally I would suggest C++ at that point but today Rust is a far better alternative https://www.rust-lang.org/
Does it have the required camera library?

(Note, you could use the V4L2 driver if there is a driver for that).

Re: Programming languages support

Posted: Thu Oct 10, 2019 3:10 pm
by Heater
jahboater,

Using select() is not massively complex, but certainly more so than using fs from node.js. Especially when dealing with many files and other I/O at the same time. Also select will block execution thus upsetting responsiveness of other parts of your program. Oh and you have to use C which of course is more complicated to use in the first place.

There is no problem with thread safety in node.js. It has no threads it is event driven.

jamesh,

On can just run the camera command line tools from node.js with system(). Or use one of the many wrapper modules: https://www.npmjs.com/package/pi-camera
https://www.npmjs.com/package/v4l2camera

Re: Programming languages support

Posted: Thu Oct 10, 2019 4:59 pm
by allampanato
ejolson wrote:
Wed Oct 09, 2019 4:40 pm

If I were writing code for a complicated multi-threaded server, I would try Go.
Yes, this is one of the problems. The RPi Zero W is the core of a smart doorbell and one of its features is streaming the camera to a bunch of clients on demand. I will have a hub for the heavy lifting, so for now I am trying to understand if it is better to just stream the video to the hub ( and from there to the clients ) or do a direct connection to the clients through ( maybe ) WebRTC.

Re: Programming languages support

Posted: Thu Oct 10, 2019 5:01 pm
by jahboater
Heater wrote:
Thu Oct 10, 2019 3:10 pm
Using select() is not massively complex, but certainly more so than using fs from node.js.
Yes, very likely. I have not used node.js, so I cannot comment (JS did not exist when I was doing this sort of thing for a living ...).
Heater wrote:
Thu Oct 10, 2019 3:10 pm
Especially when dealing with many files and other I/O at the same time.
That's where select excels. Also I would expect it to be much more scaleable.
Heater wrote:
Thu Oct 10, 2019 3:10 pm
Also select will block execution thus upsetting responsiveness of other parts of your program.
No. Just set the timeout to zero and select() will return immediately (handy to implement polling).

The JS event driven model without threads does sound nice, but that's how select() is commonly used anyway.

Select() is a Linux/UNIX system call, I wonder if its possible to strace node and see if its calling select() under the covers :)

Re: Programming languages support

Posted: Thu Oct 10, 2019 5:04 pm
by allampanato
Heater wrote:
Thu Oct 10, 2019 11:44 am

1) Dealing with web sockets in node.js is trivially easy.
Plus 1 point for Node.js then. For now I don't know if Websockets or WebRTC is the better solution. From what I have read, the difference between the two is that WebRTC supports P2P connections.
Heater wrote:
Thu Oct 10, 2019 11:44 am
The requirement for juggling inputs from GPIO, the camera and the network at the same time suggests using threads in most programming languages. The event driven programming model of Javascript in node.js makes responding to all these events trivially easy. Using treaads is not necessarily difficult but it's a complexity one would rather not deal with.
I thought about Worker threads in Node.js.

Re: Programming languages support

Posted: Thu Oct 10, 2019 6:39 pm
by ejolson
allampanato wrote:
Thu Oct 10, 2019 5:04 pm
Heater wrote:
Thu Oct 10, 2019 11:44 am

1) Dealing with web sockets in node.js is trivially easy.
Plus 1 point for Node.js then. For now I don't know if Websockets or WebRTC is the better solution. From what I have read, the difference between the two is that WebRTC supports P2P connections.
Heater wrote:
Thu Oct 10, 2019 11:44 am
The requirement for juggling inputs from GPIO, the camera and the network at the same time suggests using threads in most programming languages. The event driven programming model of Javascript in node.js makes responding to all these events trivially easy. Using treaads is not necessarily difficult but it's a complexity one would rather not deal with.
I thought about Worker threads in Node.js.
One important factor to consider in any discussion of the suitability of a particular programming language for a project is how well the language is already known by the developers and by people who might help the developers.

If a programming language is well known, then everything becomes easier compared to when it isn't.

For this reason the popularities of C/C++, Java and Python are self perpetuating. Since many know those languages well, many new programs are created using those languages. This also explains why a person who knows language X best says the task would be easy in language X.

Re: Programming languages support

Posted: Thu Oct 10, 2019 7:06 pm
by timrowledge
Gavinmc42 wrote:
Thu Oct 10, 2019 1:00 am
https://squeak.org/
Wonder if there is a an Aarch64 version?
Not yet. I’d like to be working on it but other things are in the way. We have x64 for intel devices though so we know the fundamentals work.

Re: Programming languages support

Posted: Thu Oct 10, 2019 7:37 pm
by Heater
@jahboater,
Just set the timeout to zero and select() will return immediately (handy to implement polling).
If you do that you now have a loop thrashing around as fasts as possible polling things and burning electricity.

Or you put a delay in your polling loop, which gets us back the upsetting responsiveness of other parts of your program problem and introducing latency.
Select() is a Linux/UNIX system call, I wonder if its possible to strace node and see if its calling select() under the covers
Node.js uses libuv under the hood. The modern way to do asynchronous programming in C in a cross-platform manner. Which in turn uses epoll, kqueue, IOCP, event ports.

@ejolson

Certainly there are a lot of non-technical reasons for language selection. User familiarity, cross-platform support, popularity, availability of useful libraries/modules, ease of taking those libs/modules into use, etc, etc.

For this reason the popularity of JS/node.js is self perpetuating. Since many know that language well, many new programs are creating using that language.
This also explains why a person who knows language X best says the task would be easy in language X.
Quite so. In this case I have made a lot of software in all manner of languages, professionally and otherwise, so I can offer a slightly less biased comparison. Only slightly mind :). People have to pick there own poison at the end of the day.

Re: Programming languages support

Posted: Thu Oct 10, 2019 7:44 pm
by PeterO
Heater wrote:
Thu Oct 10, 2019 7:37 pm
@jahboater,
Just set the timeout to zero and select() will return immediately (handy to implement polling).
If you do that you now have a loop thrashing around as fasts as possible polling things and burning electricity.

Or you put a delay in your polling loop, which gets us back the upsetting responsiveness of other parts of your program problem and introducing latency.
You don't seem to understand how select (or poll) actually work. From poll man page....

Code: Select all

    The timeout argument specifies the number of milliseconds that poll() should block waiting for a file descriptor to become ready.  The  call
       will block until either:

       *  a file descriptor becomes ready;

       *  the call is interrupted by a signal handler; or

       *  the timeout expires.
PeterO

Re: Programming languages support

Posted: Thu Oct 10, 2019 9:01 pm
by Heater
PeterO,

You don's seen to understand what I wrote. I'll try again with reference that man mage:

* a file descriptor becomes ready;

That could be a long time when reading from sockets, serial ports etc.

* the call is interrupted by a signal handler; or

Which may never happen.

* the timeout expires.

Well there you go, either that is a zero or very short time out, in which case you are thrashing around needlessly wasting performance and generating heat.

Or that is a longer timeout in which case you have hung up all processing for the duration.

Likely one can find a middle ground but it's a bit kludgy to my mind.

Re: Programming languages support

Posted: Thu Oct 10, 2019 10:29 pm
by jahboater
Heater wrote:
Thu Oct 10, 2019 9:01 pm
Well there you go, either that is a zero or very short time out, in which case you are thrashing around needlessly wasting performance and generating heat.
You should actually try using it .... then it would become clear.

You may set a zero timeout - which is a simple poll.
You may set an interval.
You may specify no timeout at all - so it blocks indefinitely.

For the last case your entire workload is handling events. If the server is doing other substantial work that's not related to the event driven work, then you probably should use another thread for it. Control requests by the way may be TCP messages to the server, which are handled as just another type of event.

So a program that is handling events only does:-

select() returns and you deal with all the events.
Call select() again with no timeout and repeat.

So very simple.
Its only doing work when its processing the events, otherwise the CPU overhead is zero.
No wasted time "thrashing around generating heat".

That's just about the simplest of many scenarios. Select() is a remarkably clever and flexible system call.

I guess the benefit JS provides is to convert events into callbacks. Accept registration of event handlers, and other administrative stuff.
An advantage of doing it in C is you can write your own scheduler directly.

Incidentally, the interval may be set very precisely and select() used to be used as a precise high resolution sleep function before nanosleep() arrived.