Liberation through Computer Literacy

[Heater]

I blame the "computer scientists".

They have created a computing world where the same program will produce different results depending on what machine it is run on.

If mathematicians lived on such shifting ground we would get nowhere.

In this case it was the ordering of file names when asking for a directory listing, who knows what it might be next.

[John_Spikowski]

How well languages scale isn't a topic often discussed.

[Heater]

jcyr,

Did yours look as neat as that one? Because that is really sweet.

Love to see the other side of the board !

[John_Spikowski]

Heater,

You would have a blast building the RPi Tank.

[bensimmo]

I blame the "computer scientists".

They have created a computing world where the same program will produce different results depending on what machine it is run on.

If mathematicians lived on such shifting ground we would get nowhere.

In this case it was the ordering of file names when asking for a directory listing, who knows what it might be next.

Could be anything
A decimal point here, rounding there...

For this, using the languages built in methods over making your own calls to system/os functions is the safer option.

They need to improve their literacy of the code to understand what happens, if using someone elses.
If the Arduino code base going to suffer from similar? It's very much a cut and paste and hope culture.

[gkreidl]

I thought this article goes quite well with the current topic title

Mainly the last part, "The hope is that this paper gets us to talk a bit more about how we treat and view software that we exchange back and forth," said Williams. "We somehow naively assume this stuff will work, being experimentalists who don't have a lot of background in computer science."

Chemists bitten by Python scripts: How different OSes produced different results during test number-crunching
https://www.theregister.co.uk/AMP/2019/ ... n_scripts/

Simply the result of bad programming. os.listdir calls an OS function, which may work differently on different platforms. From the Python documentation:

os.listdir(path)
Return a list containing the names of the entries in the directory given by path. The list is in arbitrary order.

[bensimmo]

I thought this article goes quite well with the current topic title

Mainly the last part, "The hope is that this paper gets us to talk a bit more about how we treat and view software that we exchange back and forth," said Williams. "We somehow naively assume this stuff will work, being experimentalists who don't have a lot of background in computer science."

Chemists bitten by Python scripts: How different OSes produced different results during test number-crunching
https://www.theregister.co.uk/AMP/2019/ ... n_scripts/

Simply the result of bad programming. os.listdir calls an OS function, which may work differently on different platforms. From the Python documentation:

os.listdir(path)
Return a list containing the names of the entries in the directory given by path. The list is in arbitrary order.

or is it bad programming, from the creators point of view it worked perfectly fine for their needs. It worked every time.
It's all the other people not literate in computer programming who just use it that may see the problem, but they are not the original intended user.
science principles worked well here, finding and sorting the problem.

[John_Spikowski]

I try to stay away from languages that require declaring and framework definitions.

[Heater]

gkreidl,

One could take a different point of view.

Fundamentally any given program, when presented with the same data should produce the same result.

If that is not the case then we don't have a program, we have a kind of random number generator. Designers and implementers of programming languages, operating systems, and all other software go to huge lengths at great effort to ensure that this determinism holds true. When it is not we call them bugs.

From a CS is perspective we like to think that all our computers are equivalent to Turing machines. Turing machines are very rigorously defined.

In this case, the same program, with the same data, did not produce the same results. If we brush that off as "OK" then we are being dishonest about our profession.

Yes, there is a get out clause, an exception, a warning, in the small print, buried in reams of documentation. There often is. That is no excuse.

People whose lives are dedicated to something other than CS and the endless details of computing systems should not be expected to suffer this kind of unpredictability from computers.

[Heater]

John_Spikowski,

I try to stay away from languages that require declaring and framework definitions.

How does that help in this case?

What does ScriptBasic do when you list the content of directories in different operating systems?

[Heater]

jcyr,

There are levels of literacy. Learning Python or Bash scripting in my opinion is a pretty low bar..

Woo, what? I have programmed all kind of things in all kind of languages for decades, from assembler to C++ to Verilog. From micro controllers to GUI applications to cloud services.

Bash scripting is beyond me. Gives me headache. I leave it to the gurus.

All of computer programming is nothing but: Sequence, Selection and Iteration. If one has mastered that in Python what else are you expecting them to have to know to be "literate"?

[ejolson]

From a CS is perspective we like to think that all our computers are equivalent to Turing machines.

Even with 4GB, the memory in the new Raspberry Pi is far from infinite. Still, given the way people write programs, it would seem we do like to think our computers are equivalent to machines with infinite memory.

One thing clear from the Fibonacci challenge is that the runtime changes each time a program executes. More generally any type of parallelism or time slicing can introduce indeterminacy in the output of a program. For example, a typical way to run the map-reduce algorithm is to return the first 20 solutions found and then cancel the rest of the parallel calculation. Depending on differences in network, storage and processor latencies, which solutions are found first can vary from run to run.

Many years ago the automatic tuning used in the FFTW Fourier transform library led to nondeterminism in some numerical simulations I conducted regarding data assimilation and the synchronisation of nonlinear dynamical systems. Depending on how the operating system allocated the arrays in physical memory, the FFTW library ordered the floating point operations used to compute the transform in a sightly different way. This resulted in a change in the rounding errors from run to run which, due to the sensitivity of the calculation on initial conditions, led to very different solution trajectories by the end of otherwise identical computations.

There are also algorithms such as genetic optimisation, simulated annealing, stochastic parameterisation, the ensemble Kalman filter and Monte Carlo integration, among others, which intentionally introduce randomness for various reasons. The outputs when running programs which implement such algorithms are different each time. Even passwords encrypted with salt result in a different hash each time.

I personally think it would be possible to let the Bayesian inference and back propagation steps in a deep-learning neural network race in parallel when making updates and that the algorithm would still work. Not having to block or synchronise the different threads could result in huge performance gains; however, the topic about robots taking over the world because people can't code in Python was a few months ago.

[ejolson]

John_Spikowski,

I try to stay away from languages that require declaring and framework definitions.

How does that help in this case?

What does ScriptBasic do when you list the content of directories in different operating systems?

Might it check the environment variable LC_COLLATE and then sort the bit patterns used for the filenames as if they were UTF-8 Unicode characters in a foreign language?

[gkreidl]

gkreidl,

One could take a different point of view.

Fundamentally any given program, when presented with the same data should produce the same result.

If that is not the case then we don't have a program, we have a kind of random number generator. Designers and implementers of programming languages, operating systems, and all other software go to huge lengths at great effort to ensure that this determinism holds true. When it is not we call them bugs.

From a CS is perspective we like to think that all our computers are equivalent to Turing machines. Turing machines are very rigorously defined.

In this case, the same program, with the same data, did not produce the same results. If we brush that off as "OK" then we are being dishonest about our profession.

Yes, there is a get out clause, an exception, a warning, in the small print, buried in reams of documentation. There often is. That is no excuse.

People whose lives are dedicated to something other than CS and the endless details of computing systems should not be expected to suffer this kind of unpredictability from computers.

If you use any kind of library (which you will always do) in any kind of language without reading the documentation you are simpy stupid.

In C, using glib, you will find exactly the same:

g_dir_open ()
GDir *
g_dir_open (const gchar *path,
guint flags,
GError **error);
Opens a directory for reading. The names of the files in the directory can then be retrieved using g_dir_read_name(). Note that the ordering is not defined.

g_dir_read_name ()
const gchar *
g_dir_read_name (GDir *dir);
Retrieves the name of another entry in the directory, or NULL. The order of entries returned from this function is not defined, and may vary by file system or other operating-system dependent factors.
NULL may also be returned in case of errors. On Unix, you can check errno to find out if NULL was returned because of an error.
On Unix, the '.' and '..' entries are omitted, and the returned name is in the on-disk encoding.
On Windows, as is true of all GLib functions which operate on filenames, the returned name is in UTF-8.

[John_Spikowski]

John_Spikowski,

I try to stay away from languages that require declaring and framework definitions.

How does that help in this case?

What does ScriptBasic do when you list the content of directories in different operating systems?

ScriptBasic directory read is very flexable and produces the same results on all OS platforms.

[ejolson]

John_Spikowski,

I try to stay away from languages that require declaring and framework definitions.

How does that help in this case?

What does ScriptBasic do when you list the content of directories in different operating systems?

ScriptBasic directory read is very flexable and produces the same results on all OS platforms.

I looked at the documentation provided and could not determine whether the sort order depends on the LC_COLLATE environment variable setting or not.

From a literacy point of view using only ASCII encodings with ASCII sort order can be understood in about ten minutes. Moreover, when computers work the same simple way for everyone, standardisation is useful and quickly empowers people all over the world so they can work together and share code.

Unicode, on the other hand, does not unify but instead creates localisations that are different for each community. Rather than liberating people, the product is a complicated collating scheme that separates individuals based on ethnic background. True liberation results from people working together, not from the current style of bugs and political segregation based on collating order.

The standard complaint used to push Unicode is that ASCII is English-language focused and therefore not inclusive. Although it's amusing this argument is often written down using the English language, mushing a bunch of different alphabets into a giant Unicode only compounds the original difficulty of having language-centric alphabets in the first place.

In my opinion, the inclusive solution would be to standardise on one all-purpose phonetic alphabet consisting of visually distinct characters suitable for writing down every human language. This should be possible using 8-bit characters because the number of distinguishable sounds people can make, while far greater than cats and dogs, is likely far less than 256. In fact, according to the lead developer of Fido Basic it would only take five additional characters to include the sounds used in all dog languages and one more for cats.

Aside from that, an example of a universal phonetic alphabet comes from the Indian subcontinent. Devaganari works just as well when writing English as it does for Sanskrit, Hindu or Marathi. Although the language-centric alphabets used for Tamil, Telugu and others are visually different, the phonetic structure is similar. In particular, it would not be difficult to create a single phonetic alphabet to handle all Indian languages. Moreover, given the success demonstrated with writing English using Devaganari, other human languages should be possible to cover without difficulty.

Although a single international phonetic alphabet has already been developed by linguists and used to describe every language ever spoken, why standardise on something simple that would liberate everyone using a computer when Unicode also includes the pile-of-poo character? And don't forget, dogs and even cats are people too.

[John_Spikowski]

Sorry, I never tried it with unicode based filenames. If someone gives it a try, please let us know how it works out.

[Heater]

Devaganari sounds like a neat idea ej.

I'm mean, how many ways does a cat need to be able to say "feed me", let alone write it?

However I fear that even if all the world was mad keen on Devaganari to start with it would soon diverge into splinter groups and factions and collapse in acrimony.

See the history of Volapük: https://www.omniglot.com/writing/volapuk.htm

Might as well embrace the chaos that is Unicode as fast as possible and get it over with.

Soon the Chinese or whoever will throw out the English from it's prime position in the 8 bit code space and replace it with whatever they like. After all, it's not fair that they have to use a less efficient coding when there is so many of them.

[ejolson]

Hmm.. Do you expect anyone to take this seriously?

Only the part about adding five more phonemes to the international phonetic alphabet for the dog languages and one more for the cat.

[Heater]

jcyr,

Do you expect anyone to take this seriously?

Somebody took this seriously enough to make it into a standard for computing.

Anything is possible.

As it happens Devanagari is already present in Unicode... all we need is to move it into the 8 bit code space for efficiency then deprecate the rest of it.

[Heater]

ejolson,

Even with 4GB, the memory in the new Raspberry Pi is far from infinite.

Damn mathematicians, always fussing over irrelevant details. Infinite, 4GB, meh close enough

Physicists would never get anywhere if they listened to the Mathematicians [1][2]

One thing clear from the Fibonacci challenge is that the runtime changes each time a program executes.

True enough.

However I don't recall Turing or the C/C++ standards ever saying anything about time.

Hmm...well, in Turing world programs either halt and produce a result or do not halt. There seem to be a discontinuity between zero time and infinite time !

[1] 'The Unreasonable Effectiveness of Physics in Maths', David Tong https://www.youtube.com/watch?v=UVuKyZ4pBzg

[2] 'The Unreasonable Effectiveness of Quantum Physics in Modern Mathematics' -- Robbert Dijkgraaf
https://www.youtube.com/watch?v=6oWLIVNI6VA

[ejolson]

Infinite, 4GB, meh close enough

Who was it that once said no one would ever need more than 64K?

My understanding is that computer programmers don't need memory any more than a drug addict needs drugs. Providing more in either case only makes the existing problems worse.

For example, implementing associative arrays as linked lists works perfectly well in 16K but falls apart in an insane sort of way when megabytes are used to categorise anagrams. Similarly, web browsing would be much simpler if no computer had more than 64K.

From a liberation through computer literacy point of view, it is much easier for a person to implement any algorithm he or she is able to think up that the computing hardware is capable of performing when the hardware in question has very few capabilities. In particular, it is important the Raspberry Pi be simple enough that understanding through mastery is within reach for typical school children.

On the other hand, debugging a scanner just took a day because because no error messages were displayed when libusb wasn't installed. Therefore, in the same way people learn to swim by falling into a well, maybe it's better to prepare people for the real world by having them struggle with frameworks, service managers and sandboxes from the beginning.

[John_Spikowski]

When all else fails.

PRINT "Got Here"

[Heater]

Ha!

Often my debug print statements are a bit stronger than that, starting with a "F" and ending with a "!". Just to emphasis how over joyed I am that it did finally get there.

[John_Spikowski]

For example, implementing associative arrays as linked lists works perfectly well in 16K but falls apart in an insane sort of way when megabytes are used to categorise anagrams. Similarly, web browsing would be much simpler if no computer had more than 64K.

A good example of using associative arrays in ScriptBasic is caching QuickBooks support table ListID cross references. 5 to 10 X speed improvement by not having to do the queries.

ScriptBasic arrays are focused on dynamic creation, matrix like and no limits on indices. The price is performance. If I need fast fixed sized generic typed arrays it becomes an C extension module function.

The HASH extension module seems to have an internal max size set that may be causing my problem getting the anagram challenge to work.