Heater
Posts: 13269
Joined: Tue Jul 17, 2012 3:02 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 5:30 am

DavidS,
So it is that one of the arguments in favor of Python is that it is as simple as Basic while being more capable than Basic.
Python is far more capable than any language with "BASIC" in it's name that I have seen.
sitting back giggling at the conversation.
...
And now the arguments are that if you wish to use pointers you should not be using Python?
I think it's rude to be giggling at us.

It's also not a wise thing to do given that you are displaying a level of ignorance and misunderstanding that could cause us to rupture a gut laughing at you.

Please consider the following example code, which happens to be Javascript but one can do similar in Python and many other languages:

Code: Select all

let node3 = {
    x: 70,
    y: 80,
    next: null
}

let node2 = {
    x: 50,
    y: 60,
    next: node3
}

let node1 = {
    x: 30,
    y: 40,
    next: node2
}

let node0 = {
    x: 10,
    y: 20,
    next: node1
}

let position = node0
while (1) {
    console.log(position.x, position.y)
    if (position.next) {
        position = position.next
    } else {
        break
    }
}
With the following output:

Code: Select all

10 20
30 40
50 60
70 80
Now, as you look at that code don't you get the eery feeling that the thing I have called "position" there is doing very much the same thing as you might do with a pointer in C ?

In fact with some small syntactic changes that code could be turned into C. Using a pointer for "position" of course.

And yet, there is no sign of anything called a "pointer" in that example. Nothing that even looks like a pointer in the style of C or languages that have pointers.

Are there actual pointers being used behind the scenes, in the interpretter or the native code this might get JITed to? Sure there are. But there are no pointers in the language and pointers are not required.

It would save a lot of confusion and effort here if you would learn something about the things you comment on before commenting on them.

jalih
Posts: 77
Joined: Mon Apr 15, 2019 3:54 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 6:46 am

jahboater wrote:
Sat Jul 13, 2019 10:46 pm
scruss wrote:
Sat Jul 13, 2019 10:24 pm
jahboater wrote:
Sat Jul 13, 2019 8:38 pm
I think I would just mmap the dictionary and keep scanning that ? No need to allocate any memory.
You're going to have to do a heck of a lot of scanning to find where the lines start and stop, and then run a regex on each one to see if it matches the definition of a word or not.
I think that could be done with a single scan, the search regex will just include the newlines as '\012'.
Treat the entire file as one large string. Also the regex code will likely have a special case optimization for this.

The perl code will have to do something similar will it not? even if its not visible to the user.

Interesting problem :)
How about showing some C code to match Perls capabilities? ;)

Just for fun, I tried finding anagrams and displaying sorted output using 8th programming language. I am currently on Windows machine, so I just used finnish word list instead of "/usr/share/dict/words". It also meant, I didn't need to use regular expressions but 8th supports PCRE regular expressions.

Word list that I used was about 1.1 MB and 3693 anagrams were found and runtime is about 4 seconds.

Code: Select all

m:new var, anamap
a:new var, anakeys


: s:sort \ s -- s
  null s:/
  ' s:cmpi a:sort
  "" a:join ;


: process-words \ word --
  s:lc
  dup
  >r
  s:sort
  anamap @
  over
  m:exists? if
    over m:@ r> a:push rot swap m:!
  else
    swap a:new r> a:push m:!
  then
  drop ;


: read-and-check-words
  "kotus_sanat.txt"
  f:open-ro
  ' process-words f:eachline
  f:close ;


: len<  \ key  --
  drop ;


: len>=  \ key --
  anakeys @ swap a:push drop ;


: fetch-ana-list \ key array --
  a:len 2 n:cmp
  1 n:+
  nip
  [ ' len< , ' len>= , ' len>= ]
  swap
  caseof ;


: key-val-cmp          \ key1 key2
  anamap @ swap m:@    \ key1 anamap key2val
  0 a:@ nip            \ key1 anamap val2
  swap rot             \ val2 anamap key1
  m:@ nip              \ val2 key1val
  0 a:@ nip            \ val2 val1
  swap                 \ val1 val2
  s:cmp ;


: sort-keys-by-first-word
  anakeys @ ' key-val-cmp a:sort drop ;


: list-words-1 \ ix value --
  nip space . ;


: list-words \ ix value --
  nip anamap @ swap m:@ nip
  ' list-words-1 a:each
  cr
  drop ;


: app:main
  read-and-check-words
  anamap @ ' fetch-ana-list m:each drop
  sort-keys-by-first-word
  anakeys @ ' list-words a:each drop
  anakeys @ a:len nip "\nAnagrams: %d\n" s:strfmt .
  bye ;

jalih
Posts: 77
Joined: Mon Apr 15, 2019 3:54 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 7:04 am

hippy wrote:
Sat Jul 13, 2019 7:53 pm
There are no data or data structure pointers in Python but it is possible to have function pointers which come in handy. That allowed me to create a quite elegant Finite State Machine design with Python.
Where FSM needs to use pointers? Below is a simple state machine based validator for strings containing numbers written in 8th programming language. Credits to for whoever I first learned this technique from ( Can't remember, sorry!).

Code: Select all


\
\ Simple state machine based string validator for numbers.
\

ns: validate

32 constant SPACE
09 constant TAB

\ Status
0 constant EMPTY
1 constant PARTIAL
2 constant OK
3 constant ERROR

\ State
0 constant S0
1 constant IPART
2 constant FPART
3 constant ESIGN
4 constant EPART

var status
var state
var dword


: make-dword  \ lword hword -- dword
  0xffff n:band 16 n:shl swap
  0xffff n:band
  n:bor ;


: s1
  IPART state !
  PARTIAL status ! ;


: s2
  IPART state ! ;


: s3
  FPART state !
  PARTIAL status ! ;


: s4
  ESIGN state !
  PARTIAL status ! ;


: s5
  OK status ! ;


: s6
  FPART state !
  OK status ! ;


: s7
  ESIGN state !
  PARTIAL status ! ;


: s8
  OK status ! ;


: s9
  ESIGN state !
  PARTIAL status ! ;


: s10
  EPART state !
  PARTIAL status ! ;


: s11
  EPART state ! ;


: s12
  OK status ! ;


[ ( SPACE S0    make-dword dword @ n:= ) ,  ' noop  ,
  ( TAB   S0    make-dword dword @ n:= ) ,  ' noop  ,
  ('+     S0    make-dword dword @ n:= ) , ' s1 ,
  ( '-    S0    make-dword dword @ n:= ) , ' s1 ,
  ( '0    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '1    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '2    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '3    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '4    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '5    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '6    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '7    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '8    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '9    S0    make-dword dword @ n:= ) , ' s2 ,
  ( '.    S0    make-dword dword @ n:= ) , ' s3 ,
  ( 'e    S0    make-dword dword @ n:= ) , ' s4 ,
  ( 'E    S0    make-dword dword @ n:= ) , ' s4 ,
  ( '0 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '1 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '2 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '3 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '4 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '5 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '6 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '7 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '8 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '9 IPART    make-dword dword @ n:= ) , ' s5 ,
  ( '. IPART    make-dword dword @ n:= ) , ' s6 ,
  ( 'e IPART    make-dword dword @ n:= ) , ' s7 ,
  ( 'E IPART    make-dword dword @ n:= ) , ' s7 ,
  ( '0 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '1 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '2 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '3 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '4 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '5 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '6 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '7 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '8 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( '9 FPART    make-dword dword @ n:= ) , ' s8 ,
  ( 'e FPART    make-dword dword @ n:= ) , ' s9 ,
  ( 'E FPART    make-dword dword @ n:= ) , ' s9 ,
  ( '+ ESIGN    make-dword dword @ n:= ) , ' s10 ,
  ( '- ESIGN    make-dword dword @ n:= ) , ' s10 ,
  ( '0 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '1 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '2 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '3 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '4 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '5 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '6 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '7 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '8 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '9 ESIGN    make-dword dword @ n:= ) , ' s11 ,
  ( '0 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '1 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '2 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '3 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '4 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '5 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '6 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '7 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '8 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( '9 EPART    make-dword dword @ n:= ) , ' s12 ,
  ( ERROR status ! ) ] var, state-table


: number
  S0 state !
  EMPTY status !
  ( status @ ERROR n:= if
      break
      2drop
    else
      nip state @ make-dword dword ! state-table @ a:when
    then ) s:each
  status @ ;

\
\  Demo starts here.
\
ns: user


: result?  \ result -- string
  [ "EMPTY" , "PARTIAL" , "OK" , "ERROR" ] swap
  caseof ;


: app:main
  " " validate:number result? . cr
  "-" validate:number result? . cr
  "5e" validate:number result? . cr
  "123.567" validate:number result? . cr
  "932e-10" validate:number result? . cr
  "29e100" validate:number result? . cr
  "123mda45" validate:number result? . cr
  "123" validate:number result? . cr
  "  8." validate:number result? . cr
  " -234e100" validate:number result? . cr
  bye ;
Demo output is:

Code: Select all

EMPTY
PARTIAL
PARTIAL
OK
OK
OK
ERROR
OK
OK
OK
Last edited by jalih on Sun Jul 14, 2019 3:06 pm, edited 1 time in total.

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 7:22 am

Heater wrote:
Sun Jul 14, 2019 4:25 am
I often quip that Python causes climate change and global warming
I noticed. But have you ever looked into the amount of power that goes into bitcoin mining nowadays? If you're worried about energy consumption of computing, it seems to me that deserves priority. Entire cities are taken over by bitcoin miners.
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 7:28 am

mahjongg wrote:
Wed Jul 10, 2019 8:59 pm
don't get me started on the uses of semicolons in Pascal
Brian W. Kernighan agrees with you. I agree with you too, having programmed in ordinary Pascal, and later Object Pascal (the latter is the lesser evil).

An interpreted language will always be slower than a compiled language
How would you define an interpreted language vs. a compiled language? Would you call Java a compiled language?
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 7:40 am

DavidS wrote:
Sun Jul 14, 2019 4:31 am
Pointers are the most natural element of programming.
Hi David, yes, you're right that pointers make it possible to write some beautiful efficient and elegant algorithms, mainly because they help in not having to move data around.

But!

Pointers are hard to grok for some people, and easy to shoot oneself in the foot with if one is not careful.

Programming with pointers is a craftmanship that needs more time and attention than writing in a language that cleans up automatically the stuff that is left behind (namely, the objects and data that are no longer referenced). Time and attention that many people nowadays don't have anymore, or are unwilling to spend.
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 8:00 am

jalih wrote:
Sun Jul 14, 2019 6:46 am
Just for fun, I tried finding anagrams (...) I used finnish word list instead of "/usr/share/dict/words"
I did the same, in Python3, using /usr/share/dict/words (2.5M file, 1 second runtime):

Code: Select all

import collections
anagrams = collections.defaultdict(set)

for word in open("/usr/share/dict/words"):
    word = word.strip().lower()
    key = "".join(sorted(word))
    anagrams[key].add(word)

for k, v in anagrams.items():
    if len(v) > 1:
        print(", ".join(v))
It works by sorting the letters in each word and using that as a key to point to a set of words with exactly those letters. So, given a word list:

ape
shop
posh
super

It builds this dict:

aep -> ape
hops -> shop, posh
eprsu -> super

It then outputs all entries which have multiple words.
Last edited by Michiel O. on Sun Jul 14, 2019 9:00 am, edited 1 time in total.
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

jahboater
Posts: 4674
Joined: Wed Feb 04, 2015 6:38 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 8:01 am

Michiel O. wrote:
Sun Jul 14, 2019 7:28 am
How would you define an interpreted language vs. a compiled language?
Anything language that executes a source file and requires a separate program to run it?

That is:

python myprog.py

node myprog.js

xxxbasic myprog.bas

For the interpreted program you have the overheads of:-

loading the interpreter
translating the source code
optimizing the code (optional)

The times for these three are included in the execution time of the program.
Optimization is therefore always a trade-off.
The interpreter is usually a large program, commonly far far larger than the target program. For example, python is 4.1MB which is a lot to load to execute a hello world program! Node is much larger.

None of that applies to a compiled program.
The compiler can spend as long as it wants optimizing the code, and importantly, the optimizer has access to the entire source file allowing global optimizations that interpreters don't usually do. It may for example prove that a certain piece of code is never executed and remove it.

jahboater
Posts: 4674
Joined: Wed Feb 04, 2015 6:38 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 8:08 am

Michiel O. wrote:
Sun Jul 14, 2019 8:00 am
I did the same, in Python3, using /usr/share/dict/words (2.5M file, 1 second runtime):
That's really elegant! It showcases the expressiveness of Python well.

That would take a good bit of hand coding in C and would not be so clean.

In your example, where did "hops" and "aep" come from?

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 8:26 am

jahboater wrote:
Sun Jul 14, 2019 8:08 am
where did "hops" and "aep" come from?
When you sort the letters in 'shop' and 'posh', you arrive at 'hops'. That is used as the key. Analogous, 'aep' are the letters in 'ape', sorted alfabetically.
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

jalih
Posts: 77
Joined: Mon Apr 15, 2019 3:54 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 8:42 am

Michiel O. wrote:
Sun Jul 14, 2019 8:00 am
I did the same, in Python3, using /usr/share/dict/words (2.5M file, 1 second runtime):
We used the same method but you don't seem to sort results alphabetically by using first word of the anagrams. Or did I miss it? It was one requirement of the original challenge and adds quite a lot more code for my 8th version.

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 8:51 am

jahboater wrote:
Michiel O. wrote: How would you define an interpreted language vs. a compiled language?
Any language that executes a source file and requires a separate program to run it?
I was thinking along the same lines, and we have to be more precise with our definitions. "Executing a source file" implies some sort of conversion from source code to executable code: compilation.

In many modern languages, that 'executable code' takes the form of bytecode (a reduced forth-like set of instructions that work on a stack machine) and this bytecode is executed in a virtual machine. Java, for example, compiles down to bytecode (in a '.class' file), an intermediate representation of the program which is also machine independent. The Java bytecode gets processed by the Java Virtual Machine (JVM) instead of the CPU. You could see the JVM as the 'separate program' you mention above. Usually, the JVM is invoked with the java command and the Java compiler with javac.

Python works similar. When you type 'python myprogram.py', Python first checks if there is a myprogram.pyc ('*.pyc' files contain the compiled code), if not, it compiles the source code to bytecode. The bytecode is then executed/interpreted by the Python Virtual Machine (PVM). Both the compiler and VM are part of the Python executable.

Only very simple interpreted languages such as shells skip the compilation step, and every line of the script is parsed, tokenized and interpreted every time it is encountered. This is not efficient at all, but then, shell scripts don't have to be efficient.

Differences in execution time are often not caused by whether a language is interpreted or compiled, but by the nature of the language: dynamic vs. static. In languages with dynamic typing, lots of decisions have to be made at runtime regarding which exact function needs to be executed based on the types of the arguments. In languages with static typing these decisions can be made beforehand by the compiler, so the runtime has lots less figuring out to do. This causes the execution time differences between Java/C/C++ (statically typed) and Python/Ruby (dynamically typed).

Even if you go down to the metal and write your programs in machine language or assembly, you're still executing high level instructions on a CPU core, which below the substrate, uses an instruction decoder and microcode to 'interpret' your machine code program. Only in this case, the 'virtual machine' is implemented in hardware :mrgreen:
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 8:58 am

jalih wrote:
Michiel O. wrote: I did the same, in Python3, using /usr/share/dict/words (2.5M file, 1 second runtime):
We used the same method but you don't seem to sort results alphabetically by using first word of the anagrams. Or did I miss it? It was one requirement of the original challenge and adds quite a lot more code for my 8th version.
Oops! I forgot the sorting. I added it, see below. It took two statements.

• When composing a line of output, I now sort the set with anagrams, so they come out alfabetically: evilness, liveness, veinless, vileness, vineless

• I create and collect the output lines in a list first. Then at the end, I output this list in a sorted manner.

This way, the entire output is sorted, from start to end, and on each line, too.

Code: Select all

import collections
anagrams = collections.defaultdict(set)

for word in open("/usr/share/dict/words"):
    word = word.strip().lower()
    key = "".join(sorted(word))
    anagrams[key].add(word)

lines = []
for k, v in anagrams.items():
    if len(v) > 1:
        lines.append(", ".join(sorted(v)))
print("\n".join(sorted(lines)))  
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

Heater
Posts: 13269
Joined: Tue Jul 17, 2012 3:02 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 9:08 am

hippy,

Thanks for the FSM example. It's a fine FSM but suffers from the same size and complexity as many others I have seen around the net. A simple FSM in C can be just a "while(1)" wrapped around a "switch(state)". Similarly in many other languages.

Oddly I see no function pointers in your example :)

jalih
Posts: 77
Joined: Mon Apr 15, 2019 3:54 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 9:09 am

Michiel O. wrote:
Sun Jul 14, 2019 8:58 am
This way, the entire output is sorted, from start to end, and on each line, too.
That works! :) I chose to collect all the keys with more than one item and sort the list based on content.

How about execution time now?

Heater
Posts: 13269
Joined: Tue Jul 17, 2012 3:02 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 9:19 am

Michiel O. latest Python anagram code running on my x86 PC:

Code: Select all

$ ls -lh /usr/share/dict/british-english-insane
-rw-r--r-- 1 root root 6.6M Apr 25  2018 /usr/share/dict/british-english-insane
michael@monster:/mnt/c/Users/michael/Documents$ wc -l /usr/share/dict/british-english-insane
654276 /usr/share/dict/british-english-insane
michael@monster:/mnt/c/Users/michael/Documents$ time python3 anagrams.py | wc -l
55286

real    0m2.340s
user    0m1.781s
sys     0m0.531s
55 thousand anagrams from 654 thousand words in 2.3 seconds.

zwieblum
Posts: 19
Joined: Sun Jul 07, 2019 6:55 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 11:32 am

just to give bash a voice in this thread:

Code: Select all

#!/bin/bash

declare -A assoc

for i in $(cat /usr/share/dict/words); do
    s=$(echo $i | tr '[:upper:]' '[:lower:]' | grep -o . | sort | tr -d '\n')
    assoc[$s]="${assoc[$s]} $i"
done

for i in ${!assoc[@]}; do
    if [ $(echo ${assoc[$i]}|wc -w) -gt 1 ]; then
        echo ${assoc[$i]}
    fi
done
runtime:
real 24m51,835s
user 16m44,783s
sys 15m55,409s

:)

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 12:08 pm

Heater wrote: It's a fine FSM but suffers from the same size and complexity as many others I have seen around the net. A simple FSM in C can be just a "while(1)" wrapped around a "switch(state)".
Can you give a small example?

Below is my take on a finite state machine in Python2:

- Related functions which belong to a state are kept together in a class.

- No global variables, using object-oriented design.

It implements the following example: a traffic light alternates between green and blue, but immediately jumps to red if an alarm button is pressed. After a while, the traffic light switches back to alternating green/blue.

It's a bit more verbosy than I like, but OTOH, I think it's quite maintainable. I think if I would revisit the code in 6 months, I wouldn't have too much difficulty understanding it.

Code: Select all

import time
import random  # Used in simulating a keypress


class State(object):
    """Base class for state objects. This is meant to be subclassed for every possible state.
       Do not instantiate or call this base class directly."""

    def __init__(self, statemachine, name, timeout_ms, pollfreq_ms):
        self.statemachine = statemachine
        self.name = name
        self.timeout_ms = float(timeout_ms)
        self.pollfreq_ms = float(pollfreq_ms)

    def enter(self):
        """Called when this state is made the active state."""
        print "%13.1lf State %s is entered" % (time.time(), self.name)
        self.run_until = time.time() + self.timeout_ms / 1000

    def leave(self):
        """Called when this state is made inactive."""
        print "%13.1lf State %s is left" % (time.time(), self.name)

    def timeout(self):
        """Called when this state is made inactive due to a timeout.
           Return a state instance to switch to the state that should become the active state."""
        print "%13.1lf State %s times out" % (time.time(), self.name)
        return None

    def loop(self):
        """Repeatedly called during the period the state is active.
           Return a state instance to switch to that state, None if no state switch is necessary."""
        print "%13.1lf State %s's loop is called" % (time.time(), self.name)
        return None

    def run(self):
        self.enter()
        while True:
            # Has this state run out of time?
            if time.time() >= self.run_until:
                new_state = self.timeout()
                self.leave()
                return new_state
            # Poll the state's loop function, possibly switching to a new state.
            new_state = self.loop()
            if new_state is not None:
                self.leave()
                return new_state
            # Wait a bit and repeat.
            time.sleep(self.pollfreq_ms / 1000)



class Green(State):
    def __init__(self, statemachine, name, timeout, pollfreq):
        super(Green, self).__init__(statemachine, name, timeout, pollfreq)

    def enter(self):
        super(Green, self).enter()
        print "              Turn the Green light on"

    def leave(self):
        super(Green, self).leave()
        print "              Turn the Green light off"

    def timeout(self):
        super(Green, self).timeout()
        return self.statemachine.blue_state

    def loop(self):
        super(Green, self).loop()
        print "              State %s is active. See if a button is pressed" % self.name
        if random.uniform(1, 100) > 95:
            print "              Randomly simulating a button press"
            return self.statemachine.alarm_state
        return None


# The Blue state is nearly identical to the Green state

class Blue(State):
    def __init__(self, statemachine, name, timeout, pollfreq):
        super(Blue, self).__init__(statemachine, name, timeout, pollfreq)

    def enter(self):
        super(Blue, self).enter()
        print "              Turn the Blue light on"

    def leave(self):
        super(Blue, self).leave()
        print "              Turn the Blue light off"

    def timeout(self):
        super(Blue, self).timeout()
        return self.statemachine.green_state  # Back to Green

    def loop(self):
        super(Blue, self).loop()
        print "              State %s is active. See if a button is pressed" % self.name
        if random.uniform(1, 100) > 95:
            print "              Randomly simulating a button press"
            return self.statemachine.alarm_state
        return None



class Alarm(State):
    def __init__(self, statemachine, name, timeout, pollfreq):
        super(Alarm, self).__init__(statemachine, name, timeout, pollfreq)

    def enter(self):
        super(Alarm, self).enter()
        print "              Turn the Red light on"

    def leave(self):
        super(Alarm, self).leave()
        print "              Turn the Red light off"

    def timeout(self):
        super(Alarm, self).timeout()
        return self.statemachine.green_state  # Back to the Green/Blue cycle. You could return None to stop the FSM.

    def loop(self):
        super(Alarm, self).loop()
        return None



class FSM(object):
    def __init__(self):
        self.green_state = Green(self, "Green", 2000, 500)
        self.blue_state = Blue(self, "Blue", 2000, 500)
        self.alarm_state = Alarm(self, "Alarm", 4000, 2000)

    def run(self, initial_state):
        active_state = initial_state
        while True:
            new_state = active_state.run()
            if new_state:
                active_state = new_state
            else:
                print "We're done"
                break



if __name__ == "__main__":
    fsm = FSM()
    fsm.run(initial_state=fsm.green_state)
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

hippy
Posts: 5935
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 12:47 pm

DavidS wrote:
Sun Jul 14, 2019 4:31 am
So it is that one of the arguments in favor of Python is that it is as simple as Basic while being more capable than Basic.
Yes, I would agree that was a pretty good argument for Python.
DavidS wrote:
Sun Jul 14, 2019 4:31 am
And now the arguments are that if you wish to use pointers you should not be using Python?
That is correct. Python doesn't support pointers. It is therefore one of the worst languages to use if you want to use pointers.
DavidS wrote:
Sun Jul 14, 2019 4:31 am
Yet pointers are the most natural ellement of programming, they are everywhere in programming, and easier to understand than the abstraction of a variable on which they are built.
And yet Python users, and those using other languages, manage to do everything without pointer support and have few problems in doing that.

I would say most programmers don't care what's going on under the hood, more care about being able to achieve what they want with the abstracted variables they are dealing with.

I thought Michiel O.'s "C optimizes for computer time, while Python optimizes for developer time" was pretty good view of things.

hippy
Posts: 5935
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 1:05 pm

Heater wrote:
Sun Jul 14, 2019 9:08 am
hippy,

Thanks for the FSM example. It's a fine FSM but suffers from the same size and complexity as many others I have seen around the net. A simple FSM in C can be just a "while(1)" wrapped around a "switch(state)". Similarly in many other languages.
To me it seemed simpler, but that's perhaps because I wrote it and fits with what I needed.

My main goal was to be able to concentrate on the states themselves, implementing those easily and simply, rather than anything else about the FSM. Most FSM's I look at seem to make it quite complicated when it comes to defining and implementing the states themselves.

For me it was a case of choosing verbosity and delivering simplicity and clarity ( as I see it ) over efficiency or speed. For what I need each state executes once every 100ms, does very little so there's little to be gained from efficiency or speed. But everything to be gained from being able to implement what I want more easily.
Heater wrote:
Sun Jul 14, 2019 9:08 am
Oddly I see no function pointers in your example :)
They are abstracted away a little, hidden behind the state handling functions.

The "state" variable is a function pointer so "state=First" and "state=Second" ( via "SetState(First)" etc ) then get the appropriate "def First():" or "def Second():" called when one does "state()".

The "First.Init" etc are also local function pointers* within their state functions. Well technically First.Init[1] is. "First.Init="text",First_Init" will have "def First_Init():" called when executing "state.Init[1]()" within the Execute() routine.

(*) Local function pointer virtual methods ? Don't know, don't care; it just works 8-)

jalih
Posts: 77
Joined: Mon Apr 15, 2019 3:54 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 1:06 pm

Michiel O. wrote:
Sun Jul 14, 2019 12:08 pm
Heater wrote: It's a fine FSM but suffers from the same size and complexity as many others I have seen around the net. A simple FSM in C can be just a "while(1)" wrapped around a "switch(state)".
Can you give a small example?
I think what Heater means, it should be as easy and straightforward as table lookup!

My numeric string validator example works like that. I can post PL/I version, it's probably easier to follow.

hippy
Posts: 5935
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 1:25 pm

Heater wrote:
Sun Jul 14, 2019 5:30 am

Code: Select all

let position = node0
while (1) {
    console.log(position.x, position.y)
    if (position.next) {
        position = position.next
    } else {
        break
    }
}
Not intending to pick on you, because I seem to see that type of code everywhere. But why that, a While(True) and a Break to escape that loop rather than ...

Code: Select all

let position = node0
while(position) {
    console.log(position.x, position.y)
    position = position.next
}
or

Code: Select all

let position = node0
do {
    console.log(position.x, position.y)
    position = position.next
} while (position)
I can see cases where a Break makes it easier, especially if it would otherwise be a complicated boolean expression for the terminating condition, but it seems to get used as the de facto coding recipe.

User avatar
scruss
Posts: 2471
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 2:03 pm

Michiel O. wrote:
Sun Jul 14, 2019 8:00 am
I did the same, in Python3, using /usr/share/dict/words (2.5M file, 1 second runtime):
Though you're not doing half the work that my example did:
  • you include proper nouns and possessives and smash everything to lower case: you need to pick out the words that are just lower case alphanumerics
  • your output isn't sorted.
And yet it takes 67% longer to run and uses 54% more memory than Perl …
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

jalih
Posts: 77
Joined: Mon Apr 15, 2019 3:54 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 2:45 pm

scruss wrote:
Sun Jul 14, 2019 2:03 pm
Though you're not doing half the work that my example did:
  • you include proper nouns and possessives and smash everything to lower case: you need to pick out the words that are just lower case alphanumerics
  • your output isn't sorted.
I updated my version for 8th programming language to match your specs:

Code: Select all


m:new var, anamap
a:new var, anakeys


: s:sort \ s -- s
  null s:/
  ' s:cmpi a:sort
  "" a:join ;


: process-words \ word --
  dup /^[a-z]+$/ r:match nip if
    dup
    >r
    s:sort
    anamap @
    over
    m:exists? if
      over m:@ r> a:push rot swap m:!
    else
      swap a:new r> a:push m:!
    then
    drop
  else
    drop
  then ;


: read-and-check-words
  "words.txt"
  f:open-ro
  ' process-words f:eachline
  f:close ;


: len<  \ key  --
  drop ;


: len>=  \ key --
  anakeys @ swap a:push drop ;


: fetch-ana-list \ key array --
  a:len 2 n:cmp
  1 n:+
  nip
  [ ' len< , ' len>= , ' len>= ]
  swap
  caseof ;


: key-val-cmp          \ key1 key2
  anamap @ swap m:@    \ key1 anamap key2val
  0 a:@ nip            \ key1 anamap val2
  swap rot             \ val2 anamap key1
  m:@ nip              \ val2 key1val
  0 a:@ nip            \ val2 val1
  swap                 \ val1 val2
  s:cmp ;


: sort-keys-by-first-word
  anakeys @ ' key-val-cmp a:sort drop ;


: list-words \ ix value --
  anamap @ swap m:@ nip ' s:cmp a:sort
  " " a:join . cr ;


: app:main
  read-and-check-words
  anamap @ ' fetch-ana-list m:each drop
  sort-keys-by-first-word
  anakeys @ ' list-words a:each! drop
  anakeys @ a:len nip "\nAnagrams: %d\n" s:strfmt .
  bye ;
It takes about 2.5 secs to run on my PC.

jalih
Posts: 77
Joined: Mon Apr 15, 2019 3:54 pm

Re: I dont get it... Why is everyone from science schools so obsessed with Perl and Python when we got C?

Sun Jul 14, 2019 2:56 pm

jalih wrote:
Sun Jul 14, 2019 1:06 pm
I think what Heater means, it should be as easy and straightforward as table lookup!

My numeric string validator example works like that. I can post PL/I version, it's probably easier to follow.

Code: Select all


*PROCESS MARGINS(1,140) LIBS(SINGLE,STATIC);

test: procedure options(main);
   dcl buffer char(255) varying;
   dcl bad_ch condition;

   /***************************************/
   /* On unit to handle bad characters.   */
   /***************************************/
   on condition(bad_ch)
     begin;
       display('bad characters in input. Allowed characters are '' 0123456789+-.eE''.');
       goto recover;
     end;

  recover:
     display('Input real?') reply(buffer);
     if verify(buffer,' 0123456789+-.eE') then
       signal condition(bad_ch);

   display(validate_real(buffer));


   /*********************************************/
   /* Validates character array for real number */
   /* Returns:                                  */
   /*   0: EMPTY                                */
   /*   1: PARTIAL                              */
   /*   2: OK                                   */
   /*   3: ERROR                                */
   /*********************************************/
   validate_real: procedure(string) returns(fixed bin(31));
     dcl string char(*) varying;

     dcl (i, state, status) fixed bin(16) unsigned;
     dcl ch char(1);

     dcl space_ch char(1) value('20'x);
     dcl tab_ch char(1) value('09'x);

     define ordinal STATUS(EMPTY, PARTIAL, OK, ERROR);
     define ordinal STATE(S0, IPART, FPART, ESIGN, EPART);

     i = 1;
     state = binvalue(S0);
     status = binvalue(EMPTY);

     do while(status ^= binvalue(ERROR) & i <= length(string));
       ch = substr(string,i,1);
       select(makelong(state,asc(ch)));
         when(makelong(binvalue(S0),asc(space_ch)), makelong(binvalue(S0),asc(tab_ch)))
           i+=1;
         when(makelong(binvalue(S0),asc('+')), makelong(binvalue(S0),asc('-')))
           do;
             i+=1;
             state = binvalue(IPART);
             status = binvalue(PARTIAL);
           end;
         when(makelong(binvalue(S0),asc('0')), makelong(binvalue(S0),asc('1')), makelong(binvalue(S0),asc('2')),
              makelong(binvalue(S0),asc('3')), makelong(binvalue(S0),asc('4')), makelong(binvalue(S0),asc('5')),
              makelong(binvalue(S0),asc('6')), makelong(binvalue(S0),asc('7')), makelong(binvalue(S0),asc('8')),
              makelong(binvalue(S0),asc('9')))
           state = binvalue(IPART);
         when(makelong(binvalue(S0),asc('.')))
           do;
             i+=1;
             state = binvalue(FPART);
             status = binvalue(PARTIAL);
           end;
         when(makelong(binvalue(S0),asc('e')), makelong(binvalue(S0),asc('E')))
           do;
             i+=1;
             state = binvalue(ESIGN);
             status = binvalue(PARTIAL);
           end;
         when(makelong(binvalue(IPART),asc('0')), makelong(binvalue(IPART),asc('1')), makelong(binvalue(IPART),asc('2')),
              makelong(binvalue(IPART),asc('3')), makelong(binvalue(IPART),asc('4')), makelong(binvalue(IPART),asc('5')),
              makelong(binvalue(IPART),asc('6')), makelong(binvalue(IPART),asc('7')), makelong(binvalue(IPART),asc('8')),
              makelong(binvalue(IPART),asc('9')))
           do;
             i+=1;
             status = binvalue(OK);
           end;
         when(makelong(binvalue(IPART),asc('.')))
           do;
             i+=1;
             state = binvalue(FPART);
             status = binvalue(OK);
           end;
         when(makelong(binvalue(IPART),asc('e')), makelong(binvalue(IPART),asc('E')))
           do;
             i+=1;
             state = binvalue(ESIGN);
             status = binvalue(PARTIAL);
           end;
         when(makelong(binvalue(FPART),asc('0')), makelong(binvalue(FPART),asc('1')), makelong(binvalue(FPART),asc('2')),
              makelong(binvalue(FPART),asc('3')), makelong(binvalue(FPART),asc('4')), makelong(binvalue(FPART),asc('5')),
              makelong(binvalue(FPART),asc('6')), makelong(binvalue(FPART),asc('7')), makelong(binvalue(FPART),asc('8')),
              makelong(binvalue(FPART),asc('9')))
           do;
             i+=1;
             status = binvalue(OK);
           end;
         when(makelong(binvalue(FPART),asc('e')), makelong(binvalue(FPART),asc('E')))
           do;
             i+=1;
             state = binvalue(ESIGN);
             status = binvalue(PARTIAL);
           end;
         when(makelong(binvalue(ESIGN),asc('+')), makelong(binvalue(ESIGN),asc('-')))
           do;
             i+=1;
             state = binvalue(EPART);
             status = binvalue(PARTIAL);
           end;
         when(makelong(binvalue(ESIGN),asc('0')), makelong(binvalue(ESIGN),asc('1')), makelong(binvalue(ESIGN),asc('2')),
              makelong(binvalue(ESIGN),asc('3')), makelong(binvalue(ESIGN),asc('4')), makelong(binvalue(ESIGN),asc('5')),
              makelong(binvalue(ESIGN),asc('6')), makelong(binvalue(ESIGN),asc('7')), makelong(binvalue(ESIGN),asc('8')),
              makelong(binvalue(ESIGN),asc('9')))
           state = binvalue(EPART);
         when(makelong(binvalue(EPART),asc('0')), makelong(binvalue(EPART),asc('1')), makelong(binvalue(EPART),asc('2')),
              makelong(binvalue(EPART),asc('3')), makelong(binvalue(EPART),asc('4')), makelong(binvalue(EPART),asc('5')),
              makelong(binvalue(EPART),asc('6')), makelong(binvalue(EPART),asc('7')), makelong(binvalue(EPART),asc('8')),
              makelong(binvalue(EPART),asc('9')))
           do;
             i+=1;
             status = binvalue(OK);
           end;
         otherwise
           status = binvalue(ERROR);
       end;
     end;

     return(status);
   end validate_real;


   /************************************/
   /* Return ascii code of a character */
   /************************************/
   asc: procedure(c) returns(fixed bin(8) unsigned);
     dcl c char(1);

     dcl 1 char_byte union,
         2 ch char(1),
         2 byte fixed bin(8) unsigned;

     ch = c;
     return(byte);
   end asc;


   /**************************************************************/
   /* Creates a LONG value by concatenating the specified values */
   /**************************************************************/
   makelong: procedure(wLow, wHigh) returns(fixed bin(32) unsigned);
     dcl (wLow, wHigh) fixed bin(16) unsigned;

     dcl 1 long union,
         2 dword fixed bin(32) unsigned,
         2 word(2) fixed bin(16) unsigned;

     word(1) = wLow;
     word(2) = wHigh;

     return(dword);
   end makelong;

end test;

Return to “C/C++”