Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 8:22 am

ejolson,
Much to my astonishment... automatically downloaded, compiled and installed 63 crates.
To be fair a lot of that downloading and building was for the stuff required to compile Rust to WASM and generate the bindings required to run it from JS in node.js and in the browser. Did you tried starting the server.py and visiting the anagram page it serves up?

Also, in the C/C++ I would have had you downloading HashBrown and other bits and bobs manually from their git repos and building/installing them. Or I might have provided a BASH script to do all that, which would have failed as such things usually do. The Rust Cargo package manager is brilliant in that respect.
Have you watched this presentation about cache-friendly hash tables?
You have discovered my secret weapon there. You see HashBrown is a go faster hash table in C++ from Google that somebody converted to Rust. The presenter in that video is Matt Kulukundis from Google. He probably wrote the original HashBrown!

I saw that video a long while ago, didn't pay much attention, what would I do with a hash table? I'll have to watch again now.

I was wondering what the execution time printed by main.rs is on the encabulator is. It's getting to the point that reading the file and writing the output is a significant part of the time reported by "time".
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1392
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

OT: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 5:36 pm

Remember my comment about open source being like setting out a canvas and asking artists to contribute?

Lego City

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 5:48 pm

Yes, and what a beautiful city we build. Even if our part is just a few bricks in a dark back alley on the wrong side of town.
Memory in C++ is a leaky abstraction .

User avatar
bensimmo
Posts: 4175
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 5:53 pm

Open source also has its downsides.
http://wiringpi.com/wiringpi-deprecated/

User avatar
John_Spikowski
Posts: 1392
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 6:00 pm

bensimmo wrote:
Sat Aug 10, 2019 5:53 pm
Open source also has its downsides.
http://wiringpi.com/wiringpi-deprecated/
Perfect example ruining a good thing over greed and selfishness.

Open source maintainers are heroes not whores.

Static linking is only shooting your own project in the foot.

Depending how this unfolds, I might create a WiringPi project in the AllBasic Sandbox Gitlab CE repository to continue the development effort. @Hippy would be a great maintainer for this project.
Last edited by John_Spikowski on Sat Aug 10, 2019 6:34 pm, edited 2 times in total.

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 6:18 pm

Sounds like poor Gordon is a victim of success. WiringPi was obviously very good and popular.

I don't know why he says "deprecated" there. It's just an open source project whose maintainer is stepping down and in need of a new one. Happens all the time.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 3578
Joined: Tue Mar 18, 2014 11:47 am

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 6:42 pm

bensimmo wrote:
Sat Aug 10, 2019 5:53 pm
Open source also has its downsides.
http://wiringpi.com/wiringpi-deprecated/
Fido became barking mad: The development of FidoBasic will always remain open source.

In particular, barked the canine coder, I will not release a language and then rant about open-source as an excuse to work on a proprietary version. When asked if there was a GitHub repository for FidoBasic, the developer looked frightened and whined, the octocat is scratchy.

Image

I replied, since Bill is a dog person maybe things will improve with the Microsoft acquisition. After that Fido calmed down and said with a hopeful woof, that would definitely help avoid a dogital apocalypse.

User avatar
davidcoton
Posts: 4147
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 9:28 pm

ejolson wrote:
Sat Aug 10, 2019 6:42 pm
I replied, since Bill is a dog person maybe things will improve with the Microsoft acquisition.
LifeBuzz wrote:Due to GDPR compliance, LifeBuzz is currently unavailable in your country. We are sorry for the inconvenience, and are working quickly to make LifeBuzz compliant in your region. If you have previously created a LifeBuzz account, and want to delete your account and all existing data stored on our servers, please email support@lifebuzz.com
Does that help or hinder the dogital apocalypse?
Signature retired

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Sat Aug 10, 2019 9:40 pm

From the lifebuzz media page:

"LifeBuzz was founded by publishing and advertising veterans who are experts at expanding audiences and generating quality traffic through social buzz. Our talented editors and writers curate and create inspirational content that celebrates how awe-inspiring and wonderful our world can be."

Sounds awful but given that mission statement I find it hard to believe the GDPR is any reason to not be allowed in Europe. What are they really doing?

"The planning and care we have built our brand around allows advertisers to reap measurable results."

Sounds even more awful. I guess you have to sign up and they sell all the data they get from you.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 3578
Joined: Tue Mar 18, 2014 11:47 am

Re: Project Digital Apocalypse Not Now

Sun Aug 11, 2019 12:22 am

Heater wrote:
Sat Aug 10, 2019 9:40 pm
Sounds even more awful. I guess you have to sign up and they sell all the data they get from you.
I didn't sign up; maybe Fido did. I wonder why the site is censored where you live. The apocalypse may be nearer than anyone thought.

The web page in question appears to be written by a blogger on the results of this ask me anything question-answer session with Bill Gates--presumably the one who started Microsoft.

To the question "Do you have a pet? If so what’s his/her name?"
Bill Gates wrote:We have two dogs. One is Oreo and the other is Nilla. I will say I spend less time with the dogs than the kids do but I really like them (when they are not barking at night and not eating things they are not supposed to and when they are well house trained). I have resisted getting a third dog.
Does either Oreo or Nilla have a GitHub account? An enquiring dog would like to know.

ejolson
Posts: 3578
Joined: Tue Mar 18, 2014 11:47 am

Re: Project Digital Apocalypse Not Now

Sun Aug 11, 2019 4:15 am

Heater wrote:
Sat Aug 10, 2019 8:22 am
I was wondering what the execution time printed by main.rs is on the encabulator is. It's getting to the point that reading the file and writing the output is a significant part of the time reported by "time".
Since Fido had deleted the instrumentation in main responsible for printing those times, I cloned the repository into a fresh directory and typed

Code: Select all

$ cargo build --release
Three more crates were downloaded and then to my surprise began the recompilation of all 68 crates including bumpalo, weedle and the hashbrown library. After this insane delay the Rust British anagram program was ready to run.

The output on the Pi 4B was

Code: Select all

$ time ./insane-british-anagram >insane.txt
439ms
305ms

real    0m0.926s
user    0m0.581s
sys 0m0.217s
$ time ./insane-british-anagram >insane.txt
437ms
308ms

real    0m0.797s
user    0m0.585s
sys 0m0.212s
Interestingly, the executable is now 6.2MB in size whereas yesterday it was only 2.4MB. For reference zanagram.c compiles to a 42KB executable which originally seemed large compared to the 13KB size of qanagrams.

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Sun Aug 11, 2019 6:18 am

ejolson,

That does seem rather large.

Despite being release builds those binaries are not stripped. Stripping the executables gets them down to 200K.

Except the last version I checked in is using the jemallocator which looks like it might be a bit faster. However it bloats the stripped executable up to 400K !

I have checked in a new version that reverts to not timing things twice, so only outputs one copy of the anagrams. It now also builds all the old versions I have. And strips them. There is a run.sh script to run them all. Sadly the names the old versions have may not match your names on the chart exactly.
Memory in C++ is a leaky abstraction .

jahboater
Posts: 4689
Joined: Wed Feb 04, 2015 6:38 pm

Re: Project Digital Apocalypse Not Now

Sun Aug 11, 2019 6:29 am

Heater wrote:
Sun Aug 11, 2019 6:18 am

That does seem rather large.
Rust programs are statically linked are they not? so that known "safe" functions get used.

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Sun Aug 11, 2019 8:32 am

jahboater
Rust programs are statically linked are they not? so that known "safe" functions get used.
Not quite.

Rust links all Rust modules statically. Not for any safety reasons but because Rust is still quite a new language and they reserve the right to change the ABI. They don't want to break any dynamic Rust libraries people may create. The ABI will likely get frozen one day.

One can build dynamic libs in Rust and use them from Rust or C or any other language. But to do so one has to provide a C ABI compatible interface. That interface is of course "unsafe" code in your Rust source but being small and well contained easy to check for correctness.

As it happens the WASM build of insane-british-anagram uses C style on it's public functions, it's only 43K bytes.
Memory in C++ is a leaky abstraction .

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Sun Aug 11, 2019 7:10 pm

Nothing much happened today... Except my online insane anagram server went rusty, no more pythons there: http://otaniemi.conveqs.fi:9000/
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1392
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 3:59 am

Sounds like it's time for another anafibo challenge.

Something with fractals would be fun.
a curve or geometric figure, each part of which has the same statistical character as the whole. Fractals are useful in modeling structures (such as eroded coastlines or snowflakes) in which similar patterns recur at progressively smaller scales, and in describing partly random or chaotic phenomena such as crystal growth, fluid turbulence, and galaxy formation.
Start with Mandelbrot and branch out from there.

Deepest Mandelbrot Set Zoom Animation ever - a New Record! 10^275 (2.1E275 or 2^915)

ejolson
Posts: 3578
Joined: Tue Mar 18, 2014 11:47 am

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 4:45 am

John_Spikowski wrote:
Mon Aug 12, 2019 3:59 am
Deepest Mandelbrot Set Zoom Animation ever - a New Record! 10^275 (2.1E275 or 2^915)
That's a fun video; however, after the Fibonacci sequence, I'm not sure I could bear another big-number arithmetic challenge.

What do you think about something fun like a graphical network version of the classic star trader game?

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 5:13 am

As much as I love the Mandlebot set I'm afraid I will have to sit that one out.

When the Mandlebrot set hit public consciousness in 1985 or so I was working with a team on a military communications project. All development in that team stopped for about a month as we all got down to trying to draw the thing as fast as possible That's when I made the handy discovery that even when using compilers for 16 bit MS-DOS one could make use of the 32 bit arithmetic instructions of the Intel 386 by crafting it in assembler, adding the 32 bit mode prefix to the various arithmetic operations.

Aside: I later tried to run that code on Windows NT the first time I saw that OS. It killed the OS stone dead immediately!

A graphical networked star trader game sounds fun, except I have no idea what said Star Trader is. Can you describe it in a paragraph? How do we know when we have a correct solution?

Now, if our software solutions, to whatever problem it is, could battle it out among each other over the internet that might be interesting. A kind of networked "core wars". Or is that kind of autonomous agent fighting it out for supremacy over the internet to close to the digital apocalypse?
Memory in C++ is a leaky abstraction .

User avatar
bensimmo
Posts: 4175
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 10:38 am

Maybe you could make an Animal Care Home simulation, rather than trading in Stars?

User avatar
John_Spikowski
Posts: 1392
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 11:34 am

bensimmo wrote:
Mon Aug 12, 2019 10:38 am
Maybe you could make an Animal Care Home simulation, rather than trading in Stars?
🦴

User avatar
scruss
Posts: 2485
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 2:36 pm

ejolson wrote:
Sun Aug 11, 2019 4:15 am
… Interestingly, the executable is now 6.2MB in size whereas yesterday it was only 2.4MB.
One could hide the entire output results (768,081 bytes) in that several times over. Indeed, a program that just prints the results:

Code: Select all

/* selfgrams_cheating.c - scruss, 2019-08 */
#include <stdio.h>

void
main ()
{
  puts ("aah: aha\n"
	"aahed: ahead\n"
	"aahs: ahas\n"
  …
	"zeno: zone\n"
	"zinco: zonic\n" "zolotink: zolotnik\n" "zoography: zoogrpahy");
}
is only about 774 K. It runs wicked fast.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 3:26 pm

scruss,

I have a bug report ready for when the /usr/share/dict/british-english-insane changes and breaks your program :)
Memory in C++ is a leaky abstraction .

ejolson
Posts: 3578
Joined: Tue Mar 18, 2014 11:47 am

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 4:20 pm

Heater wrote:
Mon Aug 12, 2019 3:26 pm
scruss,

I have a bug report readytime for when the /usr/share/dict/british-english-insane changes and breaks your program :)
It wouldn't be the insane British dictionary if it hadn't already changed between Raspbian Stretch and Buster. Fortunately zoogrpahy is in both versions of the dictionary.

Along different lines anagram-sb.bas is only 1062 bytes and just finished running on the Pi 3B+ with the output

Code: Select all

$ time ./scriba anagram-sb.bas >anagram-sb.insane

real    1628m23.595s
user    1619m38.766s
sys 8m36.237s
$ md5sum *.insane
bec74aa3b31577edbb291aeb7269a4d5  anagram-sb.insane
bec74aa3b31577edbb291aeb7269a4d5  selfgrams-pl.insane
The bar chart of fame for the Pi 3B+ has been updated accordingly.

Image

Although many may frown upon stripping in public, the following display shows what happened with some popular programs:

Code: Select all

Program      Source  Binary  Stripped
hanagram.bas   2K      54K     54K
hanagram.c     3K       9K      6K
hanav2.bas     2K      54K     54K
iba-1.rs       4K    2414K    162K
iba-2.rs       3K    2426K    170K
iba-3.rs       5K    2425K    170K
iba-4.rs       6K    2425K    169K
iba-5.rs       6K    2421K    165K
iba-gmp.cpp    4K      23K     14K
iba-hash.cpp   4K      20K     14K
iba-main.rs    6K    2410K    158K
iba-mainv2.rs  6K    6285K    506K
managram.c     3K      61K     58K
qanagrams.c    8K      13K     10K
zanagram.c     5K      42K     38K

Heater
Posts: 13357
Joined: Tue Jul 17, 2012 3:02 pm

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 5:21 pm

Hey, what happened to iba-main.rs, previously charted at 0.462 seconds?

Anyway I was neglected and rusting away when a guru passed by and suggested I try the Rust bench marking tool "cargo bench".

So I added a benchmark test harness to main.rs to time the anagrams function (So excluding reading the dictionary and printing the result)

Code: Select all

#[cfg(test)]
mod tests {
    use super::*;
    use test::Bencher;

    #[bench]
    fn bench_anagrams(b: &mut Bencher) {
        let dictionary = std::fs::read("/usr/share/dict/british-english-insane").unwrap();
        b.iter( || {
            anagrams(&dictionary)
        });
    }
} 
And run it like so:

Code: Select all

$ cargo bench
   Compiling insane-british-anagram v0.1.3 (/mnt/c/Users/michael/conveqs/insane-british-anagram-rust)
...
...
     Running target/release/deps/insane_british_anagram-d47f8595fdf17f6a

running 1 test
test tests::bench_anagrams ... bench:  75,641,200 ns/iter (+/- 852,900)

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured; 0 filtered out   
Do what? 75ms to run the anagram finder function?

Well it fits with the weird result I get from the program timing itself twice:

Code: Select all

$ time target/release/insane-british-anagram > anagrams.txt
Run    1: 306ms
Digest 1: addeda36a4631afc983f3bff13742e36
Run    2: 75ms
Digest 2: addeda36a4631afc983f3bff13742e36
That is all on the PC with the jemallocator linked in (which doubles the executable size)
Using the standard memory allocator it's a more normal 300ms:

Code: Select all

running 1 test
test tests::bench_anagrams ... bench: 299,988,300 ns/iter (+/- 1,784,170)
Now, I'd like to present the results of doing all this on a Pi. But after an hour or so getting cargo bench to build I found it crashed with a segfault. Now my Pi won't let me log in over ssh. I can enter my password and then nothing....

Looks like rust eat my Pi !
Memory in C++ is a leaky abstraction .

ejolson
Posts: 3578
Joined: Tue Mar 18, 2014 11:47 am

Re: Project Digital Apocalypse Not Now

Mon Aug 12, 2019 6:13 pm

Heater wrote:
Mon Aug 12, 2019 5:21 pm
Hey, what happened to iba-main.rs, previously charted at 0.462 seconds?
It wouldn't be insane unless there were two virtually indistinguishable bar charts of fame: one for the 3B+ and the other for the 4B. The earlier chart has not been kept up to date; however, since a particularly lengthy calculation is now over, that should soon change.
Heater wrote:
Mon Aug 12, 2019 5:21 pm
Looks like rust eat my Pi!
Sorry to hear about the Pi problems. Sounds like terminal-reaction SD-card meltdown.

Do you have a properly configured entropy distributor?

Did a hashbrown get stuck in one of the flux ports?

Return to “General programming discussion”