User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Fri Jun 21, 2019 11:59 pm

AIR's fibo doesn't have the memory requirements of hfibo I already posted it running in a loop with a steady memory use.

My guess is that ScriptBasic isn't releasing its recursive resouces until the program ends. How other languages are storing their recursive state so efficiently is a mystery to me.

AIR has already proved SB doesn't leak.

AIR is still looking at it but isn't the fibo the SB team is recommending. Please use AIR's fibo only.
Last edited by John_Spikowski on Sat Jun 22, 2019 12:17 am, edited 4 times in total.

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Sat Jun 22, 2019 12:02 am

ejolson,
Any help getting Node.js installed on the Pi and where to find four different codes is welcome.
I suggest not installing node.js using "apt-get". The node version installed by Debian/Raspbian is always ancient.

My favorite way to install node is via the Node Version Manager (NVM) script: https://github.com/nvm-sh/nvm

Code: Select all

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash

$ nvm install node 
I have some new JS Fibos to check in. Can you wait half a day of so before rounding them up?
Last edited by Heater on Sat Jun 22, 2019 12:09 am, edited 1 time in total.
Memory in C++ is a leaky abstraction .

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Sat Jun 22, 2019 12:08 am

AIR's fibo has a memory leak. I posted the leaking code to a bug report on the ScriptBacic thread
https://www.raspberrypi.org/forums/view ... 0#p1482283
https://www.raspberrypi.org/forums/view ... 0#p1482334

Last I heard it was not fixed:
https://www.raspberrypi.org/forums/view ... 0#p1482339

I have yet to see where "AIR has already proved SB doesn't leak." with the AIr fibo example. Although it was shown that ScriptBasic releases all memory before exiting. Which is not the same thing of course.
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Sat Jun 22, 2019 12:26 am

I think my testing reached the point where I called the GMP fibo function in a loop with no abnormal memory use. I can't remember how long I ran AIR's fibo in a loop.

I haven't dismissed an issue in GMP2.

ejolson
Posts: 3724
Joined: Tue Mar 18, 2014 11:47 am

Re: A Final Fibonacci Challenge

Sat Jun 22, 2019 12:34 pm

Heater wrote:
Sat Jun 22, 2019 12:02 am
ejolson,
Any help getting Node.js installed on the Pi and where to find four different codes is welcome.
I suggest not installing node.js using "apt-get". The node version installed by Debian/Raspbian is always ancient.

My favorite way to install node is via the Node Version Manager (NVM) script: https://github.com/nvm-sh/nvm

Code: Select all

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash

$ nvm install node 
I have some new JS Fibos to check in. Can you wait half a day of so before rounding them up?
Without prompting, the node installation script seems to have created a ~/.nvm directory, placed a copy of Node.js there and modified my ~/.bashrc to add the executables to the PATH environment variable among other things. I suppose that's good enough for testing.

If I wanted everything to go in /usr/local/nvm what would I do differently?

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Sat Jun 22, 2019 1:49 pm

I don't think nvm caters for global node installation. The whole point really is to manage node versions. This means one can use different versions for different jobs, verify new versions before going to production etc.

This was very useful when node was under such rapid development a few years back. Perhaps not so much now.

I like it because it does not clutter up anything outside my home directory.

You can always copy a node install from ~/.nvm/versions/node/vx.y.z to /usr/local/node/ and set yourself up to use it globally.

I have yet to do the changes to the Fibo Challenge JS directory I want to do.
Memory in C++ is a leaky abstraction .

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Sat Jun 22, 2019 4:52 pm

Suspecting it might be useful to be able to calculate Fibonacci numbers other than F(4784969) I added an input box to my online Fibonacci calculator. It's rough and ready and I suspect it has a memory leak, might hang up if you calculate big fibos too often without reloading.

It breaks the challenge rules by being C++ compiled to web assembly, but hey, it does the job in a couple of seconds, in your browser!

Oh, forgot the link: https://otaniemi.conveqs.fi:3000/public/fibo.html
Memory in C++ is a leaky abstraction .

ejolson
Posts: 3724
Joined: Tue Mar 18, 2014 11:47 am

Re: A Final Fibonacci Challenge

Sun Jun 23, 2019 12:14 pm

Heater wrote:
Sat Jun 22, 2019 4:52 pm
Suspecting it might be useful to be able to calculate Fibonacci numbers other than F(4784969) I added an input box to my online Fibonacci calculator. It's rough and ready and I suspect it has a memory leak, might hang up if you calculate big fibos too often without reloading.

It breaks the challenge rules by being C++ compiled to web assembly, but hey, it does the job in a couple of seconds, in your browser!

Oh, forgot the link: https://otaniemi.conveqs.fi:3000/public/fibo.html
Do you reset the list of memos before calculating a new Fibonacci number? Maybe the code should check if n has changed and when so clear the list.

If you include Fibonacci programs with known memory leaks in the GitHub, it would be useful to include a statement characterising the leak in the program comments.

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Sun Jun 23, 2019 1:00 pm

ejolson,
Do you reset the list of memos before calculating a new Fibonacci number?
No.

Actually I thought I'd removed the memoization altogether, it does not save very much time, turns out I did not...

I was so hung up on managing memory between the Javascript world and the C++ world via emscripten's C++ bindings that I probably forgot to check the obvious!

Having put it aside for a while I'll get back to it with fresh eyes.
Maybe the code should check if n has changed and when so clear the list.
Yeah, and there is a bunch of other user interface things that need attention.
If you include Fibonacci programs with known memory leaks in the GitHub, it would be useful to include a statement characterizing the leak in the program comments.
So far I have not knowingly done so.

My intention was not to include any buggy solutions.
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Sun Jun 23, 2019 7:14 pm

Can you post what fibo versions are known to leak?

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Sun Jun 23, 2019 7:21 pm

ScriptBasic,

Code: Select all

Can you post what fibo versions are known to leak?
See code and bug report in this post on the ScriptBasic thread:

https://www.raspberrypi.org/forums/view ... 0#p1482283
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Sun Jun 23, 2019 8:25 pm

I mentioned the GMP2 issue to AIR.

Fact is all memory used is released by ScriptBasic when the program ends. The GMP2 module works well with SB. I feel the module is usable and safe.

I would still like a definitive answer where that memory is beging used and not released until the script ends.

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 1:50 am

ScriptBasic,
Fact is all memory used is released by ScriptBasic when the program ends.
I'm sure that is true. It's totally pointless though. Linux will reclaim all it's memory back after it has killed a run away ScriptBasic program anyway! That is how things get done no matter what program or what language.
The GMP2 module works well with SB. I feel the module is usable and safe.
No it is not. Did you run that code I posted in the bug report I linked to above. It runs for a few iterations, consumes all the machines memory and then gets killed off.

You have often pointed out that ScriptBasic is used in embedded systems. Such systems are expected to run reliably forever. Any such system using the GMP extension would not be able to do that.
I would still like a definitive answer where that memory is being used and not released until the script ends.
I'm curious to know as well.
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 4:45 am

I can call tbe GMP fibo() function in loop with no memory runaway. (user created buffer)

My gut feeling is it would be better to create your own buffer to return results from rather than letting GMP do it. There is noise on the web about GMP leaking string responses or not being properly cleared.

Your C version works so go figure. You have tried running that In a loop, correct?

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 7:15 am

ScriptBasic,

Please see my reply in the ScriptBasic thread: https://www.raspberrypi.org/forums/view ... 8#p1483988
Memory in C++ is a leaky abstraction .

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 7:26 am

Now we have to do all our Fibo timings again on the Pi 4 :)
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 7:55 am

Heater wrote:
Mon Jun 24, 2019 7:26 am
Now we have to do all our Fibo timings again on the Pi 4 :)
Finally 64 bit with enough ram to run it. If there is no Raspian 64, I'm going with Ubuntu Mate for the RPi.

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 9:08 am

ScriptBasic,
Finally 64 bit with enough ram to run it.
Not really. ScriptBasic eats all the 8GB of my PC when running the fibos. I'm sure it could happily gorge itself on 16, 32 ... MB !

Most code I run on the Pi, including all our other language fibos uses very little of the old Pi's 1GB.
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 6:03 pm

You should probably avoid ScriptBasic.

Appreciate the bug report.

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 6:31 pm

ScriptBasic,

I noticed you edited your post there.

I'm quite honored to be likened to PeterO. Thank you.

Sorry about the snipey remark about ScriptBasic's memory hygiene. I was bit frustrated that my original bug report seemed not to have hit the spot.

As it turns out I'm having the same issue with using my big integer functions in C++ from Javascript. Works fine but does not seem to clean up memory.

I'm sure we will both get there in the end.
Last edited by Heater on Mon Jun 24, 2019 9:16 pm, edited 1 time in total.
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 7:11 pm

My point is this is a GMP integration issue and not a bug in ScriptBasic.

Heater
Posts: 13694
Joined: Tue Jul 17, 2012 3:02 pm

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 7:29 pm

I don't think anyone said it was.

Makes no difference anyway, it needs fixing.

Finding out and understanding what the problem is may prevent similar head scratching for future extension developments.
Memory in C++ is a leaky abstraction .

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Mon Jun 24, 2019 8:10 pm

I fully agree and we should all be working together to resolve the mystery memory issuse.

User avatar
John_Spikowski
Posts: 1592
Joined: Wed Apr 03, 2019 5:53 pm
Location: Anacortes, WA USA
Contact: Website Twitter

Re: A Final Fibonacci Challenge

Tue Jun 25, 2019 2:05 am

Is it my understanding all fibos using GMP leak? How does the Python GMP extension do running in a loop?

Maybe @hippy's approach using the pointer set of GMP functions is a cleaner way?

gkreidl
Posts: 6120
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: A Final Fibonacci Challenge

Tue Jun 25, 2019 4:39 am

ScriptBasic wrote:
Tue Jun 25, 2019 2:05 am
Is it my understanding all fibos using GMP leak? How does the Python GMP extension do running in a loop?
No leak.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

Return to “General programming discussion”