amcdonley
Posts: 182
Joined: Mon Jan 26, 2015 5:56 pm
Location: Florida, USA

Robot Shared Values / DB impact on RPi SD card

Wed Jul 31, 2019 5:16 pm

My robot is nearing the point of needing a place to share more state and history between Python processes. Currently I have a “json data module” that abstracts access to a few rarely written values persisted on the SD card file system, and two logger files which are written to once or twice an hour.

Considering the complexity of Python’s shared memory pools, and a wish to have a normalized knowledge base to allow loosely coupled behaviors in a blackboard architecture, I am thinking it might be time to add an SQL DB.

Does having a DB on the SD card quickly hasten hitting the card End of life?

blimpyway
Posts: 186
Joined: Mon Mar 19, 2018 1:18 pm

Re: Robot Shared Values / DB impact on RPi SD card

Wed Jul 31, 2019 8:49 pm

If you write infrequently and have a "high/pro endurance" class SD card I would not worry much.

You can also consider in memory databases
https://en.wikipedia.org/wiki/List_of_i ... _databases
Or. if it suits you, a faster in memory key-value datastore like LMDB https://en.wikipedia.org/wiki/Lightning ... d_Database

Or append-only databases. Many databases support append-only mode. Which means you are sure data is not overwritten unless you want it to.
If you write a couple kilobytes, or even megabytes daily do not worry about exhausting to soon a 64GByte card. You can dump/initialise/reload the database once in a few months or years.

amcdonley
Posts: 182
Joined: Mon Jan 26, 2015 5:56 pm
Location: Florida, USA

Re: Robot Shared Values / DB impact on RPi SD card

Thu Aug 01, 2019 1:08 am

Thank you - great response.
blimpyway wrote:
Wed Jul 31, 2019 8:49 pm
consider in memory databases
https://en.wikipedia.org/wiki/List_of_i ... _databases
Or. if it suits you, a faster in memory key-value datastore like LMDB https://en.wikipedia.org/wiki/Lightning ... d_Database
I have heard of, but know nothing about, SQLite which I saw is in the wiki you linked - There is a YouTube at the bottom of the entry - seems like a good place to get smarter.

and then I should be able to recognize what is and isn't in the LMDB - I am guessing it might be well suited if I try reasoning on knowledge in RDF / knowledge graphs. (When my robot learns to learn without me.)

Return to “Automation, sensing and robotics”