I deployed a Python script to simulate bells and chimes at our local Model Town and also to play organ music in the Chancel. This was tested fairly extensively at home, but there were limitations to what I could do, because some of the hardware, by necessity, had to be installed into the model.
When I first deployed it, it seemed to work OK and was left running for around 5 to 6 days before anything needed to be changed. No problems were noticed. I then did some minor enhancements to the software and re-deployed and almost immediately the script began to reset itself after running for a number of hours. I accordingly rolled back the software to the earlier version.
I have now run the new version at home for several days using my limited hardware test rig, with no issues. Also, the original version of the software has been exhibiting similar issues. I have been monitoring memory usage and it is being consumed, but I still have nearly 500 MB of RAM left, so I'm discounting that at the moment.
So I am currently considering temperature as the culprit (it has been much warmer since I changed the software). However, I have monitored the CPU temperature while the fault was occurring and it isn't really any more than I get here at home ~48 deg. The system is in a fully enclosed metal case (an old 7.62 mm ammo case) so there is no fresh air ventilation. The Pi (a 3) has its full complement of heat sinks and the case has a finned heatsink on the inside to help transfer heat from the inside to the case wall.
When the reset occurs, it's not a reboot. The script runs in the shell and when a monitor is connected, I can see the messages from the script scrolling up the screen. When the problem occurs, the screen goes balnk for a few seconds, the music stops and then restarts from the beginning.
Any ideas? Do people agree hat temperature is a likely cause of this? If so, do I need to introduce ventilation? The inside of the Minster Model, gets a bit damp when it rains, which is why we didn't put vents in in the first place.
Any other ideas about what could be causing this? We have some switches attached to GPIO pins, through a debouncing network. We did suffer from crosstalk at the beginning, but I solved this with an improved debouncing circuit. I currently dicounting interference, because it doesn't happen until the system has been on for some time, but then it recurs almost immediatly after a reboot.