sanpaulus
Posts: 16
Joined: Fri Nov 22, 2019 4:04 pm

Python script automatic start

Sat Dec 07, 2019 1:28 pm

Hello people,

i need help. I have a python script, located on an USB-stick, the path looks like: /media/pi/USBstick/Data
it needs to starts automaticly AFTER the pi boots up completly.
my script looks like this:

Code: Select all

#!/usr/bin/python3
import csv
import time
import datetime
import serial
import urllib.request
import sys
import RPi.GPIO as GPIO

SerialIn = serial.Serial("/dev/ttyUSB0",9600)
tvocURL = 'http://api.thingspeak.com/update?api_key=ZD8WW20D5ZD9WXXX&field1='
eco2URL = 'http://api.thingspeak.com/update?api_key=ZD8WW20D5ZD9WXXX&field2='

    
with open("sensor.csv", "w") as csvfile:
    sensorwriter = csv.writer(csvfile)
    sensorwriter.writerow(["datetime", "field1", "field2"])
    while True:
        temp = SerialIn.readline()
        temp = temp.decode('utf-8').strip().split(',')
        f = urllib.request.urlopen(tvocURL +str(temp[0]))
        f.read()
        g = urllib.request.urlopen(eco2URL +str(temp[1]))
        g.read()
        time_now = datetime.datetime.now()
        sensorwriter.writerow([time_now, temp[0], temp[1]])
        print("{0},{1},{2}". format(time_now, temp[0], temp[1]))
        

f.close()
g.close()
        
time.sleep(15)

Many epic fails happend when i tried different methods and to end my suffering i am seeking help now in this forum.

klricks
Posts: 6704
Joined: Sat Jan 12, 2013 3:01 am
Location: Grants Pass, OR, USA
Contact: Website

Re: Python script automatic start

Sat Dec 07, 2019 3:36 pm

If booting to desktop then use autostart:

Code: Select all

/etc/xdg/lxsession/LXDE-pi/autostart
You must specify the complete path when calling your script.
Also you must specify the complete path when opening any files within your script.
Unless specified otherwise my response is based on the latest and fully updated Raspbian Buster w/ Desktop OS.

sanpaulus
Posts: 16
Joined: Fri Nov 22, 2019 4:04 pm

Re: Python script automatic start

Mon Dec 09, 2019 10:22 am

thank you for the answer.

i found a tutorial describing what you just meant:
https://www.raspberrypi-spy.co.uk/2014/ ... e-desktop/

i tried to run leafpad at autostart. it worked, but my pi isn't booting up now anymore. no desktop. just a black screen.

User avatar
thagrol
Posts: 2097
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Python script automatic start

Mon Dec 09, 2019 1:17 pm

Autostart may not be the correct solution.

As I understand it, anything run that way will only run after login through the gui and will run on every login through the GUI. So if you login locally and via VNC you'll get two copies running. You script will probably get killed on logout too.

To run something after boot use one of the following methods:
  • cron. Via an @reboot cron job. See

    Code: Select all

    man cron
  • /etc/rc.local
  • A systemd service
Each method has advantages and disadvantages so you'll need to do some reading before you decide which to use.

As for your black screen issue, I'ev no idea. The fix is probably to undo your changes. One way to do that is to get a second SD card with a clean OS, boot from that then mount the "broken" card via a USB card reader. You should then be able to undo the changes.
Attempts to contact me outside of thes forums will be ignored unless signed in triplicate, sent in, sent back, queried, lost, found, subjected to public enquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters

klricks
Posts: 6704
Joined: Sat Jan 12, 2013 3:01 am
Location: Grants Pass, OR, USA
Contact: Website

Re: Python script automatic start

Mon Dec 09, 2019 2:24 pm

sanpaulus wrote:
Mon Dec 09, 2019 10:22 am
thank you for the answer.

i found a tutorial describing what you just meant:
https://www.raspberrypi-spy.co.uk/2014/ ... e-desktop/

i tried to run leafpad at autostart. it worked, but my pi isn't booting up now anymore. no desktop. just a black screen.
Your autostart file must contain these 2 lines:

Code: Select all

@lxpanel --profile LXDE-pi
@pcmanfm --desktop --profile LXDE-pi
Right click on the blank screen and select terminal. You should then be able to edit autostart and add those commands.

The tutorial was wrong as it did not show those commands present in method 2.
Unless specified otherwise my response is based on the latest and fully updated Raspbian Buster w/ Desktop OS.

sanpaulus
Posts: 16
Joined: Fri Nov 22, 2019 4:04 pm

Re: Python script automatic start

Mon Dec 09, 2019 6:58 pm

thanks you for helping me! My pi is back to normal conditions.

Code: Select all

@lxpanel --profile LXDE-pi
@pcmanfm --desktop --profile LXDE-pi
did the job.

but the autostart-issue is still not solved.
Tried to get a solution with

Code: Select all

crontab -e
directly tipped in the terminal

than simply scrolled down to the bottom and added the line

Code: Select all

@reboot python3 /home/pi/myscript.py
but nothing at all happens at reboot... what is going on?

klricks
Posts: 6704
Joined: Sat Jan 12, 2013 3:01 am
Location: Grants Pass, OR, USA
Contact: Website

Re: Python script automatic start

Mon Dec 09, 2019 7:19 pm

Did you edit your file and add the complete path to sensor.csv?
Unless specified otherwise my response is based on the latest and fully updated Raspbian Buster w/ Desktop OS.

User avatar
thagrol
Posts: 2097
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Python script automatic start

Mon Dec 09, 2019 7:40 pm

sanpaulus wrote:
Mon Dec 09, 2019 6:58 pm
Tried to get a solution with

Code: Select all

crontab -e
directly tipped in the terminal

than simply scrolled down to the bottom and added the line

Code: Select all

@reboot python3 /home/pi/myscript.py
but nothing at all happens at reboot... what is going on?
Are you sure nothing at all is happening at reboot? Your script will be started as a background task with no connection to a terminal so all output will be lost.

Code: Select all

ps aux | grep myscript.py
will show the process informatiom if you script is running. If it shows nothing, your script either didn't start or exited shortly after starting.

Either way, try changing you crontab entry to the one below, rebooting then looking at the contents of /tmp/myscript.log

Code: Select all

@reboot python3 /home/pi/myscript.py >/tmp/myscript.log 2>&1
Attempts to contact me outside of thes forums will be ignored unless signed in triplicate, sent in, sent back, queried, lost, found, subjected to public enquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters

sanpaulus
Posts: 16
Joined: Fri Nov 22, 2019 4:04 pm

Re: Python script automatic start

Mon Dec 09, 2019 9:21 pm

ps aux | grep myscript.py
tried that to check if the script is running. once while the script is running, and to compare once without running my script.
so yeah, the

Code: Select all

 crontab -e @reboot python3 /home/pi/myscript.py
is not starting my script when rebooting.

also i tried

Code: Select all

@reboot python3 /home/pi/myscript.py >/tmp/myscript.log 2>&1
but that's not making any difference at all...

User avatar
thagrol
Posts: 2097
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Python script automatic start

Tue Dec 10, 2019 12:26 am

sanpaulus wrote:
Mon Dec 09, 2019 9:21 pm
ps aux | grep myscript.py
tried that to check if the script is running. once while the script is running, and to compare once without running my script.
so yeah, the

Code: Select all

 crontab -e @reboot python3 /home/pi/myscript.py
is not starting my script when rebooting.

also i tried

Code: Select all

@reboot python3 /home/pi/myscript.py >/tmp/myscript.log 2>&1
but that's not making any difference at all...

Some questions and sugestions:
  1. Was there anything in the log file?
  2. Did cron send your user any emails? Or write anything to syslog?
  3. Is your script actually called "myscript.py"? If not and you ran the ps & grep command I suggested it wouldn't have been found. Try this instead:

    Code: Select all

    ps aux | more
    Read through the output looking for your script. Or replace "myscript.py" with the actual script name in my original suggestion.
  4. Was that crontab for the same user that owns the file (presumably pi)? If not, check that permissions on the file, the directory containing it and any files it needs access to allow at least reading and if necessary writing by all users.
  5. Try adding the full path to python3 to your crontab entry. If you don't know it, this will tell you:

    Code: Select all

    which python3
  6. Check whether cron is working, add (replace "pi" if required):

    Code: Select all

    @reboot echo "crontest" > /home/pi/crontest.out
  7. What's the output of

    Code: Select all

    systemctl status cron
Without capturing the output and errors from cron and/or your script there isn't a lot more I can suggest.
Attempts to contact me outside of thes forums will be ignored unless signed in triplicate, sent in, sent back, queried, lost, found, subjected to public enquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters

sanpaulus
Posts: 16
Joined: Fri Nov 22, 2019 4:04 pm

Re: Python script automatic start

Tue Dec 10, 2019 4:11 pm

The log file had something in it:
Traceback (most recent call last):
File "/home/pi/Uhrzeit.py", line 3, in <module>
window = tkinter.Tk()
File "/usr/lib/python3.5/tkinter/__init__.py", line 1880, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable
well, the "Crontest" did work :)

the output of

Code: Select all

systemctl status cron
is this:
● cron.service - Regular background program processing daemon
Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-12-10 16:58:17 CET; 8min ago
Docs: man:cron(8)
Main PID: 398 (cron)
CPU: 671ms
CGroup: /system.slice/cron.service
└─398 /usr/sbin/cron -f

Dec 10 16:58:17 raspberrypi cron[398]: (CRON) INFO (Running @reboot jobs)
Dec 10 16:58:17 raspberrypi CRON[402]: pam_unix(cron:session): session opened for user pi by (uid=0)
Dec 10 16:58:17 raspberrypi CRON[401]: pam_unix(cron:session): session opened for user pi by (uid=0)
Dec 10 16:58:17 raspberrypi CRON[403]: pam_unix(cron:session): session opened for user root by (uid=0)
Dec 10 16:58:17 raspberrypi CRON[424]: (pi) CMD (/usr/bin/python3 /home/pi/Uhrzeit.py >/tmp/Uhrzeit.log 2>&1)
Dec 10 16:58:17 raspberrypi CRON[423]: (pi) CMD (echo "crontest" > /home/pi/crontest.out)
Dec 10 16:58:17 raspberrypi CRON[425]: (root) CMD (sh /media/pi/WININSTALL/Sensor_Daten/launcher.sh >/home/pi/logs/cronlogs 2>&1)
Dec 10 16:58:17 raspberrypi CRON[402]: pam_unix(cron:session): session closed for user pi
Dec 10 16:58:17 raspberrypi CRON[403]: pam_unix(cron:session): session closed for user root
Dec 10 16:58:19 raspberrypi CRON[401]: pam_unix(cron:session): session closed for user pi
~
~
~
~

User avatar
thagrol
Posts: 2097
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Python script automatic start

Tue Dec 10, 2019 4:40 pm

sanpaulus wrote:
Tue Dec 10, 2019 4:11 pm
The log file had something in it:
Traceback (most recent call last):
File "/home/pi/Uhrzeit.py", line 3, in <module>
window = tkinter.Tk()
File "/usr/lib/python3.5/tkinter/__init__.py", line 1880, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable
I don't think that's from the same code that you first asked about.

The presence of calls to tkinter and the error message indicates that your code requires a running desktop (or at least an X11 server). Given that, cron is not the way to start it:
  • @reboot cron jobs run before the desktop has started
  • Cron jobs run in a reduced envirnoment, as such even if a desktop is running they won't know anything about it.
  • Even if a desktop has started and you tell your code about it, it still won't work. The default security setting for the raspbian desktop is to reject all connections except those made from within the current desktop session..
You've got basically two options:
  1. Go back to autostart. Can't help with that as I don't sue it.
  2. Remove the security on your desktop then change your crontab entry to something like this:

    Code: Select all

    @reboot sleep 30 && export DISPLAY=localhost:0 && python3 /home/pi/myscript.py
    The downside of this approach is that it will always start on the same display (in this example the console/directly attached display), and that any user on your Pi can start stuff on your display.
Attempts to contact me outside of thes forums will be ignored unless signed in triplicate, sent in, sent back, queried, lost, found, subjected to public enquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters

sanpaulus
Posts: 16
Joined: Fri Nov 22, 2019 4:04 pm

Re: Python script automatic start

Tue Dec 10, 2019 9:34 pm

here's what cronlog said after i tried the code from the begining:
2019-12-10 22:23:37.243580,0,400
2019-12-10 22:23:38.150592,0,400
2019-12-10 22:23:39.228499,0,400
Traceback (most recent call last):
File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/usr/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/usr/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/usr/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.5/socket.py", line 694, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.5/socket.py", line 733, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/media/pi/WININSTALL/Sensor_Daten/SGP30_bulk_upload.py", line 21, in <module>
f = urllib.request.urlopen(tvocURL +str(temp[0]))
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 1282, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
the first 3 rows just worked!!! Even "Thingspeak" reacted and plotted the data... now i have to find out why it's crashing after 3 rows... and it feels uncomfortable not to see anything whats going on with the process... Isn't there a way to see the whole thing in the terminal?

sanpaulus
Posts: 16
Joined: Fri Nov 22, 2019 4:04 pm

Re: Python script automatic start

Tue Dec 10, 2019 9:51 pm

Code: Select all

@reboot sleep 30 && export DISPLAY=localhost:0 && python3 /home/pi/myscript.py
Changed the sleep to 180 to make sure wifi conection is build up before the script starts.
It works now! problem SOLVED

Thank you guys so much

GlowInTheDark
Posts: 189
Joined: Sat Nov 09, 2019 12:14 pm

Re: Python script automatic start

Tue Dec 10, 2019 10:30 pm

sanpaulus wrote:
Tue Dec 10, 2019 9:51 pm

Code: Select all

@reboot sleep 30 && export DISPLAY=localhost:0 && python3 /home/pi/myscript.py
Changed the sleep to 180 to make sure wifi conection is build up before the script starts.
It works now! problem SOLVED

Thank you guys so much
I wonder why you have "localhost" in there.

The way I handle this problem is to go to a terminal window on the desktop and do:

Code: Select all

$ echo $DISPLAY
:0.0
$
So, now I know what the right value for DISPLAY is. I set that in my "foreign" job (e.g., a cron entry) and it works as expected. Notice that this form of DISPLAY does not have any hostname in it.

Using localhost:0 means that it goes through TCP/IP to get to the X desktop. This form of access is disabled by default (for many years if not decades now). You need to go through some hoops to re-enable TCP/IP access to the X desktop. I'm curious how it is that you got it to work that way. Seriously, I'm curious; I'm not just snarking.

Reference:

If you do a long form ps on the Xorg process, it will look something like:

root 727 21.0 3.1 221692 29608 tty7 Ssl+ Nov23 5262:04 /usr/lib/xorg/Xorg :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch

Notice the "-nolisten tcp" option in there.

User avatar
thagrol
Posts: 2097
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Python script automatic start

Tue Dec 10, 2019 11:14 pm

GlowInTheDark wrote:
Tue Dec 10, 2019 10:30 pm
sanpaulus wrote:
Tue Dec 10, 2019 9:51 pm

Code: Select all

@reboot sleep 30 && export DISPLAY=localhost:0 && python3 /home/pi/myscript.py
Changed the sleep to 180 to make sure wifi conection is build up before the script starts.
It works now! problem SOLVED

Thank you guys so much
I wonder why you have "localhost" in there.
Probably because that's what I gave the OP in my suggestions. I suspect that the script that's now working doesn't care what $DISPLAY is set to as it doesn't use it (see the original post) and it can be removed from that crontab entry.

You are correct that X secutiry settings would prevet that but if you refer to my previous postings you'll see that I'd already mentioned this.

For the OP's script that needs tkinter, the optimal solution (as I pointed out here: https://www.raspberrypi.org/forums/view ... 9#p1578798) is not cron, it's autostart.
Attempts to contact me outside of thes forums will be ignored unless signed in triplicate, sent in, sent back, queried, lost, found, subjected to public enquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters

Return to “Python”