CecilWard
Posts: 39
Joined: Sat Sep 03, 2016 10:17 pm

Long-running process and logging out of ssh

Fri Sep 08, 2017 8:24 am

I keep getting timed out by my pi while logged in using an ssh client on an iPad (the app is Prompt 2, vg btw). I think it is because the client iPad is going to sleep possibly. Any tips to prevent the ssh problem, to keep it hanging on in there ?

I am reading about 'disown' and 'bt', 'fg', ^Z and 'nohup', but I would have to remember to use these good tools all the time in order to be safe. So anything that I can employ to protect me from the effects of dropping the ssh connection would be helpful.

If the worst should happen, and I need to log back in again with ssh, am I stuffed? Can I get control over an independent, still-running process that was launched from a command-line?

User avatar
chrisoh
Posts: 217
Joined: Sun Dec 06, 2015 8:50 pm
Location: Essex, UK

Re: Long-running process and logging out of ssh

Fri Sep 08, 2017 8:34 am

Check out tmux or screen, might help in your scenario. They allow you to create a 'session' that runs seperate from your SSH session. I.e. Create a session, run commands in that session and then leave the session. Then later rejoin the session and carry on, very handy indeed. i.e. sudo apt-get install tmux
Syntax is a little daunting at first, but there's lots of cheat sheets out there to help you e.g. http://hyperpolyglot.org/multiplexers.
Start a session called 'mysession':

Code: Select all

tmux new -s mysession
Leave the session (you go back to your inital SSH prompt):

Code: Select all

Ctrl+b then d
Rejoin the session

Code: Select all

tmux attach -t mysession
Raspberry Pi 3 model B Idle @ 250mA
  • HDMI +200mA
  • Minecraft 1.9 Server +100mA
  • 2.4Ghz dongle +20mA

jahboater
Posts: 4762
Joined: Wed Feb 04, 2015 6:38 pm

Re: Long-running process and logging out of ssh

Fri Sep 08, 2017 8:58 am

Also see "setsid"

pcmanbob
Posts: 7067
Joined: Fri May 31, 2013 9:28 pm
Location: Mansfield UK

Re: Long-running process and logging out of ssh

Fri Sep 08, 2017 10:10 am

Hi.

Have a look at the settings/preferences in your ssh client on an iPad (the app is Prompt 2) and see if there is a keep awake or keep connected option, or is there is any option to change the time out on the connection or app.

basically what you need to do is keep you connection active and so prevent the time out of the app/connection, one of the above or a combination if available will do what you need.
We want information… information… information........................no information no help
The use of crystal balls & mind reading are not supported

CecilWard
Posts: 39
Joined: Sat Sep 03, 2016 10:17 pm

Re: Long-running process and logging out of ssh

Fri Sep 08, 2017 5:23 pm

Many many thanks for the tip about tmux guys.

Am I vaguely on the right lines or not concerning disown? If I remember to do it, will it provide the protection I need?

User avatar
jojopi
Posts: 3085
Joined: Tue Oct 11, 2011 8:38 pm

Re: Long-running process and logging out of ssh

Fri Sep 08, 2017 8:43 pm

What do the programs do, that you want to keep running after you are logged out? If they use the terminal interactively (even just for printing progress messages) then the screen or tmux solution is the only way you will be able to reconnect to them later. (See also reptyr.)

If the programs do not require terminal input or output, and can usefully be left running after logout, then you should consider making the programs themselves ignore hangup signals, and possibly background or daemonize themselves when starting.

CecilWard
Posts: 39
Joined: Sat Sep 03, 2016 10:17 pm

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 10:35 am

A friend just told me that all I needed to do was to stick an & on the end of the command line, which I had been doing anyway. But is that enough? When the original process that launched them, i.e. my terminal command line process, quits, lets say I type 'exit', are any processes launched in background by using '&' definitely still going to be running on ok?

I'm not convinced he's correct. What would the point of 'disown' be? I have probably just managed to confuse myself here.

Can I peep at them later on, using jobs and fg / bg or something?

If I log out by typing exit, when I later login in again, have I now lost any chance of reconnecting a command line to those hopefully still running processes? And seeing more of their continuing output to stdout etc? (or even seeing their earlier output to stdout generated in the meantime, if that is even feasible)

Not the end of the world if that isn't possible. Just a shame. And I should have just redirected their output to a log file anyway so I can find out the story later.

I suppose it would be good to find out what the returned error code was when the background process finally ended. If there is a standard universal way of determining that then that would be good to know. Otherwise I'll just have to write something to make sure the completion status gets appended to some log file.

Although I have never used *nix before, I have worked in r/d for an operating systems development company on new products for a number of years and am familiar with Windows NT family o/s and VAX/VMS. So I know the basic concepts, just don't know how the head-bone connects to the leg-bone, or whatever the cliché / analogy I'm looking for is.

Martin Frezman
Posts: 1020
Joined: Mon Oct 31, 2016 10:05 am

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 11:38 am

The "&" method is old-fashioned. 70s, 80s stuff.

I don't think it ever really worked right (though, of course, I'm sure it conforms to whatever standards docs there are). Part of the problem is that it is a shell construct, so it's behavior depends on which shell you are using and which version of that shell. In particular, in the old days, & did not inhibit SIGHUP, so you had to use "nohup" as well. Then csh came along and fixed this - creating an incompatibility. I have no idea what modern sh-type shells do, but I'll bet each one is subtly different from the others.

Anyway, as others have noted, screen is the best. tmux is also good.
If this post appears in the wrong forums category, my apologies.

User avatar
rpdom
Posts: 15366
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 12:08 pm

CecilWard wrote:
Sat Sep 09, 2017 10:35 am
A friend just told me that all I needed to do was to stick an & on the end of the command line, which I had been doing anyway. But is that enough? When the original process that launched them, i.e. my terminal command line process, quits, lets say I type 'exit', are any processes launched in background by using '&' definitely still going to be running on ok?
No, they will be killed if you log out.

If you want them to run in the background *and* stay running when you log out, you need to use something like nohup as well.

Code: Select all

nohup Your Command Line here &
nohup means "NO Hang UP" which means it ignores the "Hang-Up" (disconnect) signal and keeps on running.

Alternatively if you want to be able to log back in and see the output of the command while it has been running, you could use "screen" or "tmux", which start a terminal session that stays running and you can reconnect to it later. screen was mostly designed for the old days of unreliable modem connections. You didn't really want to start a 48 hour compilation run on a dial up connection that could drop at any time. So screen kept running when your connection dropped and you could reconnect to the same session and see how it was going.

CecilWard
Posts: 39
Joined: Sat Sep 03, 2016 10:17 pm

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 10:36 pm

Referring to chrisoh's earlier post, would that allow me to keep a process running and even survive closing the ssh client altogether? and then later I can log back in again with ssh and associate the terminal session UI with the still-running old process?

User avatar
chrisoh
Posts: 217
Joined: Sun Dec 06, 2015 8:50 pm
Location: Essex, UK

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 11:19 pm

CecilWard wrote:
Sat Sep 09, 2017 10:36 pm
Referring to chrisoh's earlier post, would that allow me to keep a process running and even survive closing the ssh client altogether? and then later I can log back in again with ssh and associate the terminal session UI with the still-running old process?
Yep, you got it. Detach your tmux/screen session and exit your SSH session and you can log in the next day and rejoin the session and continue as you were.

As a slightly more expanded example: I've one Pi that, at startup, creates 5 new tmux sessions automatically. In each tmux session an long-running program is started (a security monitor, a neural net, a cryptocurrency daemon, a FIFO monitor and a minecraft server). I rarely SSH into this Pi unless I receive an alert or it needs some maintenance. However, using tmux, I can SSH in and join any of the sessions and interact with the program as necessary, then leave it again until next time. Nifty, like jaffa cakes.
Raspberry Pi 3 model B Idle @ 250mA
  • HDMI +200mA
  • Minecraft 1.9 Server +100mA
  • 2.4Ghz dongle +20mA

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 11:28 pm

Oldschool

Code: Select all

nohup /path/to/myprog >/path/to/myprog.out 2>&1 &
tail -f /path/to/myprog.out 
works for me. I can see if it's doing what it's supposed to, and then log out and come back later to see what it did.

User avatar
chrisoh
Posts: 217
Joined: Sun Dec 06, 2015 8:50 pm
Location: Essex, UK

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 11:36 pm

asandford wrote:
Sat Sep 09, 2017 11:28 pm
Oldschool
...like Viscounts? :P
Raspberry Pi 3 model B Idle @ 250mA
  • HDMI +200mA
  • Minecraft 1.9 Server +100mA
  • 2.4Ghz dongle +20mA

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Long-running process and logging out of ssh

Sat Sep 09, 2017 11:41 pm

chrisoh wrote:
Sat Sep 09, 2017 11:36 pm
asandford wrote:
Sat Sep 09, 2017 11:28 pm
Oldschool
...like Viscounts? :P
I prefer the mint ones

CecilWard
Posts: 39
Joined: Sat Sep 03, 2016 10:17 pm

Re: Long-running process and logging out of ssh

Mon Sep 11, 2017 10:50 pm

The screen /tmux and nohup tips are brilliant. I have set up some aliases for convenience based around screen. It has already saved my bacon umpteen times.

I really need to write a shell script now that detects if you have not done already a screen -dmS yet to save a detached session and if not it does one automatically when you log in. I could either try reconnecting to a possible session and check for failure, or I could use the screen -ls command to check for existing saved sessions and parse the output, the later is likely to be a bit messy perhaps, and the former could be a lot easier. Any tips?

Aside: [off topic] I need to learn to some Linux scripting languages. It's going to be brilliant. I have been doing first VAX VMS DCL batch, then MS-DOS then masses of WinNT batch file scripting for 35 years, ulp, with a brief play with the lovely powershell some years back, but got rather overwhelmed by the latter and drowned. I suppose I should include PHP really, the new Basic, I feel somehow. A thing to relax and hack away in. I have written some huge monsters in WinNT CMD batch + Jscript, some things including XSLT programs as well, tri-lingual monstrosities, and I am looking forward to the riches available to *nix users.

User avatar
rpdom
Posts: 15366
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Long-running process and logging out of ssh

Tue Sep 12, 2017 6:38 am

CecilWard wrote:
Mon Sep 11, 2017 10:50 pm
I need to learn to some Linux scripting languages.
I can recommend Bash (for command line stuff, like .bat files in DOS/Windows, but much more flexible), Python3 (it's the "in thing" nowadays - Python devs are in demand), and PHP/Javascript for web stuff. PHP can be used for other things too.

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Long-running process and logging out of ssh

Tue Sep 12, 2017 10:41 pm

rpdom wrote:
Tue Sep 12, 2017 6:38 am

I can recommend Bash (for command line stuff, like .bat files in DOS/Windows, but much more flexible), Python3 (it's the "in thing" nowadays - Python devs are in demand), and PHP/Javascript for web stuff. PHP can be used for other things too.
Perl was all the rage back in the y2ks, strange language.

User avatar
rpdom
Posts: 15366
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Long-running process and logging out of ssh

Wed Sep 13, 2017 6:25 am

asandford wrote:
Tue Sep 12, 2017 10:41 pm
Perl was all the rage back in the y2ks, strange language.
I haven't done any Perl for months now. I agree it is strange, especially the "default variable" stuff so there is implied stuff you don't put in the code.

Return to “Beginners”