MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

SSH Issues - Looses Function after a time

Tue Aug 06, 2019 3:04 pm

I have been upgrading to Buster, my other (3) units were fine but one has an issue with ssh... I get connection refused, i log in using telnet (good thing I installed that...). I purge and re-install and it works (for a while). Then it goes back to connection refused... Any suggestions?

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Tue Aug 06, 2019 5:26 pm

Next time you get a connection refused, can you get a verbose message and post that here?

PuTTY has a log that you turn on or use ssh —v to get a log of connection output.
Need Pi spray - these things are breeding in my house...

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Tue Aug 06, 2019 7:29 pm

This what I got from ssh -v...

OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to raspberrypi.local port 22.
ssh: connect to host raspberrypi.local port 22: Connection refused

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 1:29 am

Well sure does not like you :?

Can you check the log on the Pi with

Code: Select all

tail -500 /var/log/auth.log | grep 'sshd'
It’s almost like the firewall is closing the port or something changing the config...
Need Pi spray - these things are breeding in my house...

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 12:08 pm

Aug 5 21:17:32 raspberrypi sshd[6114]: pam_unix(sshd:session): session opened f
or user pi by (uid=0)
Aug 5 21:18:29 raspberrypi sshd[6291]: Accepted password for pi from 2601:409:8
500:3945:79c0:6718:c91c:8520 port 63564 ssh2
Aug 5 21:18:29 raspberrypi sshd[6291]: pam_unix(sshd:session): session opened f
or user pi by (uid=0)
Aug 5 21:19:11 raspberrypi sshd[6436]: Did not receive identification string fr
om 2601:409:8500:3945:9576:712e:9acf:363c port 63644
Aug 5 21:19:16 raspberrypi sshd[6435]: Accepted password for pi from 2601:409:8
500:3945:79c0:6718:c91c:8520 port 63643 ssh2
Aug 5 21:19:16 raspberrypi sshd[6435]: pam_unix(sshd:session): session opened f
or user pi by (uid=0)
Aug 5 21:19:16 raspberrypi sshd[6476]: Received disconnect from 2601:409:8500:3
945:79c0:6718:c91c:8520 port 63643:11: disconnected by user
Aug 5 21:19:16 raspberrypi sshd[6476]: Disconnected from user pi 2601:409:8500:
3945:79c0:6718:c91c:8520 port 63643
Aug 5 21:19:16 raspberrypi sshd[6435]: pam_unix(sshd:session): session closed f
or user pi
Aug 5 21:19:25 raspberrypi sshd[6513]: Accepted publickey for pi from 2601:409:
8500:3945:9576:712e:9acf:363c port 63671 ssh2: RSA SHA256:/J1IQVaOhCHgn6z/XHl3Wd
PFieMlM4JeDuuXDQNrqHU
Aug 5 21:19:25 raspberrypi sshd[6513]: pam_unix(sshd:session): session opened f
or user pi by (uid=0)

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 12:48 pm

Nothing odd there for IPv6

Next time it’s dead can you try

Code: Select all

sudo netstat -ltnp
You may have to install net-tools

This should give something like:

Code: Select all

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      468/sshd            
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      409/mosquitto       
tcp        0      0 192.168.1.10:445        0.0.0.0:*               LISTEN      613/smbd            
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN      459/vncserver-x11-c 

And we can tell if sshd is still holding the port...
Need Pi spray - these things are breeding in my house...

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 12:51 pm

Sorry above example only shows IPv4 obviously it’s an IPv6 allergic box :lol: :oops: :roll:
Need Pi spray - these things are breeding in my house...

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 2:05 pm

pi@raspberrypi:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 0.0.0.0:5900 0.0.0.0:* LISTEN 319/vncserver-x11-c
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/init
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 2799/cupsd
tcp 0 0 0.0.0.0:23 0.0.0.0:* LISTEN 298/inetd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 685/exim4
tcp6 0 0 :::548 :::* LISTEN 336/afpd
tcp6 0 0 :::5900 :::* LISTEN 319/vncserver-x11-c
tcp6 0 0 :::111 :::* LISTEN 1/init
tcp6 0 0 ::1:631 :::* LISTEN 2799/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 685/exim4
tcp6 0 0 ::1:4700 :::* LISTEN 337/cnid_metad

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 2:20 pm

In one point of view it’s good nothing else is using the port :lol:

You could try

Code: Select all

/usr/sbin/sshd -d
to see if you get anymore debugging data or uncomment (remove the #) and up the log level in /etc/ssh/sshd_config

Just in case, this is my default file

Code: Select all

#       $OpenBSD: sshd_config,v 1.100 2016/08/15 12:32:04 naddy Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#PubkeyAuthentication yes

# Expect .ssh/authorized_keys2 to be disregarded by default in future.
#AuthorizedKeysFile     .ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#UsePrivilegeSeparation sandbox
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

# override default of no subsystems
Subsystem       sftp    /usr/lib/openssh/sftp-server

# Example of overriding settings on a per-user basis
#Match User anoncvs
#       X11Forwarding no
#       AllowTcpForwarding no
#       PermitTTY no
#       ForceCommand cvs server
Need Pi spray - these things are breeding in my house...

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 2:42 pm

/etc/ssh/sshd_config line 84: Unsupported option UsePAM
debug1: sshd version OpenSSH_6.7p2p1
debug1: could not open key file '/etc/ssh/ssh_host_rsa_key': Permission denied
Could not load host key: /etc/ssh/ssh_host_rsa_key
debug1: could not open key file '/etc/ssh/ssh_host_dsa_key': No such file or dir
ectory
Could not load host key: /etc/ssh/ssh_host_dsa_key
debug1: could not open key file '/etc/ssh/ssh_host_ecdsa_key': Permission denied
Could not load host key: /etc/ssh/ssh_host_ecdsa_key
Disabling protocol version 2. Could not load host key
sshd: no hostkeys available -- exiting.
You have new mail in /var/mail/pi

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 2:50 pm

You could try regenerating the keys with

Code: Select all

/etc/init.d/sshd start
Or running 
ssh-keygen -A
BUT you are heading into areas I’ve not trodden before and would hate to break your system.
Need Pi spray - these things are breeding in my house...

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 3:04 pm

pi@raspberrypi:~$ /etc/init.d/sshd start
-bash: /etc/init.d/sshd: No such file or directory
pi@raspberrypi:~$ ssh-keygen -A
ssh-keygen: generating new host keys: DSA Could not save your public key in /etc
/ssh/ssh_host_dsa_key.eAeMQ24bQA: Permission denied
pi@raspberrypi:~$

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 3:07 pm

Sorry init.d gone :oops:

Try sudo to regen the keys.
Need Pi spray - these things are breeding in my house...

SkyRise
Posts: 179
Joined: Tue Jan 24, 2012 1:20 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 3:17 pm

Sorry, been lurking here quietly but the more I see, the more I sense a corrupted filesystem...

Power supply issues perhaps?

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 4:51 pm

Regen-ed keys and rebooted....no luck

If it is corrupted FS any suggestions on how to trigger a re-load to fix? I assume it will be a lot of apt purge and apt install.....

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 11:48 pm

pi@raspberrypi:/var/log$ grep ssh syslog
Aug 7 12:49:06 raspberrypi systemd[1]: Condition check resulted in Turn on SSH
if /boot/ssh is present being skipped.
Aug 7 12:49:10 raspberrypi sshd[321]: /etc/ssh/sshd_config line 84: Unsupported
option UsePAM
Aug 7 12:49:10 raspberrypi sshd[321]: Could not load host key: /etc/ssh/ssh_hos
t_rsa_key
Aug 7 12:49:10 raspberrypi sshd[321]: Could not load host key: /etc/ssh/ssh_hos
t_dsa_key
Aug 7 12:49:10 raspberrypi sshd[321]: Could not load host key: /etc/ssh/ssh_hos
t_ecdsa_key
Aug 7 12:49:10 raspberrypi sshd[321]: Disabling protocol version 2. Could not l
oad host key
Aug 7 12:49:10 raspberrypi sshd[321]: sshd: no hostkeys available -- exiting.
Aug 7 12:49:11 raspberrypi systemd[1]: ssh.service: Control process exited, cod
e=exited, status=1/FAILURE
Aug 7 12:49:11 raspberrypi systemd[1]: ssh.service: Failed with result 'exit-co
de'.
Aug 7 12:49:12 raspberrypi systemd[1]: ssh.service: Service RestartSec=100ms ex

trejan
Posts: 522
Joined: Tue Jul 02, 2019 2:28 pm

Re: SSH Issues - Looses Function after a time

Wed Aug 07, 2019 11:58 pm

I agree with SkyRise. This looks like a corrupted filesystem. What does "ls -al /etc/ssh/" say?
Last edited by trejan on Thu Aug 08, 2019 1:30 am, edited 1 time in total.

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Thu Aug 08, 2019 1:28 am

One other thought is to remove and replace - Apt should be able to remove and reinstall openssh-server for you.
Need Pi spray - these things are breeding in my house...

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Thu Aug 08, 2019 6:57 pm

Is I re-install openssl-server it works for a few minutes then stops.

from the ls I get

drwxr-xr-x 2 root root 4096 Aug 8 06:30 .
drwxr-xr-x 141 root root 12288 Aug 8 06:28 ..
-rw-r--r-- 1 root root 565189 Apr 8 06:13 moduli
-rw-r--r-- 1 root root 1580 Apr 8 06:13 ssh_config
-rw-r--r-- 1 root root 3251 Aug 7 21:46 sshd_config
-rw------- 1 root root 1381 Aug 7 12:45 ssh_host_dsa_key
-rw-r--r-- 1 root root 606 Aug 7 12:45 ssh_host_dsa_key.pub
-rw------- 1 root root 513 Aug 5 21:15 ssh_host_ecdsa_key
-rw-r--r-- 1 root root 178 Aug 5 21:15 ssh_host_ecdsa_key.pub
-rw------- 1 root root 411 Aug 5 21:15 ssh_host_ed25519_key
-rw-r--r-- 1 root root 98 Aug 5 21:15 ssh_host_ed25519_key.pub
-rw------- 1 root root 1823 Aug 5 21:15 ssh_host_rsa_key
-rw-r--r-- 1 root root 398 Aug 5 21:15 ssh_host_rsa_key.pub

Andyroo
Posts: 4189
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: SSH Issues - Looses Function after a time

Thu Aug 08, 2019 7:07 pm

I’m stuck I’m afraid :?: :oops: :?:

It’s almost like a virus is killing it!
Need Pi spray - these things are breeding in my house...

User avatar
HawaiianPi
Posts: 4535
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: SSH Issues - Looses Function after a time

Thu Aug 08, 2019 7:11 pm

MattPackwood123 wrote:
Tue Aug 06, 2019 3:04 pm
I have been upgrading to Buster...
By "upgrading" do you mean altering your sources and upgrading from an older version (Stretch) to Buster?
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

bls
Posts: 211
Joined: Mon Oct 22, 2018 11:25 pm

Re: SSH Issues - Looses Function after a time

Thu Aug 08, 2019 7:19 pm

Is there anything in the systemlog (sudo journalctl) that says anything about the ssh daemon going away?

MattPackwood123
Posts: 17
Joined: Fri Apr 11, 2014 1:48 pm

Re: SSH Issues - Looses Function after a time

Fri Aug 09, 2019 11:50 am

Guys, thanks for the help, it is time for me to build a new image I think....

Return to “Networking and servers”