Problem with unattended upgrades

Operation
1

Hey Stevie, all,

something went wrong with unattended upgrades this morning both on my and my brother’s syncosync box. I paste the full log below. As a consequence, sftp access to my own box is not possible, restic backups fail. Also, my box shows problems connecting to the remote box, likely also because ssh/sftp login is not possible.

Please advise on how to proceed! Thanks!

Cheers,
Tim

Unattended upgrade result: All upgrades installed

Packages that attempted to upgrade:
 base-files bash cifs-utils dirmngr distro-info-data dpkg dpkg-dev
 gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client
 gpg-wks-server gpgconf gpgsm gpgv gzip libdpkg-perl libfreetype-dev
 libfreetype6 libfreetype6-dev libgnutls30 libldap-2.4-2
 libldap-common liblzma5 libnginx-mod-http-geoip
 libnginx-mod-http-image-filter libnginx-mod-http-xslt-filter
 libnginx-mod-mail libnginx-mod-stream libnginx-mod-stream-geoip
 libntfs-3g883 libssl1.1 libtiff5 libwbclient0 libxml2 logrotate nano
 nginx nginx-common nginx-core ntfs-3g openssh-client openssh-server
 openssh-sftp-server openssl rsyslog ssh wireless-regdb xz-utils
 zlib1g zlib1g-dev

Packages with upgradable origin but kept back:
 Debian stable:
  openssh-sftp-server ssh libwbclient0 ntfs-3g wireless-regdb
  nginx-common libnginx-mod-stream liblzma5 libgnutls30
  libnginx-mod-mail logrotate cifs-utils libssl1.1 base-files
  libnginx-mod-http-image-filter libfreetype6 libxml2 libntfs-3g883
  bash libfreetype6-dev distro-info-data zlib1g-dev nginx dpkg
  dpkg-dev xz-utils rsyslog openssh-client libldap-2.4-2 libdpkg-perl
  libnginx-mod-http-xslt-filter openssl nano libfreetype-dev
  openssh-server libnginx-mod-stream-geoip zlib1g
  libnginx-mod-http-geoip gzip nginx-core

Package installation log:
Log started: 2022-07-10  06:50:03
apt-listchanges: Reading changelogs...
apt-listchanges: Reading changelogs...
Preparing to unpack .../00-gpg-wks-client_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gpg-wks-client (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../01-dirmngr_2.2.27-2+deb11u2_arm64.deb ...
Unpacking dirmngr (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../02-gpg-wks-server_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gpg-wks-server (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../03-gnupg-utils_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gnupg-utils (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../04-gpg-agent_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gpg-agent (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../05-gpg_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gpg (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../06-gpgconf_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gpgconf (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../07-gnupg-l10n_2.2.27-2+deb11u2_all.deb ...
Unpacking gnupg-l10n (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../08-gnupg_2.2.27-2+deb11u2_all.deb ...
Unpacking gnupg (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../09-gpgsm_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gpgsm (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Preparing to unpack .../10-gpgv_2.2.27-2+deb11u2_arm64.deb ...
Unpacking gpgv (2.2.27-2+deb11u2) over (2.2.27-2+deb11u1) ...
Setting up gpgv (2.2.27-2+deb11u2) ...
Setting up gnupg-l10n (2.2.27-2+deb11u2) ...
Setting up gpgconf (2.2.27-2+deb11u2) ...
Setting up gpg (2.2.27-2+deb11u2) ...
Setting up gnupg-utils (2.2.27-2+deb11u2) ...
Setting up gpg-agent (2.2.27-2+deb11u2) ...
Setting up gpgsm (2.2.27-2+deb11u2) ...
Setting up dirmngr (2.2.27-2+deb11u2) ...
Setting up gpg-wks-server (2.2.27-2+deb11u2) ...
Setting up gpg-wks-client (2.2.27-2+deb11u2) ...
Setting up gnupg (2.2.27-2+deb11u2) ...
Processing triggers for man-db (2.9.4-2) ...
Log ended: 2022-07-10  06:50:15

Log started: 2022-07-10  06:50:16
apt-listchanges: Reading changelogs...
apt-listchanges: Reading changelogs...
Preparing to unpack .../libtiff5_4.2.0-1+deb11u1_arm64.deb ...
Unpacking libtiff5:arm64 (4.2.0-1+deb11u1) over (4.2.0-1) ...
Setting up libtiff5:arm64 (4.2.0-1+deb11u1) ...
Processing triggers for libc-bin (2.31-13+rpt2+rpi1+deb11u2) ...
Log ended: 2022-07-10  06:50:19

Log started: 2022-07-10  06:50:21
apt-listchanges: Reading changelogs...
apt-listchanges: Reading changelogs...
Preparing to unpack .../libldap-common_2.4.57+dfsg-3+deb11u1_all.deb ...
Unpacking libldap-common (2.4.57+dfsg-3+deb11u1) over (2.4.57+dfsg-3) ...
Setting up libldap-common (2.4.57+dfsg-3+deb11u1) ...
Processing triggers for man-db (2.9.4-2) ...
Log ended: 2022-07-10  06:50:24

Log started: 2022-07-10  06:50:26
apt-listchanges: Reading changelogs...
Preconfiguring packages ...
apt-listchanges: Reading changelogs...
Preconfiguring packages ...
Preparing to unpack .../openssh-sftp-server_1%3a8.4p1-5+deb11u1_arm64.deb ...
Unpacking openssh-sftp-server (1:8.4p1-5+deb11u1) over (1:8.4p1-5) ...
Preparing to unpack .../openssh-server_1%3a8.4p1-5+deb11u1_arm64.deb ...
Unpacking openssh-server (1:8.4p1-5+deb11u1) over (1:8.4p1-5) ...
Preparing to unpack .../openssh-client_1%3a8.4p1-5+deb11u1_arm64.deb ...
Unpacking openssh-client (1:8.4p1-5+deb11u1) over (1:8.4p1-5) ...
Setting up openssh-client (1:8.4p1-5+deb11u1) ...
Setting up openssh-sftp-server (1:8.4p1-5+deb11u1) ...
Setting up openssh-server (1:8.4p1-5+deb11u1) ...
rescue-ssh.target is a disabled or a static unit, not starting it.
Job for ssh.service failed because the control process exited with error code.
See "systemctl status ssh.service" and "journalctl -xe" for details.
invoke-rc.d: initscript ssh, action "restart" failed.
● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sun 2022-07-10 06:50:32 BST; 14ms ago
       Docs: man:sshd(8)
             man:sshd_config(5)
    Process: 660493 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS)
    Process: 660494 ExecStart=/usr/sbin/sshd -D $SSHD_OPTS (code=exited, status=255/EXCEPTION)
   Main PID: 660494 (code=exited, status=255/EXCEPTION)
        CPU: 52ms

Jul 10 06:50:32 tims-syncosync systemd[1]: Starting OpenBSD Secure Shell server...
Jul 10 06:50:32 tims-syncosync sshd[660494]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use.
Jul 10 06:50:32 tims-syncosync sshd[660494]: error: Bind to port 22 on :: failed: Address already in use.
Jul 10 06:50:32 tims-syncosync sshd[660494]: fatal: Cannot bind any address.
Jul 10 06:50:32 tims-syncosync systemd[1]: ssh.service: Main process exited, code=exited, status=255/EXCEPTION
Jul 10 06:50:32 tims-syncosync systemd[1]: ssh.service: Failed with result 'exit-code'.
Jul 10 06:50:32 tims-syncosync systemd[1]: Failed to start OpenBSD Secure Shell server.
dpkg: error processing package openssh-server (--configure):
 installed openssh-server package post-installation script subprocess returned error exit status 1
Processing triggers for man-db (2.9.4-2) ...
Errors were encountered while processing:
 openssh-server
E:Sub-process /usr/bin/dpkg returned an error code (1)
Log ended: 2022-07-10  06:50:36



Unattended-upgrades log:
Starting unattended upgrades script
Allowed origins are: origin=Debian,codename=bullseye-updates, origin=Debian,codename=bullseye-proposed-updates, origin=Debian,codename=bullseye,label=Debian, origin=Debian,codename=bullseye,label=Debian-Security, origin=syncosync,codename=bullseye
Initial blacklist: 
Initial whitelist (not strict): 
Packages that will be upgraded: base-files bash cifs-utils dirmngr distro-info-data dpkg dpkg-dev gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm gpgv gzip libdpkg-perl libfreetype-dev libfreetype6 libfreetype6-dev libgnutls30 libldap-2.4-2 libldap-common liblzma5 libnginx-mod-http-geoip libnginx-mod-http-image-filter libnginx-mod-http-xslt-filter libnginx-mod-mail libnginx-mod-stream libnginx-mod-stream-geoip libntfs-3g883 libssl1.1 libtiff5 libwbclient0 libxml2 logrotate nano nginx nginx-common nginx-core ntfs-3g openssh-client openssh-server openssh-sftp-server openssl rsyslog ssh wireless-regdb xz-utils zlib1g zlib1g-dev
Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
Installing the upgrades failed!
error message: installArchives() failed
dpkg returned a error! See /var/log/unattended-upgrades/unattended-upgrades-dpkg.log for details
Package base-files is kept back because a related package is kept back or due to local apt_preferences(5).
Package bash is kept back because a related package is kept back or due to local apt_preferences(5).
Package cifs-utils is kept back because a related package is kept back or due to local apt_preferences(5).
Package distro-info-data is kept back because a related package is kept back or due to local apt_preferences(5).
Package dpkg is kept back because a related package is kept back or due to local apt_preferences(5).
Package dpkg-dev is kept back because a related package is kept back or due to local apt_preferences(5).
Package gzip is kept back because a related package is kept back or due to local apt_preferences(5).
Package libdpkg-perl is kept back because a related package is kept back or due to local apt_preferences(5).
Package libfreetype-dev is kept back because a related package is kept back or due to local apt_preferences(5).
Package libfreetype6 is kept back because a related package is kept back or due to local apt_preferences(5).
Package libfreetype6-dev is kept back because a related package is kept back or due to local apt_preferences(5).
Package libgnutls30 is kept back because a related package is kept back or due to local apt_preferences(5).
Package libldap-2.4-2 is kept back because a related package is kept back or due to local apt_preferences(5).
Package liblzma5 is kept back because a related package is kept back or due to local apt_preferences(5).
Package libnginx-mod-http-geoip is kept back because a related package is kept back or due to local apt_preferences(5).
Package libnginx-mod-http-image-filter is kept back because a related package is kept back or due to local apt_preferences(5).
Package libnginx-mod-http-xslt-filter is kept back because a related package is kept back or due to local apt_preferences(5).
Package libnginx-mod-mail is kept back because a related package is kept back or due to local apt_preferences(5).
Package libnginx-mod-stream is kept back because a related package is kept back or due to local apt_preferences(5).
Package libnginx-mod-stream-geoip is kept back because a related package is kept back or due to local apt_preferences(5).
Package libntfs-3g883 is kept back because a related package is kept back or due to local apt_preferences(5).
Package libssl1.1 is kept back because a related package is kept back or due to local apt_preferences(5).
Package libwbclient0 is kept back because a related package is kept back or due to local apt_preferences(5).
Package libxml2 is kept back because a related package is kept back or due to local apt_preferences(5).
Package logrotate is kept back because a related package is kept back or due to local apt_preferences(5).
Package nano is kept back because a related package is kept back or due to local apt_preferences(5).
Package nginx is kept back because a related package is kept back or due to local apt_preferences(5).
Package nginx-common is kept back because a related package is kept back or due to local apt_preferences(5).
Package nginx-core is kept back because a related package is kept back or due to local apt_preferences(5).
Package ntfs-3g is kept back because a related package is kept back or due to local apt_preferences(5).
Package openssh-client is kept back because a related package is kept back or due to local apt_preferences(5).
Package openssh-server is kept back because a related package is kept back or due to local apt_preferences(5).
Package openssh-sftp-server is kept back because a related package is kept back or due to local apt_preferences(5).
Package openssl is kept back because a related package is kept back or due to local apt_preferences(5).
Package rsyslog is kept back because a related package is kept back or due to local apt_preferences(5).
Package ssh is kept back because a related package is kept back or due to local apt_preferences(5).
Package wireless-regdb is kept back because a related package is kept back or due to local apt_preferences(5).
Package xz-utils is kept back because a related package is kept back or due to local apt_preferences(5).
Package zlib1g is kept back because a related package is kept back or due to local apt_preferences(5).
Package zlib1g-dev is kept back because a related package is kept back or due to local apt_preferences(5).
2

strange, why don’t we/other get these issues? I guess just going the unattended upgrades in the usual way is not the way to go. We will change this to be started from housekeeping.

For the time now - hope you have somehow access - stop the syncosync service (service syncosync stop), try to fix the installation (apt --fix-broken install), start the syncosync service.

Cheers
Steve

3

Hey Stevie,

thanks for the - as always - very quick response, I really appreciate it!

After the failed unattended upgrades, the ssh server was not running on my box, so I could not ssh in. The problem reported in the log was that sshd could not bind to port 22 because it was already in use … no idea why that happened for both of my boxes but none of yours. The only difference I am aware of is that I did not apply the changes needed to read out whether the disk is spinning or not. Hardly sounds related.

Anyway, after a reboot I could ssh to the box as sosadmin and then do an “apt --fix-broken install” as well as an “apt upgrade”. I think my box is fine again now, but I have yet to apply the same fixes to my brother’s box in a TeamSpeak session.

Please let me know in case I can pull any other logs or the like for you to find out in detail why things went downhill!

Cheers,
Tim

PS: When doing an “sudo apt update” I get

N: Skipping acquire of configured file 'main/binary-arm64/Packages' as repository 'http://deb.syncosync.org bullseye InRelease' doesn't support architecture 'arm64'

I guess I can ignore that?

4

The reason, that sshd service could not be updated as it is not able to start as syncosync is using port 22 (or to be more precise, syncosync is starting the sshd from within syncosync due to security reasons). syncsoync disables the sshd service (we should check this every start, will check if we do so).

No thanks, it is completely understood, what is happening…

You can, still, we should fix it…

Cheers
Steve