units: add system-update-cleanup.service to guard against offline-update loops

Note: the name is "system-update-cleanup.service" rather than
"system-update-done.service", because it should not run normally, and also
because there's already "systemd-update-done.service", and having them named
so similarly would be confusing.

In https://bugzilla.redhat.com/show_bug.cgi?id=1395686 the system repeatedly
entered system-update.target on boot. Because of a packaging issue, the tool
that created the /system-update symlink could be installed without the service
unit that was supposed to perform the upgrade (and remove the symlink). In
fact, if there are no units in system-update.target, and /system-update symlink
is created, systemd always "hangs" in system-update.target. This is confusing
for users, because there's no feedback what is happening, and fixing this
requires starting an emergency shell somehow, and also knowing that the symlink
must be removed. We should be more resilient in this case, and remove the
symlink automatically ourselves, if there are no upgrade service to handle it.

This adds a service which is started after system-update.target is reached and
the symlink still exists. It nukes the symlink and reboots the machine. It
should subsequently boot into the default default.target.

This is a more general fix for
https://bugzilla.redhat.com/show_bug.cgi?id=1395686 (the packaging issue was
already fixed).

6 files changed, 48 insertions, 1 deletions

diff --git a/Makefile.am b/Makefile.am
index 5cec19fbb8..6c350b0ec4 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -529,6 +529,7 @@ nodist_systemunit_DATA = \
 	units/serial-getty@.service \
 	units/console-getty.service \
 	units/container-getty@.service \
+	units/system-update-cleanup.service \
 	units/systemd-initctl.service \
 	units/systemd-remount-fs.service \
 	units/systemd-ask-password-wall.service \
@@ -592,6 +593,7 @@ EXTRA_DIST += \
 	units/console-getty.service.m4.in \
 	units/container-getty@.service.m4.in \
 	units/rescue.service.in \
+	units/system-update-cleanup.service.in \
 	units/systemd-initctl.service.in \
 	units/systemd-remount-fs.service.in \
 	units/systemd-update-utmp.service.in \
diff --git a/man/systemd.offline-updates.xml b/man/systemd.offline-updates.xml
index 22aa0354a8..d673cf5db8 100644
--- a/man/systemd.offline-updates.xml
+++ b/man/systemd.offline-updates.xml
@@ -115,7 +115,9 @@
       <listitem>
         <para>The upgrade scripts should exit only after the update is finished. It is expected
         that the service which performs the upgrade will cause the machine to reboot after it
-        is done.</para>
+        is done. If the <filename>system-update.target</filename> is successfully reached, i.e.
+        all update services have run, and the <filename>/system-update</filename> symlink still
+        exists, it will be removed and the machine rebooted as a safety measure.</para>
       </listitem>
 
       <listitem>
diff --git a/man/systemd.special.xml b/man/systemd.special.xml
index de6a5ac6cd..b513a13b5a 100644
--- a/man/systemd.special.xml
+++ b/man/systemd.special.xml
@@ -102,6 +102,7 @@
     <filename>sysinit.target</filename>,
     <filename>syslog.socket</filename>,
     <filename>system-update.target</filename>,
+    <filename>system-update-cleanup.service</filename>,
     <filename>time-sync.target</filename>,
     <filename>timers.target</filename>,
     <filename>umount.target</filename>,
@@ -608,6 +609,7 @@
       </varlistentry>
       <varlistentry>
         <term><filename>system-update.target</filename></term>
+        <term><filename>system-update-cleanup.service</filename></term>
         <listitem>
           <para>A special target unit that is used for offline system updates.
           <citerefentry><refentrytitle>systemd-system-update-generator</refentrytitle><manvolnum>8</manvolnum></citerefentry>
@@ -615,6 +617,13 @@
           exists. For more information see
           <citerefentry><refentrytitle>systemd.offline-updates</refentrytitle><manvolnum>7</manvolnum></citerefentry>.
           </para>
+
+          <para>Updates should happen before the <filename>system-update.target</filename> is
+          reached, and the services which implement them should cause the machine to reboot. As
+          a safety measure, if this does not happen, and <filename>/system-update</filename>
+          still exists after <filename>system-update.target</filename> is reached,
+          <filename>system-update-cleanup.service</filename> will remove this symlink and
+          reboot the machine.</para>
         </listitem>
       </varlistentry>
       <varlistentry>
diff --git a/units/.gitignore b/units/.gitignore
index 8f4949258e..8fdb6e9ab5 100644
--- a/units/.gitignore
+++ b/units/.gitignore
@@ -16,6 +16,7 @@
 /rc-local.service
 /rescue.service
 /serial-getty@.service
+/system-update-cleanup.service
 /systemd-ask-password-console.service
 /systemd-ask-password-wall.service
 /systemd-backlight@.service
diff --git a/units/system-update-cleanup.service.in b/units/system-update-cleanup.service.in
new file mode 100644
index 0000000000..116be8bc2d
--- /dev/null
+++ b/units/system-update-cleanup.service.in
@@ -0,0 +1,32 @@
+#  This file is part of systemd.
+#
+#  systemd is free software; you can redistribute it and/or modify it
+#  under the terms of the GNU Lesser General Public License as published by
+#  the Free Software Foundation; either version 2.1 of the License, or
+#  (at your option) any later version.
+
+[Unit]
+Description=Remove the Offline System Updates symlink
+Documentation=man:systemd.special(5) man:systemd.offline-updates(7)
+After=system-update.target
+DefaultDependencies=no
+Conflicts=shutdown.target
+
+# system-update-generator uses laccess("/system-update"), while a plain
+# ConditionPathExists=/system-update uses access("/system-update"), so
+# we need an alternate condition to cover the case of a dangling symlink.
+#
+# This service is only invoked if /system-update exists, i.e. if the
+# condition tested by system-update-generator remains true and the system
+# would be diverted into system-update.target again after reboot. This way
+# we guard against being diverted into system-update.target again, which
+# works as a safety measure, but we will not step on the toes of the
+# update script if it successfully removed the symlink and scheduled a
+# reboot or some other action on its own.
+ConditionPathExists=|/system-update
+ConditionPathIsSymbolicLink=|/system-update
+
+[Service]
+Type=oneshot
+ExecStart=/bin/rm -fv /system-update
+ExecStart=@SYSTEMCTL@ reboot
diff --git a/units/system-update.target b/units/system-update.target
index 4054112c84..3542879706 100644
--- a/units/system-update.target
+++ b/units/system-update.target
@@ -14,3 +14,4 @@ Conflicts=shutdown.target
 After=sysinit.target
 Before=shutdown.target
 AllowIsolate=yes
+Wants=system-update-cleanup.service