units: add system-update-cleanup.service to guard against offline-update loops

Note: the name is "system-update-cleanup.service" rather than
"system-update-done.service", because it should not run normally, and also
because there's already "systemd-update-done.service", and having them named
so similarly would be confusing.

In https://bugzilla.redhat.com/show_bug.cgi?id=1395686 the system repeatedly
entered system-update.target on boot. Because of a packaging issue, the tool
that created the /system-update symlink could be installed without the service
unit that was supposed to perform the upgrade (and remove the symlink). In
fact, if there are no units in system-update.target, and /system-update symlink
is created, systemd always "hangs" in system-update.target. This is confusing
for users, because there's no feedback what is happening, and fixing this
requires starting an emergency shell somehow, and also knowing that the symlink
must be removed. We should be more resilient in this case, and remove the
symlink automatically ourselves, if there are no upgrade service to handle it.

This adds a service which is started after system-update.target is reached and
the symlink still exists. It nukes the symlink and reboots the machine. It
should subsequently boot into the default default.target.

This is a more general fix for
https://bugzilla.redhat.com/show_bug.cgi?id=1395686 (the packaging issue was
already fixed).

2 files changed, 12 insertions, 1 deletions

diff --git a/man/systemd.offline-updates.xml b/man/systemd.offline-updates.xml
index 22aa0354a8..d673cf5db8 100644
--- a/man/systemd.offline-updates.xml
+++ b/man/systemd.offline-updates.xml
@@ -115,7 +115,9 @@
       <listitem>
         <para>The upgrade scripts should exit only after the update is finished. It is expected
         that the service which performs the upgrade will cause the machine to reboot after it
-        is done.</para>
+        is done. If the <filename>system-update.target</filename> is successfully reached, i.e.
+        all update services have run, and the <filename>/system-update</filename> symlink still
+        exists, it will be removed and the machine rebooted as a safety measure.</para>
       </listitem>
 
       <listitem>
diff --git a/man/systemd.special.xml b/man/systemd.special.xml
index de6a5ac6cd..b513a13b5a 100644
--- a/man/systemd.special.xml
+++ b/man/systemd.special.xml
@@ -102,6 +102,7 @@
     <filename>sysinit.target</filename>,
     <filename>syslog.socket</filename>,
     <filename>system-update.target</filename>,
+    <filename>system-update-cleanup.service</filename>,
     <filename>time-sync.target</filename>,
     <filename>timers.target</filename>,
     <filename>umount.target</filename>,
@@ -608,6 +609,7 @@
       </varlistentry>
       <varlistentry>
         <term><filename>system-update.target</filename></term>
+        <term><filename>system-update-cleanup.service</filename></term>
         <listitem>
           <para>A special target unit that is used for offline system updates.
           <citerefentry><refentrytitle>systemd-system-update-generator</refentrytitle><manvolnum>8</manvolnum></citerefentry>
@@ -615,6 +617,13 @@
           exists. For more information see
           <citerefentry><refentrytitle>systemd.offline-updates</refentrytitle><manvolnum>7</manvolnum></citerefentry>.
           </para>
+
+          <para>Updates should happen before the <filename>system-update.target</filename> is
+          reached, and the services which implement them should cause the machine to reboot. As
+          a safety measure, if this does not happen, and <filename>/system-update</filename>
+          still exists after <filename>system-update.target</filename> is reached,
+          <filename>system-update-cleanup.service</filename> will remove this symlink and
+          reboot the machine.</para>
         </listitem>
       </varlistentry>
       <varlistentry>