This post appeared originally in our sysadvent series and has been moved here following the discontinuation of the sysadvent microsite
One common complaint about systemd is that it does «too much», where the threshold for the appropriate amount of action is left unspecified. Some of the stuff it can do is hold your hand and offer some comfort functions.
The result
is that you don’t need to know where your unit files are. You can, and systemd
will tell you, but you don’t need to know the path a priori. So here are some
functions of systemctl
that you don’t need to know about, but which can make
your life a little easier.
systemctl cat
This will print your service file, with the path in a comment at the top:
$ systemctl cat systemd-journald.service
# /usr/lib/systemd/system/systemd-journald.service
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
## blah blah blah…
The first line with the path is not actually included in the service file, and
there’s a good chance it’ll be printed in another color on your machine. This
will print the unit file that systemd will actually use for that unit. In this
case it’s running the vendor-supplied unit file, which you can tell by the fact
that it’s in /usr/lib/systemd
; your files are supposed to go in
/etc/systemd/
. Unless it’s a temporary file, they go in /run/systemd
, or it
might be a user file, in yet another set of paths that I don’t actually care to
remember or think about. Hence this blog post.
systemctl edit
Now, if you for some reason need to change a unit file, use systemctl edit
. I
prefer systemctl edit --full
which will give you the full file to edit, rather
than expecting you to write a replacement snippet blind. systemctl edit
will
give you the appropriate file to edit, and save it in the appropriate place,
and it will run systemctl daemon-reload
. And of course it relies on ed, the
standard editor.¹
By default (without --full
) it will create an appropriate
foo.d/override.conf
; with --full
it will create a foo.$unit
. My intuition
tells me that if I make no changes, no new file will be written, and that’s
correct for the default version, but with --full
you’re in effect creating
a copy of the vendor unit file as it exists at that time.
If you need to delete something from a unit file, use --full
and delete the
line there.
It has some extra features:
--force
is handy if you want to start a brand new unit file: it’ll create a unit file with that name if one doesn’t already exist. Requiring this to create new unit files means you won’t inadvertently create a new unit file if you do a typo or are confused about what the service name is (like me, I always expect «postgres», not «postgresql» (yes, I know about auto-completion)).--runtime
makes a temporary edit that will be lost on the next reboot.
systemd-delta
Once you’ve edited a unit file, it can start diverging even more from the
vendor-supplied one as upstream thinks up new and exciting ways to do
blinkenlights. systemd-delta
shows you how your system is different from a
vanilla system, with diffs for overrides, which file supersedes what, and so
on. Sadly, the output from systemd-delta
isn’t sorted or stable, and there’s
no systemd-delta-delta
.
If you’ve used edit
to create a foo.d/override.conf
, it’ll show up as
[EXTENDED]
. If you used --full
and made a copy, it’ll show up as
[OVERRIDDEN]
. The naming makes sense if you think about it, although it would
be OK too if I didn’t have to think and the override.conf
version matched with
[OVERRIDDEN]
.
systemctl is-system-running
This is a very simple indicator of whether systemd thinks everything is running
properly. systemctl is-system-running
outputs the same thing as the «State»
line in systemctl status
.
- What you want is «running» and exit code 0.
- What you might get is «degraded» and exit code > 0. In that case, do
systemctl list-units --state=failed
to see what it’s unhappy about. If you don’t want to fix something, you can comfort systemd by telling itsystemctl reset-failed $foo
. This will not restart services or anything, it will just convince systemd that things are fine (morbid comic). - There are other options, which are listed in table form in systemctl(1).
So by adding systemctl is-system-running
to your login script or monitoring
system, you can get a quick feel for whether something is
off.
¹ Blatant lie. It uses $EDITOR
.
Thoughts on the CrowdStrike Outage
Unless you’ve been living under a rock, you probably know that last Friday a global crash of computer systems caused by ‘CrowdStrike’ led to widespread chaos and mayhem: flights were cancelled, shops closed their doors, even some hospitals and pharmacies were affected. When things like this happen, I first have a smug feeling “this would never happen at our place”, then I start thinking. Could it?
Broken Software Updates
Our department do take responsibility for keeping quite a lot ... [continue reading]