Sunday, August 30, 2015

About the systemd controversy...

As a troll, one of my favorite targets is "systemd", because it generates so much hate on both sides. For bystanders, I thought I'd explain what that is. To begin with, I'll give a little background.

An operating-system like Windows, Mac OS X, and Linux comes in two parts: a kernel and userspace. The kernel is the essential bit, though on the whole, most of the functionality is in userspace.

The word "Linux" technically only refers to the kernel itself. There are many optional userspaces that go with it. The most common is called BusyBox, a small bit of userspace functionality for the "Internet of Things" (home routers, TVs, fridges, and so on). The second most common is Android (the mobile phone system), with a Java-centric userspace on top of the Linux kernel. Finally, there are the many Linux distros for desktops/servers like RedHat Fedora and Ubuntu -- the ones that power most of the servers on the Internet. Most people think of Linux in terms of the distros, but in practice, they are a small percentage of the billions of BusyBox and Android devices out there.

The first major controversy in Linux was the use of what's known as the microkernel, an idea that removes most traditional kernel functionality and puts it in userspace instead. It was all the rage among academics in the early 1990s. Linus famously rejected the microkernel approach. Apple's Mac OS X was originally based on a microkernel, but they have since moved large bits of functionality back into the kernel, so it's no longer a microkernel. Likewise, Microsoft has moved a lot of functionality from userspace into the Windows kernel (such as font rendering), leading to important vulnerabilities that hackers can exploit. Academics still love microkernels today, but in the real world it's too slow.

The second major controversy in Linux is the relationship with the GNU project. The GNU project was created long before Linux in order to create a Unix-like operating system. They failed at creating a usable kernel, but produced a lot of userland code. Since most the key parts of the userland code in Linux distros comes from GNU, some insist on saying "GNU/Linux" instead of just "Linux". If you are thinking this sounds a bit childish, then yes, you are right.

Now we come to the systemd controversy. It started as a replacement for something called init. A running Linux system has about 20 different programs running in userspace. When the system boots up, it has only one, a program called "init". This program then launches all the remaining userspace programs.

This init system harks back to the original creation of Unix back in the 1970s, and is bit of a kludge. It worked fine back then when systems were small (when 640k of memory was enough for anybody), but works less well on today's huge systems. Moreover, the slight difference in init details among the different Linux distros, as well as other Unix systems like Mac OS X, *BSD, and Solaris, is a constant headache for those of us who have to sysadmin these boxes.

Systemd replaces the init kludge with a new design. It's a lot less kludgy. It runs the same across all Linux distros. It also boots the system a lot a faster.

But on the flip side, it destroys the original Unix way of doing things, becoming a lot more like how the Windows equivalent (svchost.exe) works. The Unix init system ran as a bunch of scripts, allowing any administrator to change the startup sequence by changing a bit of code. This makes understanding the init process a lot easier, because at any point you can read the code that makes something happen. Init was something that anybody could understand, whereas nobody can say for certain exactly how things are being started in systemd.

On top of that, the designers of systemd are a bunch of jerks. Linus handles Linux controversies with maturity. While he derides those who say "GNU/Linux", he doesn't insist that it's wrong. He responds to his critics largely by ignoring them. On the flip side, the systemd engineers can't understand how anybody can think that their baby is ugly, and vigorously defend it. Linux is a big-tent system that accepts people of differing opinions, systemd is a narrow-minded religion, kicking out apostates.

The biggest flaw of systemd is mission creep. It is slowly growing to take over more and more userspace functionality of the system. This complexity leads to problems.

One example is that it's replaced traditional logging with a new journal system. Traditional, text-based logs were "rotated" in order to prevent the disk from filling up. This could be done because each entry in a log was a single line of text, so tools could parse the log files in order to chop them up. The new journal system is binary, so it's not easy to parse, and hence, people don't rotate the logs. This causes the hard drive to fill up, killing the system. This is noticeable when doing things like trying to boot a Raspberry Pi from a 4-gigabyte microSD card. It works with older, pre-systemd versions of Linux, but will quickly die with systemd if something causes a lot of logging on the system.

Another example is D-Bus. This is the core system within systemd that allows different bits of userspace to talk to each other. But it's got problems. A demonstration of the D-Bus problem is the recent Jeep hack by researchers Charlie Miller and Chris Valasek. The root problem was that D-Bus was openly (without authentication) accessible from the Internet. Likewise, the "AllJoyn" system for the "Internet of Things" opens up D-Bus on the home network. D-Bus indeed simplifies communication within userspace, but its philosophy is to put all your eggs in one basket, then drop the basket.


Personally, I have no opinion on systemd. I hate everything. Init was an ugly kludge, and systemd appears to be just as ugly, albeit for difference reasons. But, the amount of hate on both sides is so large that it needs to be trolled. The thing I troll most about is that one day, "systemd will replace Linux". As systemd replaces more and more of Linux userspace, and begins to drive kernel development, I think this joke will one day become true.


18 comments:

matejcepltest said...

I don’t want to get into whole controversy now, just to note that both of your examples seem to be just bugs which need to be fixed, but there should not be much more fuss about that. journald needs to learn how to rotate its binary database, and D-Bus needs to be fixed (perhaps with the help of kdbus?). Neither of these seem to indicate to me that systemd is fundamentally a bad idea.

Unknown said...

Systemd is just the emacs to the init vi.

Paul Miller said...

There are lots of information about latest technology and how to get trained in them, like UNIX Training Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies(UNIX Course in Chennai). By the way you are running a great blog. Thanks for sharing this.

UNIX Training Chennai | UNIX Course in Chennai

ycz51309 said...

Good overview of the historical parts of Linux.

You forgot to mention whether systemd operates in user land or in kernel land. It's replacing init, the first user land process, so obviously the answer is that it's a user land process, but the way this post is written casts doubt on that.

There is also no explanation of why it's difficult to tell what's going on in systemd. Surely there are configuration files that make it up, if not scripts. How else would one ensure that something (e.g. a service) is started on boot?

Luckily, nobody reads the comments, including... no, especially the author, so these omissions are likely to never be acknowledged nor corrected. haha.

Siegfried Kiermayer said...

I'm waiting for pulse audio beeing included in systemd to have proper a boot sound :D

FindAnISP said...

Systemd and journal handling are the main reasons I'm dumping Fedora (with its insanely short support period and two botched upgrades leaving unbootable systems). I'm content to use Ubuntu LTS and stay in the 1980s

dre said...

http://devuan.org

Richard Maw said...

Journald already has log rotation. http://www.freedesktop.org/software/systemd/man/journald.conf.html#SystemMaxUse=

The d-bus thing is unrelated to systemd. It's the traditional d-bus daemon that was spectacularly mis-configured. I recently spoke to one of the d-bus maintainers who wants to rip it all out.

Richard Maw said...

Also, I find the service file configuration a lot easier to read than init scripts.

I suggest that it's only easier for some to read init scripts because they have the experience with debugging them, since in general it's much easier to understand what's going on with declarative configuration rather than imperative.

In my experience Lennart and the other systemd developers are more civil than Linus.

Greg Nation said...

> The biggest flaw of systemd is mission creep.

I agree completely. I've lost count of the number of responsibilities that have been inappropriately given to systemd, but suffice it to say, a lot of systemd's functionality should be shifted to pulseaudio.

John Thacker said...

dumping Fedora (with its insanely short support period and two botched upgrades leaving unbootable systems). I'm content to use Ubuntu LTS and stay in the 1980s.

Surely the analogous distro to Ubuntu LTS is RHEL/CentOS, not Fedora? Ubuntu is switching to systemd anyway.

fredex said...

Mission Creep, or what some of us call "Feeping Creaturism".

Stefan Anica said...

This article is more full of bullshit than a bull stable .... with shit in it.

John Morris said...

They don't want to replace the kernel, they are more than happy to leverage Linus's good work on what they see as a collection of device drivers. No, they want to replace the GNU/X in the traditional Linux/GNU/X arrangement. All of the command line tools, up to and including bash are to go, replaced with the more Windows like tools most of the systemd developers grew up on, while X and the desktop environments all get rubbished for Wayland and GNOME3.

And I would wish them luck, the world could use more diversity in operating systems. So long as they stayed the hell over at RedHat and did their grand experiment and I could still find a Linux/GNU/X distribution to run. But they had to be borg and insist that all must bend the knee and to that I say HELL NO!

Unknown said...

I find it funny that they say that init doesn't scale and isn't made for large systems.

The person saying this has NO idea of what a UNIX system is, Sun SPARC, HP PA RISC and such don't use systemd. They use REAL UNIX with init and have no need of the toy called systemd that is really no more than a control-power grab into linux.

Don il said...
This comment has been removed by the author.
Don il said...

BTW, comments such as next:
"This article is more full of bullshit than a bull stable .... with shit in it."
bring to my mind all the comments from Microsoft fans/paid-for-shills in other forums. They tend to attack anyone not accepting things imposed on them.

Don il said...

@FindAnIsp
I downloaded all of Debian Wheezy's DVDs and I'm sticking with them for all my new Linux installations. I don't like systemd, and it bothers me that most distros have already embraced it.