Starting up Linux
Want to know what happens when you boot up your Linux PC? Neil Bothwick dissects the boot process, step by step…
No, this isn’t another article on getting going with Linux for new users. This is about how Linux starts itself. Part of the joy of Linux is that as a user you retain full control over the entire system, if you want it. To many users this is just some form of magic, except that instead of using smoke and mirrors, most of the activity is hidden behind a fancy splash screen. We’re going to look at the whole process, from when you press the power button to when the desktop appears. This can be broken down into a number of stages: POST, Bootloader Kernel loading Initial ram disk Device initialisation Init Service startup and runlevels Display Manager, Desktop startup
We’ll look at each step in turn, examining what happens and the choices you can make. As with so many aspects of Linux, there are alternative ways to handle each part of the process. Let’s start with an overview of what each stage involves.
POST is Power On Self Test, a routine run by the computer’s hardware, checking memory, looking for connected drives and so on. It’s at this stage that you can hold down a key to access the firmware’s configuration menus or pop up a menu to select which device to boot from.
The bootloader provides a menu from which you choose your operating system on dual boot systems as well as other boot options. It’s sometimes hidden, waiting a few seconds before going with the default. Pressing the space bar at this point usually shows it. The bootloader then loads the kernel, passing it some options such as the location of your system partition and, usually, also loads an initial ramdisk. That’s a disk image loaded into memory that contains the software needed to initialise your hardware.
Once the kernel is loaded it runs a program called init, which controls everything from here on. Services are background programs, such as a cron daemon or a system logger, that are fired up at this stage. Most distros define various runlevels, one of the options selectable from a boot menu, that define how the computer should boot up. For example, traditionally runlevel 1 was single user mode, 3 was multiuser text mode and 5 gave the graphical desktop. A display manager is the program that asks for your login details, and may let you select a desktop environment, before loading the graphical environment. Finally, your desktop loads, possibly starting some user-level programs and enabling you to use the computer.
Starting up
There’s not too much to say about POST. You’ll need to read your hardware’s manual to see what options are available to you. This loads the basic firmware, what was called a BIOS on older systems. You can do clever things like overclocking your computer if you’re feeling brave, or reset to defaults if you’re not. It’s worth noting that motherboard manufacturers provide firmware updates to fix bugs, so updating this is as important as updating any software in your operating system.
Now we get to one of those points where there are choices to be made. In older BIOS systems, the hardware loaded software from the first 446 bytes of the hard disk to boot the computer. This tiny piece of code in the Master Boot Record (MBR) then has to find the rest of the bootloader, usually GRUB, on the hard disk. GRUB, the Grand Unified Boot
Loader, presents a menu from which you can choose boot options. There’s generally a default option that’s activated if you don’t make a selection within a certain time. There are other bootloaders, such as lilo, but GRUB has become the standard because it handles a range of hardware and is able to generate its menus automatically, which is important when installing a new distro, particularly for dual boot. GRUB’s menu file is located at /boot/grub/grub.cfg . While you can edit this by hand, the recommended route is to change settings in /etc/default/grub and then run $ sudo grub-mkconfig -o /boot/grub/grub.cfg to generate a new menu file. Some distros hide the menu, only showing it if you press a key. You can change this, as well as the timeout, by editing /etc/default/grub as above.
GRUB is simply a bootloader, and so it then loads whichever operating system kernel you tell it to. However, this is no longer necessary with modern UEFI hardware, which can load a kernel directly if it’s been set up to do so. UEFI expects the first partition of your disk to contain a FAT filesystem and it looks there for any loadable kernels. It will load the default one, which is set in the firmware, or you can press a key for a menu listing all your options.
This removes the need for a bootloader, but most people still prefer to have a boot manager – something that gives them a choice of operating systems or kernels at boot time. You can still use GRUB for this, but it’s overkill. Systemd’s boot ctl program uses a simple configuration file to create menu options, that appear in a simple text menu. Alternatively, you can use rEFInd ( www.rodsbooks.com/refind) which uses a nice graphical menu, and is usually able to create that automatically based on what’s on your hard disk.
Kernels and ramdisks
One of the reasons for using a boot manager, even if you have a UEFI system that can boot a kernel directly, is to pass parameters to the kernel. You can usually view the parameters used to boot the kernel on your running system with the following: $ cat /proc/cmdline Some of these options are cosmetic, such as the ones that load a splash screen, but some are critical. For example, the root= parameter tells the kernel which partition holds the operating system, a kernel without the rest of the OS has rather limited usefulness. The kernel options may also specify the init system to use, which we’ll come to shortly or specify a path to an initrd.
Linux has a monolithic kernel of sorts. A monolithic kernel contains all the code and drivers needed to load an OS, as opposed to a micro-kernel that contains only the core code and loads the rest on demand. Arguments on which is better have lasted years, but Linux has ended up being a bit of both.
In the days when you compiled your own kernels, you included the parts needed to boot your system: drivers for your hard disk controllers, the filesystem you used, your network card and so on. That gave you a kernel with everything you needed, but it wouldn’t necessarily work anywhere else. Distros need to cover all bases, but including drivers for every device in the kernel would make it bloated, so loadable modules were added. These are extra parts of
driver code that are loaded on demand; your computer would load the driver for your network card and leave the others untouched, taking up only a little disk space.
That leaves the question of how to mount your hard disk if the drivers are stored as modules on that disk? And so the initial ram disk was born. This is a file containing a disk image with all the essentials included. It’s stored in the same directory as the kernel and so, like the kernel, it’s loaded by the bootloader. The kernel mounts this disk at a temporary location, loads the modules it needs and then unmounts the disk and switches the root directory to the correct location. It doesn’t matter how large this ramdisk is, within reason, because it’s only loaded into memory for a few seconds, before being discarded. The ramdisk takes care of everything that needs to be done to get at the rest of the OS, even if that’s on a strange format medium or elsewhere on the network.
It’s possible to build the ramdisk image into the kernel file, but the standard systems used by distros all use a separate file for it. Look in your /boot directory and you’ll see a corresponding initramfs or initrd file for each kernel. Strictly speaking, the kernel uses an initramfs these days, but the initrd terminology refuses to die. The difference is in the underlying structure, to a user they work in the same way.
One of the first things the kernel does is scan the available hardware and loads drivers for it. Unlike with other operating systems where driver installation is a separate process, Linux includes drivers for the vast majority of hardware by default. You can see this in the kernel messages, shown by running $ dmesg | less You’ll see that the kernel identifies the CPU, motherboard and memory early on and then goes on to find all your other hardware. By the time the operating system starts to load, all your hardware should be available to it.
Service startup
If you want a better idea of what is happening when your computer boots, get rid of the splash screen that hides the real work. You can usually do this by pressing Esc as soon as it appears, or you can use your boot menu’s editor to remove any references to splash and quiet from the kernel options. When you boot you’ll see lines of text scrolling by, most or all of them reporting OK. These are services being started.
Services, also known as daemons, are programs that run in the background, each one performing a specific task. These can be services like NTP (Network Time Protocol) that keeps your computer’s clock running accurately, or cron, a service that runs commands at regular intervals or a system logger that provides a means or recording status messages from the other programs on your computer. These are services that run on most computers, then there are the server daemons, web server, mail servers and so on that are used to provide services to other computers, that you may not be running on a desktop system.
“One of the first things the kernel does is scan the available hardware and load drivers for it.”
Running these services is taken care of by a service manager. This is a complex task: services need to be started when the computer boots and cleanly stopped when the computer shuts down or reboots, but there’s more to it than that. Some services depend on others, the order is important. There’s also a need to make sure that services continue to run and that some action be taken if a service fails.
The traditional service manager for Unix and Linux is based on SysVInit, from the Unix days. This uses a set of runlevels, each one representing a particular state of the system. The standard runlevels are: Halt Single-user mode Multi-user mode Multi-user mode with networking Not used As runlevel 3 plus display manager Reboot Services are started and stopped by scripts that are stored in /etc/init.d . There’s also a directory for each of the runlevels, /etc/rc0.d to /etc/rc6.d . These contain symlinks to the scripts in /etc/init.d . So when you boot the computer in runlevel 3, all the scripts from /etc/rc3.d are run. On startup, they’re run in order and passed with the argument “start”. On shutdown or reboot they’re run in reverse order with the argument “stop”.
How does the system know the correct order to start services? That’s done on the names of the scripts, so they all start with numbers, making it easy to specify an order. So: S01syslog-ng S02cron S03firewalld
This means the system logger starts first, then the cron daemon and then the firewall. The init scripts are run in serial, so one has to finish starting before the next one is run. It’s a fairly crude system and can slow down the boot process if one service takes a while to fire up, but it has been used since Linus Torvalds was a lad and is reliable. Note that the details of how each distro implements
SysVInit and its runlevels can vary, both in terms of the location of the scripts and the definitions of the runlevels.
The new kid
There are a number of alternative service managers, such as Upstart (now dead) and Gentoo’s rather pleasing openrc, but none of them have gained much mainstream acceptance, with one notable exception. The muchdiscussed (where discussed also includes argued over, flamed about and even reviled) systemd attempts to remedy some of the limitations of SysVInit.
One of the core differences is that SysVInit uses shell scripts to start and stop services. Each service has a complete script to handle this, with several consequences. A lot of work is duplicated, even with some systems providing helper functions, reading the scripts to see exactly what is done when a service is restarted isn’t a simple task and there’s fragmentation between distros as each one uses its own scripts to tie in with their way of doing things.
By contrast, systemd uses service files that simply tell
systemd what to do, rather than doing everything. Most service files are only a few lines and a quick glance is all that’s needed to see what will be done. As all the real work is done by systemd itself, it also means that these service files are standardised, and the same file usually works across most distros. More importantly, the service files include dependencies, so a file for a web server will state that networking needs to be started first, while the one for the firewall can specify that it needs to be started before the network comes up.
So these three will be started sequentially, each waiting for its predecessor, but other services are free to start when they want. There’s no reason to delay starting cron while the network gets an address from DHCP just because it has a higher number in front of its name. And there’s another benefit to having systemd start the programs itself: it can monitor them if requested. A service file can tell the system what to do in the event of a program failing, from restarting it (use with caution) to notifying you.
Of course, systemd does a lot more, but those are separate components of the overall systemd infrastructure and nothing to do with service starting in the main.
Networking startup
We’ve looked at starting services in general, whether by SysVInit or systemd, but there’s one group of services that deserve special attention, those relating to networking. We are not talking about services that use the network but those that are needed to make it work in the first place. Linux relies on networking, as do most people these days, even if it’s on a standalone machine. There are three points at which networking is initiated during the startup process. The localhost interface (lo) is brought up early on. This enables network-aware programs to communicate on the local computer and requires little else.
External network interfaces are brought up soon after that. Remember what we said about the order of service startup? Some services depend on a network interface, so it has to be available before they start and is usually brought up quite early. If you have a single, wired interface this is very simple. Distros generally use Dynamic Host Config Protocol to obtain an address and the other routing information it needs from a DHCP server on the network (usually your modem/router) and then sets up the network for you.
If you need a static address, you’ll have to do some setting up yourself. You can still use the DHCP server to give a static address: most routers will enable you to allocate specific addresses to individual MAC addresses (the MAC address is the unique hardware address of a network interface). Do it this way and you still use DHCP, but get the same address every time. Alternatively, use your distro’s network configuration tool to set up a static address, which must be on the same range (subnet) as the rest of your network. In addition to the computer’s address, you’ll need to give it a gateway address (usually that of your router) and one or more DNS servers. You can use your ISP’s DNS, a public one like those provided by Google, or use the address of your router, which will forward requests to an external server.
Wireless networking is another matter as multiple networks and authentication credentials may be involved. It’s usually handled by a program like NetworkManager. Depending how you have NetworkManager configured, it may search for and connect to a known network when the
computer is booting, or it may not do so until you log into the desktop. The latter approach is the default. It’s slower, because the connection negotiation doesn’t start until after the desktop loads, but it’s more secure. There’s usually an option to enable a network setting globally if you want
NetworkManager to start it during service startup. We say “usually” because there are different user interfaces used on various desktops.
What’s a Display Manager?
In the “good” old days, when all CPUs were 32-bit and all distros fitted on a single CD, you would boot to a text console, and log in as your user. Then, if you wanted one of those newfangled GUI things, you would type startx . Some people still work that way, but the normal approach is to use a display manager. This is run as a service, just like any other, but only from the GUI runlevel.
The display manager is the program that opens the graphical login screen, although it keeps running as a service after you’ve logged in, ready to handle the next login request. The are a number of display managers in common use, including gdm, lightdm and sddm. Most enable some configuration, such as changing the appearance of the login screen, but since all you normally want to do is type in your login details, time spent on this could probably be used more effectively elsewhere. If you’re the only user of a computer, you can configure most display managers to automatically login a specified user when they start. But think about the security implications when you do this. Unless you have an encrypted hard drive that requires a password, anyone turning on the computer will be logged in as you with no authentication.
Desktop startup
We’re nearly there. The bootloader has loaded the kernel, the kernel has run init and the service manager has all your background services running. You have logged it as the display manager screen, so what else is there to do? The display manager now starts an X session and loads the Desktop Environment. You may think this is the simple part, but it can get quite complicated here as each desktop environment has its own way of doing this. The DE will load the wallpaper, usually a panel and a menu or two and often some widgets. What it loads and how you configure it depends entirely on the desktop you use. There are some standard mechanisms specified by www.freedesktop.org, but the various desktops are more inclined to treat these as gentle hints rather than actual standards.
Your desktop environment’s documentation will have more information on this, or you could rummage through the settings programs to find what you want. As well as the obvious things like desktop wallpaper and window decoration styles, you can usually set up custom keyboard shortcuts, add widgets or icons to the desktop and have specific programs start up automatically when the desktop loads. There may also be extra services started with the desktop. The services we looked at earlier on are started by root and available to the whole system, but there may also be services started for a user. A good example of this is pulseaudio, which is generally not run as root and therefore starts up with the desktop, run by the logged in user.
That’s it! You’re logged in and ready to use your computer. More importantly, we hope you now have a better understanding of what just happened.
Shutting down
We’ve looked at the boot process in some detail, but what happens when you shutdown or reboot the computer? Basically, the OS retraces it steps. First thing is no new logins are allowed, then the desktop closes open applications and shuts down and user services before exiting. Then the service manager stops the various running services, largely in the reverse order, so shutting down the display manager first to unmounting filesystems last.
Some initial ramdisks keep their scripts running in the background so when init exits, they can undo whatever they started, then the kernel unloads and the screen goes blank. As with the boot process, doing this without a splash screen will let you see the progress of the shutdown, which can be useful if it seems to take longer than it should. If you can see what is causing the delay, you may be able to fix it, but at least you’ll know why it’s happening – splash screens and progress indicators are only user friendly when there’s progress.
That was complicated, right? Linux is about learning and empowering the user, modern distros are super-easy to use but everything is laid bare for you to play with and experiment with. So have fun, enjoy Linux and keep learning!