Boot Process Demystified

Why This Matters

You press the power button. Sixty seconds later, you are staring at a login prompt. What just happened in those sixty seconds?

Understanding the boot process is not academic trivia. When a server refuses to boot, when a kernel update goes wrong, when you need to recover a system with a corrupted root filesystem, when a misconfigured GRUB entry leaves you staring at a blinking cursor -- that is when this knowledge saves you. Every minute a production server is down costs money. Knowing the boot sequence means you can diagnose where it broke and fix it.

This chapter walks through every stage, from the moment electricity hits the motherboard to the moment systemd presents you with a login prompt. You will learn to configure GRUB, inspect initramfs, analyze boot performance, and rescue a system that will not boot.

Try This Right Now

# How long did your system take to boot?
systemd-analyze

# Breakdown by stage
systemd-analyze blame | head -15

# Visual chain of the boot process
systemd-analyze critical-chain

# What target (runlevel) are you running?
systemctl get-default

# When was this system last booted?
who -b

# See the kernel's boot parameters
cat /proc/cmdline

# Check for boot errors
journalctl -b 0 -p err

The Boot Sequence: Bird's Eye View

+------------------------------------------------------------------+
|  1. FIRMWARE (BIOS or UEFI)                                     |
|     - Hardware initialization (POST)                             |
|     - Find and load the bootloader                               |
+------------------------------------------------------------------+
         |
         v
+------------------------------------------------------------------+
|  2. BOOTLOADER (GRUB2)                                          |
|     - Present boot menu                                          |
|     - Load the kernel and initramfs into memory                  |
+------------------------------------------------------------------+
         |
         v
+------------------------------------------------------------------+
|  3. KERNEL                                                       |
|     - Initialize hardware, memory, CPU                           |
|     - Mount initramfs as temporary root                          |
|     - Find and mount the real root filesystem                    |
|     - Start PID 1                                                |
+------------------------------------------------------------------+
         |
         v
+------------------------------------------------------------------+
|  4. INIT SYSTEM (systemd, PID 1)                                |
|     - Mount remaining filesystems                                |
|     - Start services according to the target                    |
|     - Present login prompt                                       |
+------------------------------------------------------------------+

Let us examine each stage in detail.

Stage 1: Firmware -- BIOS and UEFI

When you press the power button, the CPU starts executing code from a chip on the motherboard -- the firmware.

BIOS (Legacy)

The Basic Input/Output System is the older standard (1980s):

1. POST (Power-On Self-Test) -- checks CPU, RAM, and basic hardware
2. Looks for a bootable device based on the boot order (hard disk, USB, network)
3. Reads the first 512 bytes of the boot device -- the Master Boot Record (MBR)
4. Executes the bootloader code found in the MBR

MBR limitations:

• Only 446 bytes for bootloader code (tiny!)
• Maximum disk size: 2 TB
• Maximum 4 primary partitions

UEFI (Modern)

The Unified Extensible Firmware Interface is the modern replacement:

1. POST -- same hardware checks
2. Reads the EFI System Partition (ESP) -- a FAT32 partition, usually mounted at /boot/efi
3. Runs the bootloader EFI application (e.g., grubx64.efi)
4. Supports Secure Boot (only runs cryptographically signed bootloaders)

UEFI advantages:

• Supports disks larger than 2 TB (GPT partitioning)
• Faster boot (can skip legacy compatibility)
• Secure Boot protects against rootkits
• Can boot directly from EFI applications (no MBR needed)

# Check if your system uses UEFI or BIOS
ls /sys/firmware/efi 2>/dev/null && echo "UEFI boot" || echo "Legacy BIOS boot"

# If UEFI, see the boot entries
efibootmgr -v 2>/dev/null

# See the EFI System Partition
mount | grep efi
ls /boot/efi/EFI/ 2>/dev/null

Think About It: Why does the EFI System Partition use FAT32 and not ext4 or XFS? (Hint: the firmware needs to read it before any Linux driver is loaded.)

Stage 2: The GRUB2 Bootloader

GRUB2 (GRand Unified Bootloader version 2) is the standard bootloader on most Linux distributions. Its job is to load the kernel and initramfs into memory.

What GRUB Does

1. Presents a boot menu (if configured to show one)
2. Lets you select which kernel to boot, edit boot parameters, or enter a recovery shell
3. Loads the selected kernel image (vmlinuz-*) into memory
4. Loads the initial RAM filesystem (initramfs-* or initrd-*) into memory
5. Passes control to the kernel with the specified boot parameters

GRUB Configuration

# Main GRUB config file (DO NOT edit directly)
ls -l /boot/grub/grub.cfg      # Debian/Ubuntu
ls -l /boot/grub2/grub.cfg     # RHEL/Fedora

# Edit GRUB defaults instead
cat /etc/default/grub

The /etc/default/grub file contains the settings you should modify:

# Key settings in /etc/default/grub:

GRUB_DEFAULT=0                    # Default menu entry (0 = first)
GRUB_TIMEOUT=5                    # Seconds to show menu (0 = skip, -1 = wait forever)
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"  # Kernel params for default boot
GRUB_CMDLINE_LINUX=""             # Kernel params for ALL boots
GRUB_DISABLE_RECOVERY="false"    # Show recovery entries?

Common kernel parameters you might add:

Regenerating GRUB Configuration

After changing /etc/default/grub, regenerate the actual config:

# Debian/Ubuntu
sudo update-grub

# RHEL/Fedora
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

# For UEFI systems on RHEL/Fedora
sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

Distro Note: Debian/Ubuntu provide the update-grub convenience wrapper. RHEL/Fedora require the full grub2-mkconfig command. Both do the same thing -- regenerate /boot/grub/grub.cfg from templates and /etc/default/grub.

WARNING: Never edit /boot/grub/grub.cfg directly. Your changes will be overwritten the next time update-grub runs (e.g., during a kernel update). Always edit /etc/default/grub and regenerate.

GRUB Interactive Editing

At the GRUB menu (press Shift during boot on BIOS, or Esc on UEFI):

• Press e to edit the selected entry's boot parameters
• Find the line starting with linux (the kernel command line)
• Add or modify parameters
• Press Ctrl+X or F10 to boot with those changes (temporary, one-time)
• Press c to drop to a GRUB command-line shell

This is invaluable for recovery. For example, to boot into a root shell:

# At the GRUB edit screen, add to the end of the 'linux' line:
init=/bin/bash
# Then press Ctrl+X to boot

GRUB Installation

# Install GRUB to a disk's MBR (BIOS)
sudo grub-install /dev/sda

# Install GRUB to EFI System Partition
sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi

# Verify installation
sudo grub-install --recheck /dev/sda

Stage 3: The Kernel Loads

Once GRUB loads the kernel into memory and passes control, the kernel takes over:

Kernel Initialization Sequence

  1. Decompress kernel image (vmlinuz -> vmlinux)
  2. Initialize CPU and memory management
  3. Initialize essential hardware (console, timer, interrupts)
  4. Mount initramfs as temporary root (/)
  5. Run /init from initramfs
  6. initramfs loads necessary drivers (storage, filesystem)
  7. Mount the real root filesystem
  8. Pivot root from initramfs to real filesystem
  9. Execute /sbin/init (systemd) as PID 1

You can watch this process in dmesg:

# First few kernel messages -- decompression and initialization
dmesg | head -30

# Look for the root filesystem mounting
dmesg | grep -i "root\|mount\|ext4\|xfs"

# Look for the transition to userspace
dmesg | grep -i "init\|systemd\|pid 1"

initramfs: The Bridge

The kernel cannot mount your root filesystem without the right drivers. But those drivers are stored ON the root filesystem. Chicken-and-egg problem.

The solution is initramfs (initial RAM filesystem) -- a compressed archive containing just enough to find and mount the root filesystem:

+------------------------------------------+
|  initramfs contents:                     |
|  - Filesystem drivers (ext4, xfs, etc.)  |
|  - Storage drivers (SCSI, NVMe, RAID)   |
|  - LVM tools (if root is on LVM)        |
|  - LUKS tools (if root is encrypted)    |
|  - /init script (orchestrates mounting)  |
|  - udev rules for device detection      |
+------------------------------------------+

# See your initramfs files
ls -lh /boot/initrd.img-*    # Debian/Ubuntu
ls -lh /boot/initramfs-*     # RHEL/Fedora

# Examine initramfs contents
lsinitramfs /boot/initrd.img-$(uname -r) | head -30              # Debian/Ubuntu
lsinitrd /boot/initramfs-$(uname -r).img | head -30              # RHEL/Fedora

# Rebuild initramfs (if you've changed modules or drivers)
sudo update-initramfs -u -k $(uname -r)    # Debian/Ubuntu
sudo dracut --force                          # RHEL/Fedora

Think About It: If your root filesystem is on a simple SATA drive with ext4, the kernel might have those drivers built-in and could theoretically skip initramfs. But if your root is on an encrypted LVM volume on top of software RAID, initramfs is absolutely essential. Why?

Stage 4: systemd Takes Over (PID 1)

Once the kernel mounts the root filesystem, it executes /sbin/init, which on modern distributions is a symlink to systemd. systemd becomes PID 1 -- the parent of all processes.

Boot Targets (Modern Runlevels)

systemd uses targets instead of the old SysVinit runlevels:

+----------+-----------------------+-------------------------------+
| Runlevel | systemd Target        | Description                   |
+----------+-----------------------+-------------------------------+
|    0     | poweroff.target       | Halt the system               |
|    1     | rescue.target         | Single-user mode, root shell  |
|    2     | multi-user.target *   | (Debian: same as 3)           |
|    3     | multi-user.target     | Full multi-user, no GUI       |
|    4     | multi-user.target     | Unused (customizable)         |
|    5     | graphical.target      | Multi-user with GUI           |
|    6     | reboot.target         | Reboot                        |
+----------+-----------------------+-------------------------------+

# What target is currently active?
systemctl get-default

# Change the default target
sudo systemctl set-default multi-user.target    # Server (no GUI)
sudo systemctl set-default graphical.target     # Desktop (with GUI)

# Switch to a different target right now (without reboot)
sudo systemctl isolate rescue.target            # Single-user mode
sudo systemctl isolate multi-user.target        # Back to normal

What systemd Does During Boot

# See the dependency order of boot
systemd-analyze critical-chain

# Output (example):
# graphical.target @12.345s
# └─multi-user.target @12.300s
#   └─network-online.target @10.200s
#     └─NetworkManager-wait-online.service @5.100s +5.099s
#       └─NetworkManager.service @3.200s +1.899s
#         └─dbus.service @2.800s +0.399s
#           └─basic.target @2.700s
#             └─sockets.target @2.699s

Analyzing Boot Performance

# Total boot time breakdown
systemd-analyze
# Output: Startup finished in 2.5s (firmware) + 3.1s (loader) +
#         4.2s (kernel) + 12.3s (userspace) = 22.1s

# Which services took the longest?
systemd-analyze blame | head -20

# Generate an SVG boot chart (visual timeline)
systemd-analyze plot > boot-chart.svg
# Open in a browser to see a graphical timeline

# Check for units that are slowing down boot
systemd-analyze critical-chain --no-pager

# See target dependencies
systemctl list-dependencies multi-user.target | head -30

Boot Log

# View complete boot log for this boot
journalctl -b 0

# View boot log for the previous boot
journalctl -b -1

# Show only errors from this boot
journalctl -b 0 -p err

# Show kernel messages from boot
journalctl -b 0 -k

# List all recorded boots
journalctl --list-boots

Troubleshooting Boot Failures

Identifying Where Boot Failed

+----------------------------------------------------+
|  Symptom                    | Stage That Failed     |
|-----------------------------|-----------------------|
|  No display, no beep       | Hardware/BIOS         |
|  BIOS screen, then blank   | Bootloader (GRUB)     |
|  GRUB menu appears,        | Kernel loading or     |
|  then kernel panic          | initramfs             |
|  Kernel loads, services     | systemd / services    |
|  fail, no login             |                       |
|  Login appears but can't    | Authentication / PAM  |
|  log in                     |                       |
+----------------------------------------------------+

Rescue Mode

If the system boots far enough for GRUB to work:

Method 1: GRUB Kernel Parameter Edit

1. At the GRUB menu, press e
2. Find the line starting with linux
3. Add systemd.unit=rescue.target at the end
4. Press Ctrl+X to boot
5. You will get a root shell (may require root password)

Method 2: Emergency Mode

For more serious problems (filesystem issues):

1. At GRUB, press e
2. Add systemd.unit=emergency.target to the kernel line
3. Press Ctrl+X
4. Only the root filesystem is mounted (read-only)

# In emergency mode, remount root as read-write
mount -o remount,rw /

# Fix the problem (e.g., edit /etc/fstab)
vi /etc/fstab

# Reboot
reboot

Method 3: init=/bin/bash

When even systemd will not start:

1. At GRUB, press e
2. Replace ro quiet splash with rw init=/bin/bash
3. Press Ctrl+X
4. You drop directly to a bash shell as root, no authentication

# Root filesystem may be read-only; remount it
mount -o remount,rw /

# Reset a forgotten root password
passwd root

# Fix other issues as needed

# When done, remount read-only and reboot
mount -o remount,ro /
exec /sbin/reboot -f

WARNING: init=/bin/bash bypasses ALL authentication. Anyone with physical access to the GRUB menu can become root. This is why physical security matters, and why some organizations configure GRUB passwords.

Fixing a Broken GRUB

If GRUB itself is corrupted:

# Boot from a live USB/ISO
# Mount the installed system's partitions:
sudo mount /dev/sda2 /mnt          # Root partition
sudo mount /dev/sda1 /mnt/boot/efi # EFI partition (if UEFI)
sudo mount --bind /dev /mnt/dev
sudo mount --bind /proc /mnt/proc
sudo mount --bind /sys /mnt/sys

# Chroot into the installed system
sudo chroot /mnt

# Reinstall GRUB
grub-install /dev/sda              # BIOS
grub-install --target=x86_64-efi --efi-directory=/boot/efi  # UEFI

# Regenerate config
update-grub                        # Debian/Ubuntu
grub2-mkconfig -o /boot/grub2/grub.cfg  # RHEL/Fedora

# Exit chroot and reboot
exit
sudo reboot

Fixing a Broken /etc/fstab

A common boot failure: you added an entry to /etc/fstab with a typo, and now the system drops to an emergency shell on boot.

# In emergency mode:
# The error message will tell you which line failed
# Mount root as read-write
mount -o remount,rw /

# Edit fstab
vi /etc/fstab

# Either fix the typo or comment out the problematic line
# Save and reboot
reboot

Think About It: Why does a bad /etc/fstab entry prevent booting, even if the bad entry is for a non-root filesystem like /data? (Hint: look at the mount options -- is nofail set?)

Pro tip: always add nofail to non-critical mount entries in /etc/fstab:

/dev/sdb1  /data  ext4  defaults,nofail  0  2

This tells systemd: "Try to mount this, but do not fail the boot if it cannot be mounted."

Hands-On: Boot Analysis

# 1. Analyze your boot time
systemd-analyze

# 2. Find the slowest services
systemd-analyze blame | head -10

# 3. See the critical chain (the longest dependency path)
systemd-analyze critical-chain

# 4. Check for failed services during boot
systemctl --failed

# 5. See what the kernel was told at boot
cat /proc/cmdline

# 6. Examine the initramfs for your current kernel
file /boot/initrd.img-$(uname -r) 2>/dev/null || \
file /boot/initramfs-$(uname -r).img 2>/dev/null

# 7. Check GRUB configuration
cat /etc/default/grub

# 8. List available kernels in GRUB
grep menuentry /boot/grub/grub.cfg 2>/dev/null | head -10 || \
grep menuentry /boot/grub2/grub.cfg 2>/dev/null | head -10

# 9. Check for boot errors
journalctl -b 0 -p err --no-pager | head -30

# 10. See how many boots have been recorded
journalctl --list-boots

GRUB Password Protection

To prevent unauthorized users from editing GRUB entries (and bypassing authentication with init=/bin/bash):

# Generate a GRUB password hash
grub-mkpasswd-pbkdf2
# Enter and confirm a password
# Output: grub.pbkdf2.sha512.10000.HASH...

# Add to /etc/grub.d/40_custom
sudo tee -a /etc/grub.d/40_custom << 'EOF'
set superusers="admin"
password_pbkdf2 admin grub.pbkdf2.sha512.10000.YOUR_HASH_HERE
EOF

# Regenerate GRUB config
sudo update-grub

Now editing GRUB entries at boot will require the GRUB password.

Debug This: System Drops to Emergency Shell

After a reboot, the system drops to:

You are in emergency mode. After logging in, type "journalctl -xb" to view
system logs, "systemctl reboot" to reboot, "systemctl default" to try again
to boot into default mode.
Give root password for maintenance:

Diagnosis steps:

# 1. Enter root password (if set)

# 2. Check the journal for the cause
journalctl -xb --no-pager | grep -E "Failed|Error|error" | head -20

# 3. Common cause #1: /etc/fstab error
# Look for filesystem mount failures
systemctl --failed
# If a mount unit failed:
cat /etc/fstab
# Fix the bad line

# 4. Common cause #2: Disk UUID changed
blkid
# Compare UUIDs to what is in /etc/fstab

# 5. Common cause #3: Filesystem needs fsck
# The error message often says which device
fsck /dev/sda2

# 6. Remount root as read-write to make changes
mount -o remount,rw /

# 7. Make your fix, then:
systemctl default
# Or simply:
reboot

The Complete Boot Timeline

Here is the full sequence with approximate times for a modern SSD-based system:

  T+0.0s   Power button pressed
  T+0.5s   UEFI POST completes
  T+1.0s   UEFI finds and loads GRUB
  T+1.5s   GRUB loads kernel + initramfs
  T+2.0s   Kernel decompresses and initializes
  T+3.0s   initramfs mounts real root filesystem
  T+3.5s   systemd starts as PID 1
  T+4.0s   systemd mounts filesystems from /etc/fstab
  T+5.0s   systemd starts udev (device manager)
  T+6.0s   Network services start
  T+8.0s   Login services start (sshd, getty)
  T+10.0s  System is fully booted
            (graphical target may take longer)

# Verify this timeline on your system
systemd-analyze
# Output: Startup finished in 1.8s (firmware) + 2.3s (loader) +
#         3.5s (kernel) + 8.7s (userspace) = 16.3s

What Just Happened?

+------------------------------------------------------------------+
|  Chapter 14 Recap: Boot Process Demystified                      |
|------------------------------------------------------------------|
|                                                                  |
|  - Boot sequence: Firmware -> GRUB -> Kernel -> systemd          |
|  - BIOS is legacy (MBR, 2TB limit); UEFI is modern (GPT).       |
|  - GRUB2 loads the kernel and initramfs into memory.             |
|  - Edit /etc/default/grub, then run update-grub.                |
|  - initramfs provides drivers needed to mount the root FS.       |
|  - The kernel starts systemd as PID 1.                           |
|  - systemd targets replaced SysVinit runlevels.                  |
|  - systemd-analyze shows boot timing and bottlenecks.            |
|  - Rescue mode: add systemd.unit=rescue.target to kernel line.   |
|  - Emergency recovery: init=/bin/bash bypasses all auth.         |
|  - GRUB passwords prevent unauthorized boot parameter edits.     |
|  - Always use 'nofail' for non-critical mounts in /etc/fstab.   |
|  - journalctl -b 0 -p err shows boot errors.                    |
|                                                                  |
+------------------------------------------------------------------+

Try This

Exercise 1: Boot Time Analysis

Run systemd-analyze blame | head -20 and identify the three slowest services. Research what each one does. Could any of them be disabled on a server that does not need a GUI?

Exercise 2: Kernel Command Line

Read /proc/cmdline. Look up each parameter. Try temporarily adding quiet (if not present) or removing it (if present) by editing the GRUB entry at boot time (press e, modify, Ctrl+X). Notice the difference in boot verbosity.

Exercise 3: initramfs Exploration

List the contents of your initramfs (lsinitramfs or lsinitrd). Find the filesystem driver modules included. Find the /init script. What does it do?

Exercise 4: Target Practice

Check your current default target (systemctl get-default). If it is graphical.target, switch to multi-user.target and reboot. Log in via the text console. Switch back and reboot again. Compare boot times between the two.

Bonus Challenge

Intentionally break your boot process in a VM (not on a real machine!) by adding a nonexistent UUID to /etc/fstab. Boot the VM and practice recovering:

1. Use GRUB edit mode to boot into rescue mode
2. Fix the /etc/fstab entry
3. Reboot and verify normal boot

Then, practice the GRUB recovery process:

1. From a live USB/ISO, mount the installed system's partitions
2. Chroot into the installed system
3. Reinstall and reconfigure GRUB
4. Reboot into the repaired system