Kernel Upgrades & Custom Kernels

Why This Matters

Your cloud provider just announced a new generation of instances with hardware features that require kernel 6.5 or newer, but your distribution ships kernel 5.15. Or a kernel vulnerability (like the Dirty Pipe exploit, CVE-2022-0847) is announced and you need to patch every server in your fleet within hours. Or you are building an embedded system where the default kernel includes thousands of drivers you will never need, and you need to strip it down to reduce boot time and attack surface.

The kernel is the most critical piece of software on your system. Every process, every file operation, every network packet passes through it. Understanding how to upgrade it safely, how to roll back when things go wrong, and how to build a custom kernel when you need one -- these are skills that separate system administrators from system engineers.

This chapter covers kernel versioning, upgrading via package manager, DKMS for third-party modules, building a custom kernel from source, configuring it with menuconfig, installing it alongside your existing kernel, updating GRUB, and rolling back when things go wrong.

Try This Right Now

Check your current kernel version and see what other kernels are available on your system:

# What kernel are you running right now?
$ uname -r
6.1.0-18-amd64

# What other kernels are installed?
# Debian/Ubuntu:
$ dpkg --list | grep linux-image
ii  linux-image-6.1.0-17-amd64   6.1.69-1    amd64  Linux 6.1 for 64-bit PCs
ii  linux-image-6.1.0-18-amd64   6.1.76-1    amd64  Linux 6.1 for 64-bit PCs

# Fedora/RHEL:
$ rpm -qa | grep kernel-core
kernel-core-6.6.9-200.fc39.x86_64
kernel-core-6.7.3-200.fc39.x86_64

# Arch:
$ pacman -Q linux
linux 6.7.4.arch1-1

Notice that most distributions keep at least two kernel versions installed. This is your safety net.

Kernel Versioning

Linux kernel versions follow a specific numbering scheme that tells you exactly what you are running.

6.1.76
^ ^ ^^
│ │ └── Patch version (bug fixes, security patches)
│ └──── Minor version (new features within the major series)
└────── Major version

Since kernel 3.0 (2011), the major number increments when the minor number gets "large enough" -- there is no technical significance to the boundary. Linus Torvalds bumps the major version when the minor version would get unwieldy (he bumped from 5.19 to 6.0, for example). Features can land in any release.

Kernel Release Types

┌───────────────────────────────────────────────────────────┐
│  Kernel Release Types                                      │
│                                                            │
│  Mainline     Latest development release from Linus        │
│               (e.g., 6.8-rc3)                              │
│               New features land here first.                │
│                                                            │
│  Stable       Bug-fix-only releases from mainline          │
│               (e.g., 6.7.5)                                │
│               New stable release every ~1 week.            │
│                                                            │
│  LTS          Long-term support, maintained for 2-6 years  │
│  (Longterm)   (e.g., 6.1.76, 5.15.148, 5.10.209)         │
│               Distribution kernels are typically based      │
│               on LTS releases.                             │
│                                                            │
│  Distro       Distribution-specific patched kernel         │
│               (e.g., 6.1.0-18-amd64 on Debian)            │
│               May include backported features and          │
│               distribution-specific patches.               │
└───────────────────────────────────────────────────────────┘

You can check the latest versions at kernel.org:

# Quick check from the command line
$ curl -s https://www.kernel.org/finger_banner
The latest stable version of the Linux kernel is:    6.7.5
The latest mainline version of the Linux kernel is:  6.8-rc5
The latest stable 6.6 version is:                    6.6.17
The latest longterm 6.1 version is:                  6.1.78
The latest longterm 5.15 version is:                 5.15.148
...

Think About It: Your distribution ships kernel 6.1.76. The latest mainline kernel is 6.8-rc5. Should you upgrade to 6.8 on a production server? What are the trade-offs?

Upgrading the Kernel via Package Manager

The safest way to upgrade a kernel is through your distribution's package manager. The distribution maintainers test the kernel, apply patches, and handle the GRUB configuration for you.

Debian/Ubuntu

# Check available kernel updates
$ apt list --upgradable 2>/dev/null | grep linux-image
linux-image-6.1.0-19-amd64/stable 6.1.82-1 amd64 [upgradable from: 6.1.76-1]

# Install the new kernel
$ sudo apt install linux-image-6.1.0-19-amd64

# The package post-install script automatically:
# - Installs the kernel image to /boot
# - Generates an initramfs (initial RAM filesystem)
# - Updates GRUB configuration
# - Does NOT reboot (you choose when to reboot)

# Verify it is installed
$ ls /boot/vmlinuz-*
/boot/vmlinuz-6.1.0-17-amd64
/boot/vmlinuz-6.1.0-18-amd64
/boot/vmlinuz-6.1.0-19-amd64

# Reboot to use the new kernel
$ sudo reboot

# After reboot, confirm
$ uname -r
6.1.0-19-amd64

Installing matching kernel headers (needed for building kernel modules):

$ sudo apt install linux-headers-$(uname -r)

Fedora/RHEL

# Check for kernel updates
$ dnf check-update kernel
kernel.x86_64    6.7.4-200.fc39    updates

# Install (DNF keeps multiple kernel versions by default)
$ sudo dnf upgrade kernel

# Fedora keeps the last 3 kernels by default
# Configured in /etc/dnf/dnf.conf:
#   installonly_limit=3

# Reboot
$ sudo reboot

# Confirm
$ uname -r
6.7.4-200.fc39.x86_64

Arch Linux

# Arch upgrades the kernel as part of a full system upgrade
$ sudo pacman -Syu

# This replaces the kernel (Arch does not keep old versions by default)
# Reboot to use the new kernel
$ sudo reboot

Distro Note: Arch Linux does not keep old kernel versions by default. If a kernel update breaks something, you need a rescue medium to roll back. Consider installing the linux-lts package as a fallback kernel.

Safety Warning: Never reboot a production server right after a kernel upgrade without a plan to roll back. Test on a staging system first. Make sure you can access the GRUB menu (or have console access) to select a previous kernel if the new one fails to boot.

Understanding /boot

The /boot directory contains everything needed to start the kernel:

$ ls -lh /boot/
-rw-r--r-- 1 root root 256K  config-6.1.0-18-amd64
-rw-r--r-- 1 root root  62M  initrd.img-6.1.0-18-amd64
-rw-r--r-- 1 root root 3.5M  System.map-6.1.0-18-amd64
-rw-r--r-- 1 root root 7.5M  vmlinuz-6.1.0-18-amd64
drwxr-xr-x 5 root root 4.0K  grub

File	Purpose
`vmlinuz-*`	The compressed kernel image (the actual kernel binary)
`initrd.img-` or `initramfs-`	The initial RAM disk -- a small filesystem loaded into RAM with drivers needed to mount the real root filesystem
`config-*`	The kernel configuration file (every option used to build this kernel)
`System.map-*`	Symbol table for the kernel (maps memory addresses to function names, useful for debugging)
`grub/`	GRUB bootloader configuration

GRUB: Selecting and Configuring Kernels

GRUB (GRand Unified Bootloader) is the bootloader on most Linux distributions. It presents a menu at boot time where you can select which kernel to boot.

Viewing GRUB Configuration

# The main GRUB config (auto-generated, do NOT edit directly)
$ cat /boot/grub/grub.cfg | grep menuentry
menuentry 'Debian GNU/Linux, with Linux 6.1.0-19-amd64' ...
menuentry 'Debian GNU/Linux, with Linux 6.1.0-18-amd64' ...
menuentry 'Debian GNU/Linux, with Linux 6.1.0-17-amd64' ...

Customizing GRUB

Edit /etc/default/grub for configuration, then regenerate:

$ cat /etc/default/grub
GRUB_DEFAULT=0                # Boot the first entry by default
GRUB_TIMEOUT=5                # Show the menu for 5 seconds
GRUB_CMDLINE_LINUX_DEFAULT="quiet"   # Default kernel parameters
GRUB_CMDLINE_LINUX=""                 # Kernel parameters for ALL entries

After editing, regenerate the GRUB configuration:

# Debian/Ubuntu
$ sudo update-grub

# Fedora/RHEL
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg

# Arch
$ sudo grub-mkconfig -o /boot/grub/grub.cfg

Kernel Parameters

Kernel parameters are passed on the command line at boot. They control kernel behavior.

Common parameters:

Parameter	Effect
`quiet`	Suppress most boot messages
`splash`	Show a graphical splash screen instead of text
`nomodeset`	Disable kernel mode-setting (useful for graphics issues)
`single` or `1`	Boot into single-user mode (recovery)
`init=/bin/bash`	Skip init, drop to a root shell (emergency recovery)
`mem=4G`	Limit usable RAM to 4 GB
`maxcpus=2`	Limit to 2 CPUs
`panic=10`	Reboot automatically 10 seconds after a kernel panic
`net.ifnames=0`	Use classic network interface names (eth0 instead of enp0s3)

Adding temporary kernel parameters (one-time at boot):

At the GRUB menu, press e to edit the selected entry
Find the line starting with linux (the kernel command line)
Add your parameter at the end of that line
Press Ctrl+X or F10 to boot

Adding permanent kernel parameters:

# Edit /etc/default/grub
$ sudo nano /etc/default/grub

# Add to the GRUB_CMDLINE_LINUX_DEFAULT line:
GRUB_CMDLINE_LINUX_DEFAULT="quiet panic=10"

# Regenerate GRUB config
$ sudo update-grub

Checking current kernel parameters:

$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-6.1.0-18-amd64 root=UUID=a1b2c3d4... ro quiet

Rolling Back to a Previous Kernel

When a kernel upgrade causes problems (drivers not loading, system not booting, performance regression), you need to roll back.

The simplest approach -- just select the old kernel at the GRUB menu:

Reboot the system
Hold Shift (BIOS) or Esc (UEFI) during boot to show the GRUB menu
Select "Advanced options"
Choose the previous kernel version
The system boots with the old kernel

Method 2: Change Default Kernel

To make the older kernel the default:

# List available kernels with their menu entries
# Debian/Ubuntu:
$ grep menuentry /boot/grub/grub.cfg

# Set the default to a specific menu entry
# (entries are numbered starting from 0, or use the full string)
$ sudo nano /etc/default/grub
GRUB_DEFAULT="Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-18-amd64"

$ sudo update-grub

Method 3: Remove the Problematic Kernel

# Debian/Ubuntu (make sure you are NOT running the kernel you are removing)
$ uname -r    # Verify you are on the OLD kernel
6.1.0-18-amd64

$ sudo apt remove linux-image-6.1.0-19-amd64
$ sudo update-grub

# Fedora/RHEL
$ sudo dnf remove kernel-core-6.7.4-200.fc39.x86_64

Safety Warning: Never remove the kernel you are currently running (uname -r). Always keep at least two kernel versions installed so you have a fallback.

Think About It: On a remote server where you cannot physically access the GRUB menu, how would you handle a kernel upgrade that might fail? (Hint: think about kexec, remote console access, or cloud provider console features.)

DKMS: Dynamic Kernel Module Support

When you install third-party kernel modules (like VirtualBox guest additions, NVIDIA drivers, or ZFS), those modules are compiled against a specific kernel version. When you upgrade the kernel, those modules need to be recompiled.

DKMS automates this process. It automatically rebuilds registered modules whenever a new kernel is installed.

┌───────────────────────────────────────────────────────────┐
│  Without DKMS:                                             │
│                                                            │
│  1. Install kernel 6.1.0-18                                │
│  2. Compile VirtualBox modules for 6.1.0-18  ✓             │
│  3. Upgrade to kernel 6.1.0-19                             │
│  4. VirtualBox modules are MISSING for 6.1.0-19  ✗         │
│  5. Manually recompile modules  (annoying)                 │
│                                                            │
├───────────────────────────────────────────────────────────┤
│  With DKMS:                                                │
│                                                            │
│  1. Install kernel 6.1.0-18                                │
│  2. Install VirtualBox modules via DKMS  ✓                 │
│  3. Upgrade to kernel 6.1.0-19                             │
│  4. DKMS automatically rebuilds modules for 6.1.0-19  ✓    │
│  5. Everything just works                                  │
└───────────────────────────────────────────────────────────┘

Using DKMS

# Install DKMS
$ sudo apt install dkms    # or dnf/pacman

# Check registered DKMS modules
$ dkms status
virtualbox-guest/7.0.14, 6.1.0-18-amd64, x86_64: installed
zfs/2.2.2, 6.1.0-18-amd64, x86_64: installed

# Manually rebuild all modules for the current kernel
$ sudo dkms autoinstall -k $(uname -r)

# Build and install a specific module
$ sudo dkms build -m virtualbox-guest -v 7.0.14 -k 6.1.0-19-amd64
$ sudo dkms install -m virtualbox-guest -v 7.0.14 -k 6.1.0-19-amd64

DKMS modules are stored in /usr/src/:

$ ls /usr/src/
linux-headers-6.1.0-18-amd64/
virtualbox-guest-7.0.14/
zfs-2.2.2/

Each directory contains the module source code and a dkms.conf file that tells DKMS how to build it.

Why Build a Custom Kernel?

The distribution kernel works for most cases. You should only build a custom kernel when you have a specific reason:

Hardware support: You need a newer kernel for new hardware, but your distro has not released one
Performance tuning: You want to enable/disable specific scheduler options, change the tick rate, or enable real-time preemption
Minimalism: You want a kernel with only the drivers you need (smaller, faster boot, smaller attack surface)
Patching: You need to apply a patch that has not been merged upstream or into your distro's kernel
Learning: Understanding the kernel build process deepens your understanding of Linux

┌───────────────────────────────────────────────────────────┐
│  DO build a custom kernel when:                            │
│  - You need hardware support not in your distro kernel     │
│  - You need specific performance tuning options             │
│  - You are building an embedded or specialized system       │
│  - You need to apply custom patches                        │
│  - You want to learn                                       │
│                                                            │
│  DO NOT build a custom kernel when:                        │
│  - Your distro kernel works fine                           │
│  - You just want "the latest" (wait for your distro)       │
│  - You manage many servers (use distro kernels + DKMS)     │
│  - You cannot commit to maintaining it                     │
└───────────────────────────────────────────────────────────┘

Building a Custom Kernel

This is a complete walkthrough of building a Linux kernel from source. We will compile and install it alongside your existing kernel so you always have a fallback.

Step 1: Install Prerequisites

# Debian/Ubuntu
$ sudo apt install build-essential libncurses-dev bison flex libssl-dev \
    libelf-dev bc dwarves

# Fedora/RHEL
$ sudo dnf install gcc make ncurses-devel bison flex elfutils-libelf-devel \
    openssl-devel bc dwarves perl

# Arch
$ sudo pacman -S base-devel xmlto kmod inetutils bc libelf git cpio perl

Step 2: Download the Kernel Source

$ cd /usr/src
$ sudo wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.7.5.tar.xz

# Verify the download (always verify for security)
$ sudo wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.7.5.tar.sign
$ sudo unxz linux-6.7.5.tar.xz
# Import the kernel signing key and verify (optional but recommended)

# Extract
$ sudo tar xf linux-6.7.5.tar
$ cd linux-6.7.5

Step 3: Configure the Kernel

The kernel has thousands of configuration options. You need to decide which features, drivers, and modules to include.

Option A: Start from your current kernel's config (recommended for first build).

# Copy your running kernel's configuration
$ cp /boot/config-$(uname -r) .config

# Update it for the new kernel version
# (answers new options with their default values)
$ make olddefconfig

Option B: Use menuconfig for interactive configuration.

$ make menuconfig

This launches a text-based menu interface:

┌──────────── Linux/x86 6.7.5 Kernel Configuration ─────────────┐
│  Arrow keys navigate the menu. <Enter> selects submenus -->.   │
│  Highlighted letters are hotkeys. Pressing <Y> includes, <N>   │
│  excludes, <M> builds as a module. <Esc><Esc> to go back.      │
│                                                                 │
│       General setup  --->                                       │
│       Processor type and features  --->                         │
│       Power management and ACPI options  --->                   │
│       Bus options (PCI etc.)  --->                              │
│       Firmware Drivers  --->                                    │
│       [*] Networking support  --->                              │
│       Device Drivers  --->                                      │
│       File systems  --->                                        │
│       Security options  --->                                    │
│       -*- Cryptographic API  --->                               │
│       Library routines  --->                                    │
│       Kernel hacking  --->                                      │
│                                                                 │
│         <Select>   < Exit >   < Help >   < Save >   < Load >   │
└─────────────────────────────────────────────────────────────────┘

Key concepts in menuconfig:

[*] = Built into the kernel (always loaded)
[M] = Built as a loadable module (loaded on demand)
[ ] = Not included
Press Y to include, N to exclude, M for module
Press / to search for a specific option
Press ? on any option for help text

Important configuration areas:

General setup --->
    Local version: -custom1          # Appended to the version string
    (This becomes part of `uname -r`, e.g., 6.7.5-custom1)

Processor type and features --->
    Processor family: (Generic-x86-64)
    # Change to match your specific CPU for slight performance gains

Device Drivers --->
    # Enable only the drivers you need
    # Common: SATA/SCSI, network drivers, USB, input devices
    # Disable drivers for hardware you don't have to reduce kernel size

File systems --->
    # Make sure your root filesystem type is built-in [*], not module [M]
    # ext4, XFS, or Btrfs depending on your setup

Networking support --->
    Networking options --->
        # TCP/IP, Netfilter (iptables/nftables) for firewalls

Safety Warning: If you are building a kernel for a system that boots from a specific filesystem or block device, make sure the relevant drivers are built-in ([*]), not as modules ([M]). If they are modules, the kernel cannot read the disk to find the modules. Alternatively, ensure the initramfs includes the necessary modules.

Option C: Use localmodconfig for a minimal kernel based on currently loaded modules.

# This configures only the modules currently loaded on your running system
$ make localmodconfig

This produces a much smaller kernel but will only include drivers for hardware currently connected. It is excellent for dedicated servers where the hardware is fixed.

Step 4: Compile the Kernel

# Compile the kernel, modules, and device tree blobs
$ make -j$(nproc)

This will take 15-90 minutes depending on your hardware and configuration. On a modern multi-core system, make -j$(nproc) uses all available CPU cores.

You will see output like:

  CALL    scripts/checksyscalls.sh
  CC      init/main.o
  CC      init/version.o
  ...
  LD      vmlinux
  SORTTAB vmlinux
  SYSMAP  System.map
  OBJCOPY arch/x86/boot/compressed/vmlinux.bin
  ...
  BUILD   arch/x86/boot/bzImage
Setup is 17500 bytes (padded to 17920 bytes).
System is 12148 kB
Kernel: arch/x86/boot/bzImage is ready  (#1)

When you see "bzImage is ready," the kernel compiled successfully.

Step 5: Install Modules

$ sudo make modules_install

This copies compiled modules to /lib/modules/6.7.5-custom1/.

$ ls /lib/modules/
6.1.0-18-amd64/     # Your old kernel's modules
6.7.5-custom1/      # Your new kernel's modules

Step 6: Install the Kernel

$ sudo make install

This copies the kernel image and configuration to /boot/ and runs the distribution's kernel installation hooks (which typically update GRUB and generate an initramfs).

On some distributions, you may need to generate the initramfs manually:

# Debian/Ubuntu
$ sudo update-initramfs -c -k 6.7.5-custom1

# Fedora/RHEL
$ sudo dracut --force /boot/initramfs-6.7.5-custom1.img 6.7.5-custom1

Step 7: Update GRUB

# Debian/Ubuntu
$ sudo update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.7.5-custom1
Found initrd image: /boot/initrd.img-6.7.5-custom1
Found linux image: /boot/vmlinuz-6.1.0-18-amd64
Found initrd image: /boot/initrd.img-6.1.0-18-amd64
done

# Fedora/RHEL
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Step 8: Reboot and Test

$ sudo reboot

At the GRUB menu, you should see your custom kernel as an option. Select it and boot.

After booting:

$ uname -r
6.7.5-custom1

$ uname -a
Linux myhost 6.7.5-custom1 #1 SMP PREEMPT_DYNAMIC Sat Feb 10 14:22:33 UTC 2024 x86_64 GNU/Linux

If the custom kernel fails to boot, select your previous kernel at the GRUB menu.

Hands-On: Complete Custom Kernel Build Summary

Here is the entire process in a concise reference:

# 1. Install prerequisites
$ sudo apt install build-essential libncurses-dev bison flex libssl-dev \
    libelf-dev bc dwarves

# 2. Download and extract kernel source
$ cd /usr/src
$ sudo wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.7.5.tar.xz
$ sudo tar xJf linux-6.7.5.tar.xz
$ cd linux-6.7.5

# 3. Configure
$ cp /boot/config-$(uname -r) .config
$ make olddefconfig          # or: make menuconfig

# 4. Compile
$ make -j$(nproc)

# 5. Install modules
$ sudo make modules_install

# 6. Install kernel
$ sudo make install

# 7. Update bootloader
$ sudo update-grub           # Debian/Ubuntu
# or: sudo grub2-mkconfig -o /boot/grub2/grub.cfg  (Fedora/RHEL)

# 8. Reboot
$ sudo reboot

# 9. Verify
$ uname -r

Removing a Custom Kernel

If you no longer need a custom kernel:

# Remove the kernel files from /boot
$ sudo rm /boot/vmlinuz-6.7.5-custom1
$ sudo rm /boot/initrd.img-6.7.5-custom1
$ sudo rm /boot/config-6.7.5-custom1
$ sudo rm /boot/System.map-6.7.5-custom1

# Remove the modules
$ sudo rm -rf /lib/modules/6.7.5-custom1/

# Update GRUB
$ sudo update-grub

Kernel Modules

Kernel modules are pieces of kernel code that can be loaded and unloaded at runtime without rebooting. Most device drivers are modules.

Working with Modules

# List currently loaded modules
$ lsmod
Module                  Size  Used by
nf_tables             303104  0
xt_conntrack            16384  1
nf_conntrack          176128  1 xt_conntrack
btrfs                1654784  1
...

# Show info about a module
$ modinfo e1000e
filename:       /lib/modules/6.1.0-18-amd64/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
version:        3.8.7-NAPI
license:        GPL v2
description:    Intel(R) PRO/1000 Network Driver
...

# Load a module
$ sudo modprobe e1000e

# Unload a module (only if not in use)
$ sudo modprobe -r e1000e

# Load a module with parameters
$ sudo modprobe bonding mode=4 miimon=100

Persisting Module Options

To load a module at boot:

# Create a file in /etc/modules-load.d/
$ echo "bonding" | sudo tee /etc/modules-load.d/bonding.conf

To set module parameters:

# Create a file in /etc/modprobe.d/
$ echo "options bonding mode=4 miimon=100" | sudo tee /etc/modprobe.d/bonding.conf

To blacklist a module (prevent it from loading):

$ echo "blacklist nouveau" | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
$ sudo update-initramfs -u    # Rebuild initramfs

Debug This

After building and installing a custom kernel, a colleague reports that the system boots but networking does not work. WiFi and Ethernet are both dead. ip link shows only the loopback interface.

$ uname -r
6.7.5-custom1

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

What happened?

When they configured the kernel with make menuconfig, they likely:

Disabled the network drivers needed for their hardware, OR
Built the drivers as modules ([M]) but the modules were not loaded, OR
Used make localmodconfig on a system where the network interface was not active at the time

How to diagnose:

# Step 1: What network hardware does the system have?
$ lspci | grep -i net
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I218-LM
03:00.0 Network controller: Intel Corporation Wireless 8265

# Step 2: What driver does this hardware need?
# (Check from a working kernel)
$ lspci -k | grep -A3 "Ethernet"
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I218-LM
	Kernel driver in use: e1000e
	Kernel modules: e1000e

# Step 3: Is the module available in the custom kernel?
$ find /lib/modules/6.7.5-custom1/ -name "e1000e*"
# (no output -- the module was not compiled)

# Step 4: Check the kernel config
$ grep E1000E /boot/config-6.7.5-custom1
# CONFIG_E1000E is not set

The fix:

$ cd /usr/src/linux-6.7.5
$ make menuconfig
# Navigate to: Device Drivers -> Network device support -> Ethernet driver support
# Enable Intel(R) PRO/1000 PCI-Express (e1000e) as [M] or [*]
$ make -j$(nproc)
$ sudo make modules_install
$ sudo make install
$ sudo reboot

Kernel Upgrade Checklist for Production

Before upgrading a kernel on a production server, follow this checklist:

┌──────────────────────────────────────────────────────────────┐
│  Pre-Upgrade Checklist                                        │
│                                                               │
│  [ ] Test the new kernel on a staging/test system first       │
│  [ ] Verify all DKMS modules build successfully               │
│  [ ] Verify all critical applications work on the new kernel  │
│  [ ] Ensure at least one previous kernel is installed          │
│  [ ] Ensure you have console access (IPMI, cloud console)     │
│      in case the new kernel fails to boot                     │
│  [ ] Schedule a maintenance window                            │
│  [ ] Notify stakeholders                                      │
│  [ ] Take a snapshot/backup if possible                       │
│                                                               │
│  Post-Upgrade Verification                                    │
│                                                               │
│  [ ] uname -r shows the expected new version                 │
│  [ ] dmesg shows no critical errors                          │
│  [ ] All network interfaces are up                           │
│  [ ] All filesystems are mounted                             │
│  [ ] All critical services are running                       │
│  [ ] DKMS modules loaded (dkms status shows "installed")     │
│  [ ] Application smoke tests pass                            │
│                                                               │
└──────────────────────────────────────────────────────────────┘

What Just Happened?

┌──────────────────────────────────────────────────────────────┐
│                                                               │
│  In this chapter, you learned:                                │
│                                                               │
│  - Kernel versions follow major.minor.patch. LTS kernels      │
│    are maintained for years. Distribution kernels are based   │
│    on LTS releases with additional patches.                   │
│                                                               │
│  - Upgrading via package manager is the safest approach.      │
│    apt/dnf handle kernel image, initramfs, and GRUB.         │
│                                                               │
│  - GRUB allows selecting between installed kernels.           │
│    Kernel parameters are set in /etc/default/grub.           │
│                                                               │
│  - DKMS automatically rebuilds third-party kernel modules     │
│    when a new kernel is installed.                            │
│                                                               │
│  - Building a custom kernel:                                  │
│    1. Download source from kernel.org                        │
│    2. Configure with make menuconfig (or olddefconfig)       │
│    3. Compile with make -j$(nproc)                           │
│    4. Install with make modules_install && make install       │
│    5. Update GRUB                                            │
│                                                               │
│  - Always keep a previous working kernel as a fallback.       │
│                                                               │
│  - Kernel modules can be loaded, unloaded, configured,        │
│    and blacklisted at runtime.                                │
│                                                               │
│  - Always test kernel upgrades on staging before production.  │
│                                                               │
└──────────────────────────────────────────────────────────────┘

Try This

Exercises

Kernel inventory: Run uname -r and look up your kernel version on kernel.org. Is it an LTS kernel? When will it reach end of life? Is there a newer patch version available?
GRUB exploration: Read /boot/grub/grub.cfg (do not edit it). Identify the menu entries for each installed kernel. Find the kernel command line parameters for each entry.
Module exploration: Run lsmod and pick three modules you do not recognize. Use modinfo to find out what they do. Then check if your hardware actually uses them with lspci -k.
Kernel config comparison: Compare the config files for two installed kernels: diff /boot/config-VERSION1 /boot/config-VERSION2. What changed between versions?
DKMS status: If you have DKMS installed, run dkms status to see what modules are managed. If not, install dkms and examine its directory structure in /usr/src/.

Bonus Challenge

On a virtual machine or test system (never production), download the latest stable kernel from kernel.org and build it from source. Use make localmodconfig to create a minimal configuration based on your currently loaded modules. Measure the compile time, the resulting kernel image size, and the number of modules compared to your distribution kernel. Boot the custom kernel and verify that all hardware works.

What's Next

With a solid understanding of packages, compilation, shared libraries, and the kernel, you have mastered the software stack from userspace down to the kernel itself. Part XIV takes you into virtualization and containers -- how Linux creates isolated environments that behave like separate machines, all on the same kernel.

Linux Book: From First Boot to Production