Embedded Linux
Why This Matters
Look around you. Your Wi-Fi router runs Linux. Your smart TV runs Linux. The infotainment system in your car almost certainly runs Linux. Medical devices, industrial robots, drones, security cameras, smart home hubs, point-of-sale terminals, digital billboards -- Linux is everywhere, and most of it is not running on anything resembling a traditional server.
This is embedded Linux: Linux running on specialized hardware with limited resources, often without a screen or keyboard, performing a specific set of tasks reliably for years without human intervention.
Understanding embedded Linux matters for several reasons. If you work in IoT, automotive, or industrial automation, you will encounter it daily. Even if you are a server administrator, understanding how Linux works under extreme resource constraints deepens your knowledge of the operating system fundamentals. And if you own a Raspberry Pi, you are already running embedded Linux -- even if you did not know it.
Try This Right Now
If you have any Linux system, you can see how your current system compares to an embedded one:
# How much RAM do you have?
$ free -m | grep Mem
Mem: 7964 2134 1204 125 4625 5425
# An embedded device might have 64MB or even 16MB of RAM.
# How big is your root filesystem?
$ df -h / | tail -1
/dev/sda1 50G 28G 20G 59% /
# An embedded device might have 256MB of flash storage total.
# How many processes are running?
$ ps aux | wc -l
247
# An embedded device might run 15-20 processes.
# How big is your kernel?
$ ls -lh /boot/vmlinuz-$(uname -r)
-rw-r--r-- 1 root root 11M Jan 15 10:00 /boot/vmlinuz-6.1.0-18-amd64
# An embedded kernel can be stripped down to 1-2MB.
The difference is dramatic. Embedded Linux is about doing more with less.
What Is Embedded Linux?
An embedded system is a computer designed to perform a dedicated function within a larger system. Unlike a general-purpose computer, an embedded device runs a fixed set of software tailored to its specific purpose.
┌──────────────────────────────────────────────────────────────┐
│ GENERAL-PURPOSE vs EMBEDDED LINUX │
│ │
│ GENERAL-PURPOSE (Desktop/Server) │
│ ───────────────────────────────── │
│ • Full distro (Ubuntu, Fedora) │
│ • Thousands of packages available │
│ • User installs and runs any software │
│ • 4-128 GB RAM, 100+ GB storage │
│ • Keyboard, display, network │
│ • Boots in 15-60 seconds │
│ • Updated regularly by user │
│ │
│ EMBEDDED │
│ ──────── │
│ • Minimal custom Linux build │
│ • Only needed software included │
│ • Runs a specific application │
│ • 16 MB - 1 GB RAM, 32 MB - 4 GB storage │
│ • Often headless (no display) │
│ • Boots in 1-5 seconds │
│ • Updated via firmware OTA or manual flash │
│ │
└──────────────────────────────────────────────────────────────┘
Where Embedded Linux Runs
- Networking: Routers, switches, firewalls, access points (OpenWrt)
- Consumer electronics: Smart TVs, set-top boxes, streaming devices
- Automotive: Infotainment systems, telematics, dashcams (Automotive Grade Linux)
- Industrial: PLCs, HMIs, factory robots, CNC machines
- Medical: Patient monitors, imaging devices, infusion pumps
- IoT: Smart home hubs, sensors, environmental monitors
- Aerospace: Drones, satellite subsystems, ground control stations
- Retail: Point-of-sale terminals, digital signage, vending machines
The Embedded Linux Stack
An embedded Linux system has the same fundamental components as a desktop system, but each is minimized and customized:
┌──────────────────────────────────────────────────────────────┐
│ EMBEDDED LINUX STACK │
│ │
│ ┌────────────────────────────────────────┐ │
│ │ Application │ │
│ │ (your specific software) │ │
│ ├────────────────────────────────────────┤ │
│ │ Root Filesystem │ │
│ │ (BusyBox, libraries, configs) │ │
│ ├────────────────────────────────────────┤ │
│ │ Linux Kernel │ │
│ │ (custom-configured, stripped down) │ │
│ ├────────────────────────────────────────┤ │
│ │ Bootloader │ │
│ │ (U-Boot, barebox) │ │
│ ├────────────────────────────────────────┤ │
│ │ Hardware │ │
│ │ (SoC: CPU + RAM + peripherals) │ │
│ └────────────────────────────────────────┘ │
│ │
│ Total image size: 8 MB - 500 MB (vs. 2+ GB for desktop) │
│ │
└──────────────────────────────────────────────────────────────┘
The Bootloader: U-Boot
Most embedded Linux devices use U-Boot (Universal Boot Loader) instead of GRUB. U-Boot supports dozens of CPU architectures and is specifically designed for embedded systems.
Boot sequence on embedded device:
1. Hardware powers on → CPU loads bootloader from flash
2. U-Boot initializes RAM, sets up hardware
3. U-Boot loads the Linux kernel and device tree
4. Kernel initializes, mounts root filesystem
5. Init system starts the application
The Kernel: Custom-Configured
An embedded kernel is stripped to the bare minimum:
# A desktop kernel has thousands of modules
$ find /lib/modules/$(uname -r) -name "*.ko" | wc -l
5847
# An embedded kernel might compile everything needed directly in,
# with zero loadable modules, for a smaller and faster kernel.
Embedded engineers use make menuconfig to carefully select only the drivers and features needed for their specific hardware.
Cross-Compilation
Most embedded devices use ARM, MIPS, or RISC-V processors -- not x86. You cannot compile software directly on a device with 64MB of RAM. Instead, you cross-compile: build on your powerful x86 workstation, targeting the embedded architecture.
┌──────────────────────────────────────────────────────────────┐
│ CROSS-COMPILATION │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Build Machine │ │ Target Device │ │
│ │ (x86_64) │ │ (ARM) │ │
│ │ │ │ │ │
│ │ Cross-compiler │ ──────► │ Runs the │ │
│ │ arm-linux-gcc │ deploy │ compiled │ │
│ │ │ │ binary │ │
│ └─────────────────┘ └─────────────────┘ │
│ │
│ The compiler runs on x86 but produces ARM binaries. │
│ │
└──────────────────────────────────────────────────────────────┘
Installing a Cross-Compiler
# On Debian/Ubuntu: install ARM cross-compilation toolchain
$ sudo apt install -y gcc-aarch64-linux-gnu
# Verify
$ aarch64-linux-gnu-gcc --version
Cross-Compiling a Simple Program
$ cat > hello.c << 'EOF'
#include <stdio.h>
int main() {
printf("Hello from embedded Linux!\n");
return 0;
}
EOF
# Compile for ARM64
$ aarch64-linux-gnu-gcc -o hello-arm64 hello.c
# Check the binary -- it is ARM, not x86
$ file hello-arm64
hello-arm64: ELF 64-bit LSB executable, ARM aarch64, ...
# Compare with native compilation
$ gcc -o hello-x86 hello.c
$ file hello-x86
hello-x86: ELF 64-bit LSB executable, x86-64, ...
You cannot run hello-arm64 on your x86 machine (unless you use QEMU emulation), but you can copy it to an ARM device and run it there.
Think About It: Why is cross-compilation necessary for embedded development? Could you install a compiler directly on the target device? What would be the trade-offs?
BusyBox: The Swiss Army Knife
On a desktop system, basic commands like ls, cp, cat, grep, mount, and sh are separate binaries, each several hundred kilobytes or more. On an embedded device with 32MB of storage, that is wasteful.
BusyBox combines hundreds of common Unix utilities into a single small binary:
# On a desktop, each command is a separate binary:
$ ls -l /bin/ls /bin/cp /bin/cat /bin/grep
-rwxr-xr-x 1 root root 142144 /bin/ls
-rwxr-xr-x 1 root root 153432 /bin/cp
-rwxr-xr-x 1 root root 43416 /bin/cat
-rwxr-xr-x 1 root root 219456 /bin/grep
# Total: ~550 KB for just 4 commands
# BusyBox provides 300+ commands in ~1 MB
How BusyBox Works
BusyBox is a single binary. Symbolic links point to it with different names. When you run ls, BusyBox checks what name it was called with and executes that command:
/bin/ls → /bin/busybox
/bin/cp → /bin/busybox
/bin/cat → /bin/busybox
/bin/grep → /bin/busybox
/bin/mount → /bin/busybox
/bin/sh → /bin/busybox
... (300+ more)
Trying BusyBox on Your System
# Install BusyBox
$ sudo apt install -y busybox # Debian/Ubuntu
$ sudo dnf install -y busybox # Fedora/RHEL
# See all included commands
$ busybox --list | head -20
# Use BusyBox versions of commands
$ busybox ls /
$ busybox df -h
$ busybox uname -a
# Check BusyBox binary size
$ ls -lh $(which busybox)
-rwxr-xr-x 1 root root 1.1M ... /bin/busybox
Distro Note: BusyBox is available on all major distributions for testing, but it is primarily used in embedded systems, Docker scratch/alpine images, and recovery environments. Alpine Linux uses BusyBox as its default userland.
Buildroot: Building an Embedded Linux System
Buildroot is a tool that automates building a complete embedded Linux system: cross-compiler, kernel, root filesystem, and bootloader -- all from source.
How Buildroot Works
┌──────────────────────────────────────────────────────────────┐
│ BUILDROOT WORKFLOW │
│ │
│ 1. Select target architecture (ARM, MIPS, x86, RISC-V) │
│ 2. Configure kernel, packages, filesystem format │
│ 3. Buildroot downloads source code for everything │
│ 4. Cross-compiles the entire system │
│ 5. Produces a ready-to-flash image │
│ │
│ ┌──────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ Config │──► │ Buildroot │──► │ Output images │ │
│ │ (.config)│ │ (make) │ │ - kernel │ │
│ │ │ │ │ │ - rootfs.ext4 │ │
│ │ │ │ Downloads, │ │ - sdcard.img │ │
│ │ │ │ compiles, │ │ │ │
│ │ │ │ assembles │ │ Flash to device│ │
│ └──────────┘ └──────────────┘ └────────────────┘ │
│ │
│ Build time: 15-60 minutes depending on packages selected │
│ Output size: 8 MB - 200 MB depending on configuration │
│ │
└──────────────────────────────────────────────────────────────┘
Quick Start with Buildroot
# Download Buildroot
$ wget https://buildroot.org/downloads/buildroot-2024.02.tar.gz
$ tar xzf buildroot-2024.02.tar.gz
$ cd buildroot-2024.02
# List available board configs
$ ls configs/ | grep raspberry
raspberrypi0_defconfig
raspberrypi3_64_defconfig
raspberrypi4_64_defconfig
# Configure for Raspberry Pi 4
$ make raspberrypi4_64_defconfig
# Customize (optional)
$ make menuconfig
# Build (this takes a while -- go get coffee)
$ make -j$(nproc)
# Output images are in output/images/
$ ls output/images/
sdcard.img rootfs.ext4 Image bcm2711-rpi-4-b.dtb
The sdcard.img file can be written directly to an SD card and booted on a Raspberry Pi 4. It contains a minimal Linux system you built from source.
Yocto / OpenEmbedded: Industrial-Grade Build System
For production embedded products, Yocto (built on OpenEmbedded) is the industry standard. It is more complex than Buildroot but offers greater flexibility, better dependency management, and support for commercial products.
┌──────────────────────────────────────────────────────────────┐
│ BUILDROOT vs YOCTO │
│ │
│ BUILDROOT YOCTO / OPENEMBEDDED │
│ ───────── ─────────────────── │
│ Simple, Makefile-based Complex, BitBake-based │
│ Good for small projects Industry standard │
│ Full rebuild on changes Incremental builds │
│ Smaller learning curve Steep learning curve │
│ Single config file Layer-based architecture │
│ Limited package management Full package management │
│ (RPM, DEB, IPK) │
│ │
│ Choose Buildroot for: Choose Yocto for: │
│ - Learning - Commercial products │
│ - Simple projects - Long-term maintenance │
│ - Quick prototyping - Large teams │
│ - BSP vendor support │
│ │
└──────────────────────────────────────────────────────────────┘
Yocto Concepts
- Recipe: Instructions for building a single package (like a Makefile on steroids)
- Layer: A collection of related recipes (meta-networking, meta-python, etc.)
- BitBake: The build engine that processes recipes
- Poky: The reference distribution that includes BitBake and core layers
- BSP Layer: Board Support Package -- hardware-specific recipes provided by chip vendors
# Clone Poky (Yocto reference distribution)
$ git clone git://git.yoctoproject.org/poky
$ cd poky
$ git checkout kirkstone # LTS release
# Initialize build environment
$ source oe-init-build-env
# Build a minimal image (takes 1-3 hours on first build)
$ bitbake core-image-minimal
Device Trees
Desktop PCs have BIOS/UEFI to describe hardware to the operating system. Embedded boards do not. Instead, they use device trees -- data structures that describe the hardware layout.
┌──────────────────────────────────────────────────────────────┐
│ DEVICE TREE CONCEPT │
│ │
│ Problem: The kernel needs to know what hardware exists. │
│ PCs have ACPI/UEFI. Embedded boards do not. │
│ │
│ Solution: A device tree (.dtb) file describes the hardware: │
│ - CPU type and speed │
│ - Memory address ranges │
│ - Peripheral locations (UART, SPI, I2C, GPIO) │
│ - Interrupt routing │
│ - Clock frequencies │
│ │
│ Boot: U-Boot loads kernel + device tree → kernel reads DTB │
│ and knows how to talk to all hardware │
│ │
└──────────────────────────────────────────────────────────────┘
A device tree source file (.dts) looks like this:
/ {
model = "My Custom Board";
compatible = "mycompany,myboard";
memory@80000000 {
device_type = "memory";
reg = <0x80000000 0x10000000>; /* 256MB at address 0x80000000 */
};
leds {
compatible = "gpio-leds";
status-led {
label = "status";
gpios = <&gpio1 15 0>; /* GPIO1 pin 15 */
};
};
serial@44e09000 {
compatible = "ti,omap3-uart";
reg = <0x44e09000 0x2000>;
interrupts = <72>;
};
};
The .dts source is compiled into a binary .dtb file that the bootloader passes to the kernel.
Boot Process Differences
The embedded boot process differs significantly from desktop/server Linux:
┌──────────────────────────────────────────────────────────────┐
│ DESKTOP/SERVER BOOT vs EMBEDDED BOOT │
│ │
│ BIOS/UEFI ROM bootloader │
│ │ │ │
│ ▼ ▼ │
│ GRUB (bootloader) U-Boot │
│ │ │ │
│ ▼ ▼ │
│ Kernel + initramfs Kernel + device tree │
│ │ │ │
│ ▼ ▼ │
│ systemd (full init) BusyBox init or custom │
│ │ │ │
│ ▼ ▼ │
│ 200+ services start 5-10 services start │
│ │ │ │
│ ▼ ▼ │
│ Login prompt (30-60s) Application ready (1-5s) │
│ │
└──────────────────────────────────────────────────────────────┘
Fast boot is critical in embedded systems. A car's infotainment system cannot take 30 seconds to boot. Techniques for fast boot include:
- Minimal kernel with only required drivers compiled in (no modules)
- No initramfs (mount root directly)
- Minimal init (skip systemd, use BusyBox init or a direct exec of the application)
- Kernel command-line tuning (
quiet,lpj=to skip calibration)
Resource Constraints
Embedded development is the art of working within tight limits:
┌──────────────────────────────────────────────────────────────┐
│ RESOURCE COMPARISON │
│ │
│ Resource Desktop/Server Embedded Device │
│ ───────── ────────────── ──────────────── │
│ RAM 8-256 GB 16 MB - 1 GB │
│ Storage 100 GB - 10 TB 32 MB - 4 GB (flash) │
│ CPU 2-128 cores 1-4 cores (low clock) │
│ Network 1-100 Gbps 10/100 Mbps or WiFi │
│ Power 200-2000 W 0.5-10 W │
│ Cooling Fans, liquid Passive (no fans) │
│ Display 1-4 monitors None, or small LCD │
│ Lifetime 3-5 years 5-15 years │
│ │
└──────────────────────────────────────────────────────────────┘
Managing Flash Storage
Flash memory wears out after a limited number of write cycles (typically 10,000-100,000). Embedded Linux must be careful about writes:
- Use read-only root filesystem where possible
- Mount
/tmpand/var/logas tmpfs (RAM-based) - Use wear-leveling filesystems like UBIFS, JFFS2, or F2FS
- Avoid excessive logging to flash
# Common embedded filesystem mount strategy:
# / → read-only squashfs or ext4 (ro)
# /tmp → tmpfs (in RAM, lost on reboot)
# /var/log → tmpfs or size-limited log partition
# /data → read-write partition for persistent data
Real-Time Linux: PREEMPT_RT
Some embedded applications need real-time guarantees: a robot arm must respond to sensor input within microseconds, or an airbag controller must fire within a strict deadline.
Standard Linux is not a real-time operating system. It optimizes for throughput, not guaranteed latency. The PREEMPT_RT patch set modifies the kernel to provide hard real-time capabilities:
- Makes nearly all kernel code preemptible
- Converts spinlocks to sleeping locks
- Provides priority inheritance to prevent priority inversion
- Reduces worst-case latency from milliseconds to microseconds
# Check if your kernel has PREEMPT_RT
$ uname -a | grep -i rt
# or
$ grep -i preempt /boot/config-$(uname -r)
CONFIG_PREEMPT_VOLUNTARY=y # Desktop default (not RT)
# vs
CONFIG_PREEMPT_RT=y # Real-time kernel
Think About It: Why would you not use a real-time kernel for everything? What are the trade-offs of PREEMPT_RT in terms of overall system throughput?
Raspberry Pi as a Learning Platform
The Raspberry Pi is the perfect platform for learning embedded Linux because it is cheap, widely available, well-documented, and powerful enough to run a full Linux distribution.
┌──────────────────────────────────────────────────────────────┐
│ RASPBERRY PI FOR EMBEDDED LEARNING │
│ │
│ Hardware: Raspberry Pi 4 / Pi 5 │
│ • ARM Cortex-A76 CPU (Pi 5) / Cortex-A72 (Pi 4) │
│ • 1-8 GB RAM │
│ • MicroSD for storage │
│ • GPIO pins for hardware interfacing │
│ • HDMI, USB, Ethernet, WiFi, Bluetooth │
│ │
│ Good for learning: │
│ • Cross-compilation │
│ • Building custom kernels │
│ • Buildroot / Yocto images │
│ • GPIO programming │
│ • Device tree overlays │
│ • Boot process customization │
│ • Real-time applications (with PREEMPT_RT) │
│ │
└──────────────────────────────────────────────────────────────┘
Hands-On: Raspberry Pi Exploration
If you have a Raspberry Pi running Raspberry Pi OS:
# Check the hardware
$ cat /proc/cpuinfo | grep -i "model name\|hardware\|revision"
# See device tree information
$ ls /proc/device-tree/
compatible model name serial-number ...
$ cat /proc/device-tree/model
Raspberry Pi 4 Model B Rev 1.4
# Check GPIO pins
$ cat /sys/kernel/debug/gpio
# See the boot config
$ cat /boot/config.txt
# Check temperature (thermal management is important for embedded)
$ vcgencmd measure_temp
temp=42.3'C
# Check CPU frequency (may throttle under thermal stress)
$ vcgencmd measure_clock arm
frequency(48)=1500000000
Hands-On: Build a Minimal Root Filesystem
You can understand embedded Linux rootfs structure by building a minimal one on your workstation:
# Create a minimal root filesystem structure
$ mkdir -p ~/embedded-rootfs/{bin,sbin,etc,proc,sys,dev,tmp,var/log}
# Copy BusyBox as the userland
$ cp $(which busybox) ~/embedded-rootfs/bin/
# Create symlinks for common commands
$ cd ~/embedded-rootfs/bin
$ for cmd in sh ls cat echo mount mkdir; do
ln -s busybox $cmd
done
$ cd ~
# Create a minimal init script
$ cat > ~/embedded-rootfs/etc/init.d/rcS << 'EOF'
#!/bin/sh
mount -t proc proc /proc
mount -t sysfs sysfs /sys
mount -t tmpfs tmpfs /tmp
echo "Embedded Linux booted!"
echo "Hostname: $(cat /proc/sys/kernel/hostname)"
echo "Uptime: $(cat /proc/uptime | cut -d' ' -f1) seconds"
EOF
$ chmod +x ~/embedded-rootfs/etc/init.d/rcS
# See how small it is
$ du -sh ~/embedded-rootfs/
2.4M /home/user/embedded-rootfs/
That 2.4MB directory contains a functional (if minimal) Linux userland. With a kernel and bootloader, this could boot on real hardware.
Debug This
An embedded device you are developing is experiencing these symptoms:
- The device boots normally the first 50-100 times
- After that, the boot starts failing with filesystem corruption errors
- The syslog shows thousands of writes per minute to
/var/log/syslog
What is going on, and how do you fix it?
Diagnosis: The flash storage is wearing out. The syslog daemon is writing excessively to the flash-based root filesystem, exceeding the write endurance of the flash chips.
Fixes:
- Mount
/var/logas tmpfs:tmpfs /var/log tmpfs size=10M 0 0in/etc/fstab - Reduce log verbosity or disable unnecessary logging
- Use a log rotation scheme with strict size limits
- Make the root filesystem read-only and use a separate wear-leveled partition for writes
- Use an appropriate flash filesystem (UBIFS) instead of ext4
What Just Happened?
┌──────────────────────────────────────────────────────────────┐
│ CHAPTER 72 RECAP │
│──────────────────────────────────────────────────────────────│
│ │
│ Embedded Linux = Linux on resource-constrained hardware │
│ running dedicated applications. │
│ │
│ Key concepts: │
│ • Cross-compilation: build on x86, run on ARM/MIPS │
│ • BusyBox: 300+ utilities in a 1MB binary │
│ • Device trees: describe hardware to the kernel │
│ • U-Boot: embedded bootloader (replaces GRUB) │
│ • Fast boot: 1-5 seconds (vs. 30-60 for desktop) │
│ │
│ Build systems: │
│ • Buildroot: simple, fast, good for learning │
│ • Yocto/OpenEmbedded: industry standard, complex │
│ │
│ Constraints: │
│ • Limited RAM, storage, CPU, power │
│ • Flash wear: minimize writes │
│ • Read-only root filesystems │
│ • PREEMPT_RT for real-time requirements │
│ │
│ Raspberry Pi is the best learning platform for │
│ embedded Linux development. │
│ │
└──────────────────────────────────────────────────────────────┘
Try This
Exercise 1: BusyBox Exploration
Install BusyBox on your system and compare the output of busybox ls -la / with /bin/ls -la /. Note any differences in output format or options.
Exercise 2: Cross-Compile a Program
Install the ARM cross-compiler, write a simple C program, and cross-compile it. Use file to verify it is an ARM binary. If you have a Raspberry Pi, copy it over and run it.
Exercise 3: Minimal Rootfs
Expand the minimal root filesystem from the hands-on section. Add:
- A
/etc/passwdfile with a root user - A
/etc/hostnamefile - An
inittabfor BusyBox init
Research how to package it into a cpio archive that could be used as an initramfs.
Exercise 4: Buildroot Configuration
Download Buildroot and run make menuconfig. Explore the options without building. Note how you can select:
- Target architecture
- Kernel version
- BusyBox configuration
- Additional packages
- Filesystem format (ext4, squashfs, cpio)
Bonus Challenge
If you have a Raspberry Pi, build a custom Buildroot image for it. Include only: BusyBox, an SSH server (dropbear -- a lightweight SSH implementation), and a simple web server (busybox httpd). Flash it to an SD card and boot it. Your goal: a Linux system that boots in under 5 seconds, runs SSH and HTTP, and uses less than 32MB of storage.