Bash In Depth

Why This Matters

You already know how to type commands into a terminal. But have you ever been confused by why echo $HOME prints your home directory while echo '$HOME' prints the literal text $HOME? Have you wondered why rm * deletes all files but rm "*" tries to delete a file literally named *? Or why echo {1..10} prints 1 2 3 4 5 6 7 8 9 10?

These behaviors are not random. Bash processes every command line through a precise sequence of expansions before executing it. Understanding this sequence is what separates someone who uses the shell from someone who truly controls it.

This chapter takes you deep into how Bash works: the expansion order, quoting rules, variables, arrays, and special parameters. Master these, and you will write commands and scripts that do exactly what you intend, every time.

Try This Right Now

Run each of these and observe the differences carefully:

# Brace expansion
echo {a,b,c}-{1,2}

# Tilde expansion
echo ~
echo ~root

# Parameter expansion
name="Linux"
echo "Hello, $name"
echo "Hello, ${name}!"
echo "Length: ${#name}"

# Command substitution
echo "Today is $(date +%A)"

# Arithmetic expansion
echo "2 + 3 = $((2 + 3))"

# Globbing
echo /etc/*.conf | tr ' ' '\n' | head -5

Now observe how quoting changes things:

echo $HOME         # Expanded
echo "$HOME"       # Expanded
echo '$HOME'       # NOT expanded -- literal text
echo \$HOME        # NOT expanded -- escaped

The Shell Expansion Order

When you type a command and press Enter, Bash does not simply pass your text to the program. It processes it through a specific sequence of expansions, in this exact order:

+-------------------------------------------------------------------+
|  1. Brace Expansion        {a,b,c}  {1..5}                       |
|  2. Tilde Expansion        ~  ~user                               |
|  3. Parameter Expansion    $var  ${var}  ${var:-default}          |
|  4. Command Substitution   $(cmd)  `cmd`                          |
|  5. Arithmetic Expansion   $((expr))                              |
|  6. Word Splitting         (on unquoted results of 3, 4, 5)      |
|  7. Pathname Expansion     *  ?  [abc]  (globbing)                |
+-------------------------------------------------------------------+

Each step operates on the output of the previous step. This ordering matters because it determines what you can combine and what you cannot.

Step 1: Brace Expansion

Braces generate multiple strings. This happens first, before any variable expansion:

# Comma-separated list
echo {cat,dog,fish}
# cat dog fish

# Sequence
echo {1..5}
# 1 2 3 4 5

# Sequence with step
echo {0..20..5}
# 0 5 10 15 20

# Letter sequence
echo {a..f}
# a b c d e f

# Combinations (cartesian product)
echo {web,db}-{01,02}
# web-01 web-02 db-01 db-02

# Practical: create multiple directories
mkdir -p project/{src,tests,docs}/{v1,v2}
# Creates: project/src/v1  project/src/v2  project/tests/v1 ...

# Practical: backup a file
cp config.yml{,.bak}
# Equivalent to: cp config.yml config.yml.bak

Key rule: Brace expansion happens before variable expansion. This means you cannot use a variable inside braces for sequence generation:

n=5
echo {1..$n}
# Output: {1..5}  <-- Not expanded! Braces happen first, $n isn't resolved yet

Step 2: Tilde Expansion

The tilde ~ expands to home directories:

echo ~                # /home/yourusername
echo ~root            # /root
echo ~nobody          # /nonexistent (or wherever nobody's home is)
echo ~/Documents      # /home/yourusername/Documents

This only works at the beginning of a word. A tilde in the middle of text is just a literal tilde.

Step 3: Parameter Expansion

This is where variables get replaced with their values. Bash offers far more than just $var:

name="hello world"

# Basic expansion
echo $name          # hello world
echo ${name}        # hello world (braces clarify boundaries)
echo "${name}ish"   # hello worldish

# String length
echo ${#name}       # 11

# Default values
echo ${unset_var:-default}    # default (use default if unset/empty)
echo ${unset_var:=default}    # default (AND assign the default)

# Substring extraction
str="Hello, World!"
echo ${str:7}       # World!
echo ${str:7:5}     # World

# Pattern removal
path="/home/user/documents/file.tar.gz"
echo ${path##*/}    # file.tar.gz  (remove longest prefix matching */)
echo ${path#*/}     # home/user/documents/file.tar.gz  (remove shortest prefix)
echo ${path%%.*}    # /home/user/documents/file  (remove longest suffix matching .*)
echo ${path%.*}     # /home/user/documents/file.tar  (remove shortest suffix)

# Substitution
echo ${path/user/admin}      # /home/admin/documents/file.tar.gz  (first match)
echo ${path//o/0}            # /h0me/user/d0cuments/file.tar.gz  (all matches)

# Case modification (Bash 4+)
greeting="hello world"
echo ${greeting^}    # Hello world  (capitalize first letter)
echo ${greeting^^}   # HELLO WORLD  (capitalize all)
upper="HELLO"
echo ${upper,}       # hELLO  (lowercase first letter)
echo ${upper,,}      # hello  (lowercase all)

Think About It: Given the file path /var/log/nginx/access.log, how would you extract just the filename access.log using parameter expansion? What about just the extension log?

Step 4: Command Substitution

Replace a command with its output:

# Modern syntax (preferred)
echo "Today is $(date +%Y-%m-%d)"

# Old syntax (backticks -- avoid for readability)
echo "Today is `date +%Y-%m-%d`"

# Nested command substitution (much cleaner with $() than backticks)
echo "Config dir: $(dirname $(readlink -f /etc/resolv.conf))"

# Assign to variable
file_count=$(ls /etc/*.conf 2>/dev/null | wc -l)
echo "Found $file_count conf files"

Always prefer $(...) over backticks. Backticks are harder to read and cannot nest cleanly.

Step 5: Arithmetic Expansion

Perform integer math directly:

echo $((2 + 3))         # 5
echo $((10 / 3))        # 3  (integer division!)
echo $((10 % 3))        # 1  (modulo)
echo $((2 ** 10))       # 1024  (exponentiation)

x=10
echo $((x + 5))         # 15  (no $ needed inside $(()))
echo $((x++))           # 10  (post-increment, x is now 11)
echo $x                 # 11

# Comparison (returns 1 for true, 0 for false)
echo $((5 > 3))         # 1
echo $((5 < 3))         # 0

WARNING: Bash arithmetic is integer only. $((10 / 3)) gives 3, not 3.333. For floating point, use bc or awk.

Step 6: Word Splitting

After parameter expansion, command substitution, and arithmetic expansion, Bash splits the results into separate words based on the IFS (Internal Field Separator) variable.

# Default IFS is space, tab, newline
files="file1.txt file2.txt file3.txt"
for f in $files; do        # Word splitting splits into three words
    echo "Processing: $f"
done

# This is why quoting is critical:
filename="my important file.txt"
touch "$filename"           # Creates ONE file: "my important file.txt"
touch $filename             # Creates THREE files: "my", "important", "file.txt"

Word splitting does not happen on text inside double quotes. This is why you should almost always quote your variables.

Step 7: Pathname Expansion (Globbing)

The final step expands wildcard patterns into matching filenames:

# * matches any string (including empty)
echo /etc/*.conf

# ? matches exactly one character
echo /etc/host?

# [abc] matches one character from the set
echo /dev/sd[a-c]

# [!abc] or [^abc] matches one character NOT in the set
echo /dev/sd[!a]

# ** matches directories recursively (needs shopt -s globstar)
shopt -s globstar
echo /etc/**/*.conf

Globbing only happens on unquoted text. This is another reason quoting matters:

echo *.txt        # Expands to matching files
echo "*.txt"      # Literal string: *.txt

Quoting Rules

Quoting controls which expansions happen. This is one of the most important things to understand in Bash.

No Quotes

All expansions happen. Word splitting and globbing happen.

echo $HOME/*.txt
# Expands $HOME, then globs for .txt files

Double Quotes (")

Parameter expansion, command substitution, and arithmetic expansion happen. Word splitting and globbing do not happen.

echo "$HOME/*.txt"
# Expands $HOME to /home/user, but *.txt stays literal
# Output: /home/user/*.txt

# Preserves whitespace in variables
greeting="  hello   world  "
echo $greeting         # hello world  (whitespace collapsed)
echo "$greeting"       #   hello   world   (whitespace preserved)

Single Quotes (')

Nothing is expanded. Everything is literal.

echo '$HOME is not expanded'
# Output: $HOME is not expanded

echo 'No $(commands) or $((math)) either'
# Output: No $(commands) or $((math)) either

Escape Character ()

Removes the special meaning of the next character:

echo \$HOME            # $HOME  (literal dollar sign)
echo "She said \"hi\"" # She said "hi"
echo 'It'\''s here'    # It's here (ending and reopening single quotes)

The $'...' Syntax

Allows escape sequences like \n, \t:

echo $'Line 1\nLine 2\tTabbed'
# Line 1
# Line 2	Tabbed

Summary Table

Context	Variable	Command Sub	Glob	Word Split
Unquoted	Yes	Yes	Yes	Yes
Double `"..."`	Yes	Yes	No	No
Single `'...'`	No	No	No	No

Hands-On: Quoting Pitfalls

Try each of these to internalize the differences:

# Setup
mkdir -p /tmp/quoting-lab
cd /tmp/quoting-lab
touch "file one.txt" "file two.txt" "file three.txt"

# WRONG: word splitting breaks filenames with spaces
for f in $(ls); do
    echo "File: $f"
done
# Output: each word separately (broken!)

# RIGHT: use glob instead of ls, with proper quoting
for f in *.txt; do
    echo "File: $f"
done
# Output: correct filenames

# The difference between $@ and $*
# Create a test script:
cat > /tmp/quoting-lab/test-args.sh << 'SCRIPT'
#!/bin/bash
echo '--- $* (unquoted) ---'
for arg in $*; do echo "  [$arg]"; done

echo '--- "$*" (quoted) ---'
for arg in "$*"; do echo "  [$arg]"; done

echo '--- $@ (unquoted) ---'
for arg in $@; do echo "  [$arg]"; done

echo '--- "$@" (quoted -- correct) ---'
for arg in "$@"; do echo "  [$arg]"; done
SCRIPT
chmod +x /tmp/quoting-lab/test-args.sh

# Test it with arguments that contain spaces
/tmp/quoting-lab/test-args.sh "hello world" "foo bar"

# Clean up
rm -rf /tmp/quoting-lab

Variables

Setting Variables

# No spaces around the = sign!
name="Alice"        # Correct
name = "Alice"      # WRONG -- Bash thinks "name" is a command

# No need to quote simple values
count=42

# Quote when value contains spaces or special characters
greeting="Hello, World!"
path="/home/user/my files"

Local Variables

By default, variables are local to the current shell:

color="blue"
echo $color          # blue

# Start a subshell
bash -c 'echo $color'  # (empty -- variable not inherited)

Exported Variables (Environment Variables)

Use export to make a variable available to child processes:

export DATABASE_URL="postgres://localhost:5432/mydb"

# Now child processes can see it
bash -c 'echo $DATABASE_URL'
# postgres://localhost:5432/mydb

# Or set and export in one step
export API_KEY="secret123"

Readonly Variables

readonly PI=3.14159
PI=3.0
# bash: PI: readonly variable

Unsetting Variables

temp="something"
unset temp
echo $temp           # (empty)

Arrays

Bash supports indexed arrays (like lists) and associative arrays (like dictionaries/maps).

Indexed Arrays

# Declare an array
fruits=("apple" "banana" "cherry" "date")

# Access elements (0-indexed)
echo ${fruits[0]}     # apple
echo ${fruits[2]}     # cherry

# All elements
echo ${fruits[@]}     # apple banana cherry date

# Number of elements
echo ${#fruits[@]}    # 4

# Add an element
fruits+=("elderberry")
echo ${#fruits[@]}    # 5

# Iterate
for fruit in "${fruits[@]}"; do
    echo "I like $fruit"
done

# Slice (elements 1 through 2)
echo ${fruits[@]:1:2}   # banana cherry

# Indices
echo ${!fruits[@]}    # 0 1 2 3 4

# Remove an element (leaves a gap!)
unset 'fruits[1]'
echo ${fruits[@]}     # apple cherry date elderberry
echo ${!fruits[@]}    # 0 2 3 4   (index 1 is gone, others unchanged)

Associative Arrays (Bash 4+)

# Must declare with -A
declare -A config

config[host]="localhost"
config[port]="5432"
config[database]="mydb"
config[user]="admin"

# Access
echo ${config[host]}         # localhost
echo ${config[port]}         # 5432

# All values
echo ${config[@]}            # localhost 5432 mydb admin (order not guaranteed)

# All keys
echo ${!config[@]}           # host port database user

# Number of elements
echo ${#config[@]}           # 4

# Iterate over key-value pairs
for key in "${!config[@]}"; do
    echo "$key = ${config[$key]}"
done

# Check if a key exists
if [[ -v config[host] ]]; then
    echo "host is set"
fi

Think About It: Why might you prefer an associative array over a series of individual variables when managing configuration values?

Special Variables

Bash provides several special variables that are essential for scripting:

Variable	Meaning
`$$`	PID of the current shell
`$!`	PID of the last background process
`$BASHPID`	PID of the current Bash process (differs from `$$` in subshells)
`$PPID`	Parent process ID

echo "My PID: $$"
sleep 100 &
echo "Background PID: $!"
echo "Parent PID: $PPID"

Variable	Meaning
`$0`	Name of the script or shell
`$1`-`$9`	Positional parameters (first 9 arguments)
`${10}`	10th argument and beyond (braces required)
`$#`	Number of positional parameters
`$@`	All arguments (preserves quoting when in `"$@"`)
`$*`	All arguments (joins into single string when in `"$*"`)

# In a script:
echo "Script name: $0"
echo "First arg: $1"
echo "All args: $@"
echo "Arg count: $#"

The difference between "$@" and "$*" is critical:

# "$@" preserves each argument as a separate word -- USUALLY WHAT YOU WANT
# "$*" joins all arguments into a single string separated by first char of IFS

Variable	Meaning
`$?`	Exit status of the last command (0 = success)
`$_`	Last argument of the previous command

ls /etc/hosts
echo $?          # 0 (success)

ls /nonexistent
echo $?          # 2 (error)

echo hello world
echo $_          # world (last argument of previous command)

Shell Configuration

Variable	Meaning
`$HOME`	Home directory
`$USER`	Current username
`$HOSTNAME`	System hostname
`$PWD`	Current working directory
`$OLDPWD`	Previous working directory
`$PATH`	Executable search path
`$SHELL`	Default shell path
`$IFS`	Internal Field Separator
`$RANDOM`	Random integer (0-32767)
`$LINENO`	Current line number in a script
`$SECONDS`	Seconds since the shell started

echo "User $USER on $HOSTNAME in $PWD"
echo "Random number: $RANDOM"
echo "Shell has been running for $SECONDS seconds"

Hands-On: Expansion Mastery

Challenge 1: Build File Paths

# Use parameter expansion to manipulate this path:
filepath="/var/log/nginx/access.log.2.gz"

# Extract just the filename
echo ${filepath##*/}
# access.log.2.gz

# Extract just the directory
echo ${filepath%/*}
# /var/log/nginx

# Extract the extension
echo ${filepath##*.}
# gz

# Remove all extensions
echo ${filepath%%.*}
# /var/log/nginx/access

# Replace nginx with apache
echo ${filepath/nginx/apache}
# /var/log/apache/access.log.2.gz

Challenge 2: Batch Rename Using Arrays

mkdir -p /tmp/rename-lab
cd /tmp/rename-lab
touch photo_{001..005}.JPG

# Rename .JPG to .jpg using parameter expansion
for f in *.JPG; do
    mv "$f" "${f%.JPG}.jpg"
done
ls
# photo_001.jpg  photo_002.jpg  photo_003.jpg  photo_004.jpg  photo_005.jpg

# Clean up
rm -rf /tmp/rename-lab

Challenge 3: Default Values in Practice

# A script that uses default values
DB_HOST=${DB_HOST:-localhost}
DB_PORT=${DB_PORT:-5432}
DB_NAME=${DB_NAME:-myapp}

echo "Connecting to $DB_HOST:$DB_PORT/$DB_NAME"
# If no env vars set: Connecting to localhost:5432/myapp

Debug This: Quoting Gone Wrong

Someone wrote this script and it does not work with filenames containing spaces:

#!/bin/bash
# BUG: This breaks on filenames with spaces
for file in $(find /data -name "*.csv"); do
    wc -l $file
done

Problems:

$(find ...) is subject to word splitting
$file is unquoted, so it splits on spaces

Fixed version:

#!/bin/bash
# FIXED: Use find -exec or a while loop with null delimiter
find /data -name "*.csv" -print0 | while IFS= read -r -d '' file; do
    wc -l "$file"
done

Or even simpler with find -exec:

find /data -name "*.csv" -exec wc -l {} \;

What Just Happened?

+------------------------------------------------------------------+
|                     CHAPTER 18 RECAP                              |
+------------------------------------------------------------------+
|                                                                  |
|  Bash expansion order:                                           |
|  1. Brace  2. Tilde  3. Parameter  4. Command Sub               |
|  5. Arithmetic  6. Word Splitting  7. Globbing                   |
|                                                                  |
|  Quoting:                                                        |
|  - Double quotes: variables expand, no glob/split                |
|  - Single quotes: nothing expands                                |
|  - Always quote "$variable" to prevent word splitting            |
|                                                                  |
|  Parameter expansion: ${var:-default}, ${var##pattern},          |
|  ${var%pattern}, ${var/old/new}, ${#var}                         |
|                                                                  |
|  Arrays: fruits=(...), ${fruits[@]}, ${#fruits[@]}               |
|  Associative arrays: declare -A, ${map[key]}                     |
|                                                                  |
|  Special vars: $?, $$, $!, $@, $#, $0                            |
|                                                                  |
+------------------------------------------------------------------+

Try This

Exercise 1: Expansion Order

Predict the output of each line before running it, then verify:

echo {1..3}_{a,b}
echo ~root/test
echo "Today: $(date +%H:%M) - PID $$"
echo '$HOME is' $HOME
echo $((2**8))

Exercise 2: Parameter Expansion Practice

Given url="https://www.example.com/path/to/page.html?query=1", use parameter expansion to extract:

Just the protocol (https)
Just the filename (page.html?query=1)
The URL with example.com replaced by mysite.org

Linux Book: From First Boot to Production