Advanced Bash Scripting Techniques

Bash Patterns Beyond the Basics#

Most bash content stops at "use shellcheck and quote your variables." That's important but a narrow slice of what bash can do. After enough years of writing operational scripts, I keep reaching for a specific set of intermediate features. This post is those features, with examples of when each pays off and when it's the wrong tool.

When bash earns its place#

Before any pattern: bash is the right tool when the script is mostly orchestrating other commands. Pure logic, complex data structures, anything that would benefit from types — Python is a better choice. The patterns below are for when bash is genuinely the right call.

The line for me is around 150 lines. Above that, the script's complexity usually justifies a real language.

Arrays#

Bash arrays let you handle lists of items cleanly:

files=(
  "/etc/passwd"
  "/etc/hosts"
  "/etc/resolv.conf"
)

for f in "${files[@]}"; do
  echo "Checking $f"
  [ -f "$f" ] || echo "  Missing!"
done

The "${files[@]}" is the right way to expand an array — quoted, with @. Using * joins with the first IFS character; using unquoted forms breaks on filenames with spaces.

Reading lines from a file into an array:

mapfile -t lines < input.txt
echo "Read ${#lines[@]} lines"

mapfile -t (also called readarray) reads each line as an array element, stripping the trailing newline. Much better than the old "while read" pattern for whole-file ingestion.

Associative arrays (bash 4+):

declare -A counts
counts["error"]=0
counts["warning"]=0

while read line; do
  if [[ "$line" =~ ERROR ]]; then
    counts["error"]=$((counts["error"]+1))
  elif [[ "$line" =~ WARN ]]; then
    counts["warning"]=$((counts["warning"]+1))
  fi
done < log.txt

for key in "${!counts[@]}"; do
  echo "$key: ${counts[$key]}"
done

Hash maps in bash. Useful for counting, grouping, lookups. Mostly for shorter scripts; for anything substantial, Python is cleaner.

Trap for cleanup#

trap runs a command when a signal arrives or when the script exits:

TMPDIR=$(mktemp -d)
trap 'rm -rf "$TMPDIR"' EXIT
trap 'echo "Interrupted; cleaning up..."; rm -rf "$TMPDIR"; exit 130' INT TERM

# Use $TMPDIR confidently — it'll get cleaned up

The EXIT trap runs whether the script succeeds or fails — perfect for cleanup. INT (Ctrl-C) and TERM (kill) traps catch user-initiated interruption.

Common cleanup uses:

• Remove temp files / directories
• Reset terminal state (e.g., re-enable echo if you turned it off)
• Send a "I died" notification
• Release locks

I add a trap to nearly every non-trivial script that creates state. Without it, interrupting the script leaves cruft.

Process substitution#

<(...) and >(...) are bash-specific (not POSIX) features for treating command output as a file:

diff <(sort file1) <(sort file2)

This sorts both files and diffs them, without writing temporary files. Each <(...) is a virtual file (technically /dev/fd/N).

Common uses:

• Comparing command outputs (diff, comm)
• Reading from a command into a while read loop without subshell scoping issues
• Feeding a command's output to a tool that wants a filename

# Avoid the subshell trap of "while read | piped output"
while read -r line; do
  count=$((count + 1))
done < <(some_command_that_emits_lines)
echo "$count"

The < <(...) (note the space) feeds the output of a command as the script's stdin to the while loop. Unlike ... | while read, this doesn't run the loop in a subshell, so variables modified inside persist.

Parameter expansion: more than just $var#

Bash has rich parameter expansion — string manipulation without forking subprocesses:

# Default if unset
name="${USER:-unknown}"

# Default and assign if unset
: "${LOG_LEVEL:=INFO}"

# Error if unset
: "${REQUIRED_VAR:?must be set}"

# Substring
str="hello world"
echo "${str:0:5}"        # hello
echo "${str:6}"          # world

# Length
echo "${#str}"           # 11

# Pattern removal (greedy with ##, %% — non-greedy with #, %)
filename="/path/to/file.txt"
echo "${filename##*/}"   # file.txt (basename)
echo "${filename%/*}"    # /path/to (dirname)
echo "${filename%.*}"    # /path/to/file (without extension)
echo "${filename##*.}"   # txt (extension)

# Replacement
str="foo bar foo"
echo "${str/foo/baz}"    # baz bar foo (first occurrence)
echo "${str//foo/baz}"   # baz bar baz (all)

# Case modification
echo "${str^^}"          # FOO BAR FOO (upper)
echo "${str,,}"          # foo bar foo (lower, lowercase variant)

These are faster than spawning sed/cut/awk for simple string ops. For anything complex, those tools are still right.

Brace expansion#

Bash expands brace patterns before execution:

echo {1..5}              # 1 2 3 4 5
echo {a..f}              # a b c d e f
echo file_{1..3}.txt     # file_1.txt file_2.txt file_3.txt
echo dir/{prod,staging,dev}/config.yml

Useful for terse iteration:

for env in {dev,staging,prod}; do
  ./deploy.sh "$env"
done

# Backup files atomically
cp /etc/nginx/nginx.conf{,.bak}
# Equivalent to: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

Conditional patterns: [[ over [#

[[ ... ]] is bash's enhanced test ([ ... ] is the POSIX original). [[ doesn't word-split, supports regex, and has clearer syntax:

# Better
if [[ "$var" =~ ^[a-z]+$ ]]; then ...
if [[ -n "$var" && "$var" != "default" ]]; then ...

# Worse / fragile
if [ "$var" != "" ] && [ "$var" != "default" ]; then ...

For new scripts, prefer [[ unless you specifically need POSIX compatibility (in which case, POSIX shell isn't bash anyway).

Heredocs and herestrings#

Heredoc: multi-line string literal:

cat <<EOF > /etc/myconfig
listen = 8080
host = ${HOSTNAME}
log_level = INFO
EOF

Variables expand inside (use <<'EOF' with single quotes to disable expansion).

Herestring: short single-line input:

read -r var <<< "input string"

Both pass content to a command's stdin without a temporary file or echo.

Functions with proper return values#

Functions in bash can't return arbitrary values; they return exit status (0-255). For other return values, use stdout:

get_user_id() {
  local username="$1"
  id -u "$username" 2>/dev/null
}

# Usage
if user_id=$(get_user_id "myuser"); then
  echo "User ID: $user_id"
else
  echo "User not found"
fi

The function writes to stdout; the caller captures with $(...). The function's exit status (whether id succeeded) is checkable via if.

For functions that need multiple outputs, multi-value packing via printf:

get_user_info() {
  local username="$1"
  printf "%s|%s|%s" "$(id -u "$username")" "$(id -g "$username")" "$(id -gn "$username")"
}

IFS='|' read -r uid gid gname <<< "$(get_user_info "myuser")"

Awkward but functional. If you're doing this a lot, that's a sign to move to a real language.

Command-line option parsing#

For more than 2-3 options, use getopts or a hand-rolled parser:

verbose=0
output=""

while getopts ":vo:" opt; do
  case $opt in
    v) verbose=1 ;;
    o) output="$OPTARG" ;;
    \?) echo "Invalid option: -$OPTARG" >&2; exit 1 ;;
    :) echo "Option -$OPTARG requires argument" >&2; exit 1 ;;
  esac
done
shift $((OPTIND-1))

For long options (--verbose), getopts doesn't help; you need hand-rolled or getopt (the external command, different from bash's getopts).

Honestly, for non-trivial CLI parsing, Python with argparse is much nicer.

Locking to prevent concurrent runs#

For scripts that shouldn't run concurrently:

LOCKFILE="/var/run/myscript.lock"
exec 200>"$LOCKFILE"
flock -n 200 || { echo "Already running"; exit 1; }

# Now we have the lock; do work

flock on a file descriptor; non-blocking (-n) so we exit if locked. The lock is released when the script exits (file descriptor closes).

Useful for cron jobs that should not overlap themselves.

Strict mode patterns#

The strict mode preamble:

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

Plus extras for debugging:

# Trace execution (verbose)
set -x

# Run with `bash -x script.sh` instead if you don't want to modify the script

# Or, more selective:
PS4='+ ${BASH_SOURCE}:${LINENO}: ${FUNCNAME[0]:-main}() '

The PS4 makes the trace show source location, which is invaluable for debugging.

What I avoid#

A few patterns that look clever but cause problems:

Multiline eval. Already covered: eval is dangerous. If you need dynamic execution, use a real language.

Function return values via global variables. Named "out parameters." Works but obscures data flow. Functions returning via stdout are cleaner.

Heavy use of bash -c "$dynamic_string". Same risks as eval.

Custom argument-parsing libraries in bash. They exist but they're awkward. Use Python at this point.

printf for complex formatting. Useful for simple cases; for anything elaborate, a templating language or real string-formatting code is better.

Testing bash scripts#

For scripts beyond a certain complexity, tests:

# tests/test_helper.bash
test_get_user_id() {
  source ../script.sh
  result=$(get_user_id "root")
  [ "$result" = "0" ] || { echo "FAIL"; return 1; }
}

test_get_user_id

For more elaborate tests, bats (Bash Automated Testing System) is the standard tool. Each @test block is a test case; pass/fail per block.

shellcheck for static analysis (mandatory in our CI).

What I'd tell a team#

Quote everything. set -euo pipefail. shellcheck in CI. The boring fundamentals matter most.

Use arrays for lists. Don't pretend space-separated strings are arrays.

trap for cleanup. Every script that creates state should clean up on exit.

Process substitution to avoid temp files. <(cmd) is your friend.

Parameter expansion for simple string ops. Faster and more readable than calling sed/cut/awk.

Move to Python at ~150 lines. Bash for short orchestration; Python for anything substantive.

The intermediate bash patterns above are the ones that earn their place in operational scripts. Knowing them keeps your bash readable and your scripts maintainable. They're not a substitute for moving to a real language when the task outgrows shell — but for the work that genuinely belongs in bash, they're the difference between scripts that survive and scripts that need rewriting in 6 months.