Parsing structured data files is one of those things you end up writing constantly in shell.
I spent years reaching for awk or fiddling with cut options before
landing on something far simpler, built right into how the shell reads data.
If you want to parse a colon-delimited file like /etc/passwd, all you need is:
IFS=':'
IFS is the Internal Field Separator. Set it to your delimiter and word splitting
does the rest.
In Practice
line='nobody:foo:-2:-2:Unprivileged User:/var/empty:/usr/bin/false'
IFS=":"
array=($line)
echo "username: ${array[0]}"
echo "uid: ${array[2]}"
echo "home: ${array[5]}"
echo "shell: ${array[6]}"
Note the single quotes around $line. Double quotes would let the shell expand
the * wildcard to filenames in the current directory.
Here's a complete script you can invoke as script.sh < /etc/passwd:
#!/bin/bash
while read line
do
# skip comment lines
[[ ${line} == \#* ]] && continue
set -f # disable globbing so '*' isn't expanded to filenames
IFS=':'
array=(${line})
username="${array[0]}"
uid="${array[2]}"
gid="${array[3]}"
info="${array[4]}"
home="${array[5]}"
shell="${array[6]}"
echo "$username uid=$uid home=$home shell=$shell"
set +f # re-enable globbing
done
Other Useful Array Tricks
Iterate over elements:
for element in "${array[@]}"; do echo "$element"; done
Iterate with index:
for index in "${!array[@]}"; do echo "$index ${array[index]}"; done
Count elements:
echo "${#array[@]}"
Here Be Dragons
Shell arrays have no safety wires. You can unset individual elements or insert at arbitrary indices, which means the array you're working with may be sparse:
unset "array[5]" # remove element 5
array[9]="ha ha" # insert at index 9, skipping 7 and 8
So this will silently give you the wrong answer:
# breaks on sparse arrays
count=${#array[@]}
last_index=`expr $count - 1`
echo "${array[$last_index]}"
Use this instead:
# Bash 2.05b+
echo "${array[@]: -1:1}"
# Bash 4.2+
echo "${array[-1]}"
Shell arrays: powerful, footgun-adjacent.