How to Read a File Line by Line in Bash?

The most robust and common way to read a file line by line in Bash, as well as in other compatible shells like Ksh and Zsh, is by using a while read loop with input redirection. This method reliably processes each line of a file, making it ideal for scripting tasks.

The Standard `while read` Loop

The fundamental syntax for reading a file line by line is:

while IFS= read -r line; do
  # Process the "$line" variable here
  echo "Read line: $line"
done < input.file

Let's break down this powerful command:

while ... do ... done: This is a standard Bash loop construct that continues as long as the condition (in this case, the read command) returns a true (zero) exit status.
IFS=: Setting IFS (Internal Field Separator) to an empty string prevents read from interpreting leading/trailing whitespace and allows it to read the entire line, including spaces. This is crucial for preserving the exact content of each line.
read -r line:
- The read command reads a single line from standard input.
- The -r option is critical. It prevents backslash escapes from being interpreted, ensuring that backslashes (\) are treated as literal characters and not special sequences (like \n for newline or \t for tab). This is essential for accurate data processing.
- line is a user-defined variable that will store the content of each line read from the file.
COMMAND "$line": Inside the do ... done block, you place any commands that should operate on the currently read line. It's good practice to quote $line ("$line") to prevent word splitting and globbing issues, especially if lines contain spaces or special characters.
< input.file: This is input redirection. It tells the while loop to take its standard input from input.file instead of the keyboard.

Practical Examples

Let's explore some common use cases for reading files line by line.

1. Basic Line Display

To simply print each line of a file, you can use echo:

#!/bin/bash

# Create a sample file
echo -e "Line one\nSecond line with spaces\nThird\\line" > my_data.txt

echo "--- Reading my_data.txt ---"
while IFS= read -r line; do
  echo "$line"
done < my_data.txt

# Expected Output:
# Line one
# Second line with spaces
# Third\line

Notice how Third\\line is correctly displayed as Third\line thanks to the -r option. Without -r, Third\\line would likely be interpreted as Third\line.

2. Processing Each Line

You can perform more complex operations, such as counting characters or words, or filtering lines.

#!/bin/bash

echo -e "apple\nbanana split\ncherry pie" > fruits.txt

echo "--- Counting characters in each line ---"
line_num=1
while IFS= read -r fruit_name; do
  char_count=${#fruit_name}
  echo "Line $line_num: '$fruit_name' has $char_count characters."
  ((line_num++))
done < fruits.txt

# Expected Output:
# Line 1: 'apple' has 5 characters.
# Line 2: 'banana split' has 12 characters.
# Line 3: 'cherry pie' has 10 characters.

3. Skipping Empty Lines or Comments

Often, you might want to ignore blank lines or lines starting with a comment character (e.g., #).

#!/bin/bash

echo -e "# This is a comment\n\nFirst item\n  # Another comment\nSecond item" > config.txt

echo "--- Processing config.txt, skipping comments and blank lines ---"
while IFS= read -r entry; do
  # Skip empty lines and lines starting with '#'
  if [[ -z "$entry" || "$entry" =~ ^[[:space:]]*# ]]; then
    continue
  fi
  echo "Valid entry: $entry"
done < config.txt

# Expected Output:
# Valid entry: First item
# Valid entry: Second item

4. Reading Specific Fields (Delimited Data)

If your file uses a specific delimiter (e.g., comma, tab), you can adjust IFS inside the loop to read individual fields.

#!/bin/bash

echo -e "Alice,30,New York\nBob,24,London" > users.csv

echo "--- Reading CSV data ---"
while IFS=',' read -r name age city; do
  echo "Name: $name, Age: $age, City: $city"
done < users.csv

# Expected Output:
# Name: Alice, Age: 30, City: New York
# Name: Bob, Age: 24, City: London

Notice IFS=',' is placed before read inside the while condition itself. This applies the IFS change only to the read command, not globally.

Important Considerations for `read`

The read command offers several options to fine-tune its behavior. Here are some key ones:

Option	Description
`-r`	Raw input: Prevents backslash escapes from being interpreted. (Highly Recommended)
`-a`	Reads fields into an indexed array.
`-n N`	Reads at most `N` characters.
`-p PROM`	Displays `PROM` on standard error without a trailing newline before attempting to read.
`-t TIMEOUT`	Times out after `TIMEOUT` seconds if input is not received.
`-u FD`	Reads from file descriptor `FD` instead of standard input (0).

For more details, consult the Bash Manual on read built-in.

Why Avoid `for line in $(cat file)`?

While you might see examples like for line in $(cat file), this approach is generally discouraged for reading files line by line due to several issues:

Word Splitting: $(cat file) expands into a single string which Bash then splits into "words" based on the IFS variable (by default, space, tab, newline). This means lines with spaces will be split into multiple items.
Globbing: If any "word" contains wildcard characters (e.g., *, ?, [), Bash will attempt to expand them into filenames, leading to unexpected behavior.
Inefficiency: cat reads the entire file into memory (or a pipe) before the loop even starts, which can be inefficient for very large files.

The while IFS= read -r loop, on the other hand, reads the file line by line directly, avoiding these pitfalls and offering a much more robust and predictable way to process file contents.

By mastering the while IFS= read -r construct, you gain a powerful and reliable tool for file processing in your Bash scripts.