One of the shell's most important features is it's ability to redirect the standard input and standard output of any program. Programs that ordinarly write to the screen can have that output sent to a file. Programs that ordinarly read from the keyboard can have their input redirected from a file. For example

$ prog >afile.txt

Now afile.txt contains the text that prog would have displayed on the screen.

$ prog <bfile.txt

Now prog reads its input from bfile.txt rather than from the keyboard.

Both the input and the output can be redirected.

$ prog <bfile.txt >afile.txt

The I/O redirection part of the command line is not sent to the program. As far as prog is concerned in the examples above, it is being executed with no arguments. If prog wants arguments, they can be put on the command line anywhere relative to the I/O redirection operators. For example

$ prog <bfile.txt arg1 >afile.txt
$ prog arg1 >afile.txt <bfile.txt
$ prog > afile.txt < bfile.txt arg1

all do the same thing.

Ordinarily when output redirection is used, the shell truncates the output file to zero size first if it already exists. Commonly you want to append the new output to the end of the file. This can be done with the '>>' operator. For example

$ echo "Finished processing on `date`" >>$HOME/results

This saves the current date (and time) into the file results in the home directory. It adds a new record onto the end of any existing records.

Many Unix programs that process the data inside files will accept the name of the file(s) to process on the command line. If there are no such names presented, a typical Unix program will process whatever it finds at its standard input. Consider the commands below. They both search the file afile.txt looking for lines that contain the string "Peter."

$ grep 'Peter'   afile.txt
$ grep 'Peter' < afile.txt

In the second case, the grep command does not see any file name on the command line. It thus reads it's standard input (which happens to be the same file as named in the first command).

This behavior allows you to test a program without creating a special data file. Just run it and type at it. For example

$ grep 'Peter'
Hello.
Anybody there?
My name is Peter.
My name is Peter.
What's your name?
Is it Peter also?
Is it Peter also?
^D
$

The '^D' (control+D) causes the terminal to generate an EOF character. Notice how grep printed out lines that contained the string 'Peter.'

The shell allows you to direct the standard output of one program into the standard input of another. This is done with the pipe operator '|.' This ability, coupled with the behavior described above, makes the Unix system powerful. For example

$ ls -l | grep '^d'

This command displays all lines in the output of the ls -l command that start with a 'd'. Such lines are entries for subdirectories.

$ who | wc -l

This command sends the output of who to the word count program to count the number of lines in who's output. The result is a count of the number of users logged into the system.

$ prog args | mail pchapin

This command mails the output of the prog command to pchapin. This works because the mail program accepts the message to be mailed at it's standard input.

It is possible to redirect or pipe the output of entire loops. For example

$ for FILE in *
> do
>   process $FILE
>   echo "Done with $FILE on `date`"
> done > $HOME/results
$

Note the redirection done after the 'done' keyword. All the output generated within the loop (except for stuff explicitly redirected) is put into $HOME/results. This is faster and cleaner than redirecting with append each command within the loop.

Pipes are also possible.

$ for FILE in *
> do
>   process $FILE
> done | sort > $HOME/results
$

When you redirect the output of a loop, the loop is run in a subshell. This may be significant if the point of the loop is to set shell variables.