Editing in Place - Redirecting Command Output to the Same Source File

Sometimes you may want to use the command line to edit a file in place without creating a temporary file. Some commands, such as sed, natively support this functionality, but most do not.

---

For example, suppose you had the following file scores.txt:

Last      First   Score
Smith     Dave    95
Jones     Tom     74
Cooper    Ann     85

scores.txt

You can use the cut command to remove the middle column and output only the last name and score:

$ cut -f1,3 scores.txt

Last    Score
Smith   95
Jones   74
Cooper  85

But suppose you wanted to save the output back to scores.txt. You cannot simply use redirection to solve this problem:

$ cut -f1,3 scores.txt > scores.txt

Instead of outputting the result of the cut command to scores.txt, the redirection above will result in scores.txt being overwritten with an empty file.

One solution is to output the result of the cut command to a temporary file, delete the original scores.txt file, and then rename the temporary file. Here is a one-line example:

cut -f1,3 scores.txt > tmp.txt; rm scores.txt; mv tmp.txt scores.txt

While this works, it is quite lengthy and requires the generation of a temporary file.

To help simplify this and eliminate the need for a temporary file I created a bash script, ow, to output a stream to a specified file, including overwriting the original source file of a command pipeline.

#!/usr/bin/env bash
#Command Line Wizardry
#commandlinewizardry.com

#Overwrite
#Version 1.0

if [ $# == 0 ] 
then
    echo "$0: Output file name required"
    exit 1
fi

out=""

while IFS= read LineIn
do
    out="$out$LineIn\n"
done

echo -en $out>$1

To use it, just put it at the end of a command pipeline and specify a file to write the stream to. To overwrite the original source file specify the source filename as the argument for ow.

In the example below the output of the cut command is piped into ow and the result is written back to scores.txt.

 cut -f1,3 scores.txt | ow scores.txt

You can download the ow script from our GitHub repository. To make it even more useful be sure to add ow to your path and make it executable so the script can be accessed from anywhere on your command line.

Note that ow is written primary for text output. If your command pipeline outputs binary data ow may not process it correctly.