Editing in Place - Redirecting Command Output to the Same Source File
Updated: Jun 30
Sometimes you may want to use the command line to edit a file in place without creating a temporary file. Some commands, such as sed, natively support this functionality, but most do not.
For example, suppose you had the following file scores.txt:
Last First Score Smith Dave 95 Jones Tom 74 Cooper Ann 85
You can use the cut command to remove the middle column and output only the last name and score:
$ cut -f1,3 scores.txt Last Score Smith 95 Jones 74 Cooper 85
But suppose you wanted to save the output back to scores.txt. You cannot simply use redirection to solve this problem:
$ cut -f1,3 scores.txt > scores.txt
Instead of outputting the result of the cut command to scores.txt, the redirection above will result in scores.txt being overwritten with an empty file.
One solution is to output the result of the cut command to a temporary file, delete the original scores.txt file, and then rename the temporary file. Here is a one-line example:
cut -f1,3 scores.txt > tmp.txt; rm scores.txt; mv tmp.txt scores.txt
While this works, it is quite lengthy and requires the generation of a temporary file.
To help simplify this and eliminate the need for a temporary file I created a bash script, ow, to output a stream to a specified file, including overwriting the original source file of a command pipeline.
#!/usr/bin/env bash #Command Line Wizardry #commandlinewizardry.com #Overwrite #Version 1.0 if [ $# == 0 ] then echo "$0: Output file name required" exit 1 fi out="" while IFS= read LineIn do out="$out$LineIn\n" done echo -en $out>$1
To use it, just put it at the end of a command pipeline and specify a file to write the stream to. To overwrite the original source file specify the source filename as the argument for ow.
In the example below the output of the cut command is piped into ow and the result is written back to scores.txt.
cut -f1,3 scores.txt | ow scores.txt
You can download the ow script from our GitHub repository. To make it even more useful be sure to add ow to your path and make it executable so the script can be accessed from anywhere on your command line.
Note that ow is written primary for text output. If your command pipeline outputs binary data ow may not process it correctly.