Redirecting stdout/stderr
What Doesn't Work
There are several solutions that have been discussed that are nearly right. They all revolve around the use of tee.
tee is a very simple piece of code that reads from stdin and writes everything to both stdout and a supplied file. It sounds like it's the tool for the job:
tee file
which looks very simple -- and it is.
So, we could run:
cmd args | tee file
which works fine for stdout but doesn't do anything with stderr.
OK, so we could write:
cmd args |& tee file
or
cmd args 2>&1 | tee file
When we look in file we'll see both stdout and stderr in the right sort of order (there's always some slight buffering issues) and you'll get the same output on stdout.
Natch, there's the problem. Whilst we can see stderr in amongst the stdout, the stderr output is embedded in the stdout. It is no longer on a separate stream.
Do we care? Yes we do. Suppose you've wrapped the above into a script, foo:
foo >/dev/null
Me? I'd expect to see stderr still but I don't see anything because stderr has been embedded within stdout in foo and on the command line I'm innocently redirecting stdout to /dev/null -- I should see stderr but I don't. Someone's violated the principle of least surprise.
Let's try another option that a bit more complicated and involves "process substitution" which isn't available on all platforms. But we've got it so let's try the basics:
cmd args > >(tee file)
With process substitution, the expression >(...) is replaced with the filename of the file descriptor opened for writing to ... (in this case tee file) so that would make our command line expand to:
cmd args > /dev/fd/63
for example, ie, an ordinary looking redirect to a file. Of course, /dev/fd/63 is the file system name for file descriptor 63 which is available for writing to. Sat reading file descriptor 63 is our process tee file and so the net effect of all this mularky is that stdout from cmd args goes to the bucket, file descriptor 63, from where it is read by tee who writes the contents to its stdout (inherited, in turn, from the shell) and also to file. In others words, it works just like:
cmd args | tee file
So, it seems like a whole load of trouble for nothing but we've only just started. The stderr variant of the above looks like:
cmd args 2> >(tee file >&2)
which looks a bit crazy but let's break it down:
cmd args 2> >(...)
or
cmd args 2> /dev/fd/63
ie. send stderr of cmd args to this file system name for a file descriptor and our substituted process is:
tee file >&2
which is just like tee file except we're adding >&2 which says to redirect the stdout of tee to stderr which is a good thing as the stdin of tee was the stderr of cmd args and the stderr of tee has been inherited, in turn, from the shell. So the net effect of all that was that the stderr of cmd args goes to both stderr and file.
Great!
Except it isn't. When we've combined the two, we have:
cmd args > >(tee file) 2> >(tee file >&2)
(two separate redirections, one for each of stdout and stderr where they both are sent to file and will separately be sent to stdout and stderr).
What's wrong with that, then? Well, now you've got two separate tee processes both writing to the same file in the filesystem. There might well be ways and means you believe you can coordinate those two processes but, at the end of the day, if you have two processes writing to a file they will end up corrupting each other.
Document Actions