Pipes are a very powerful core feature of Linux. By chaining programs together, you can perform complex tasks with a small set of tools. But pipelines are not always the best solution; when not needed, it can increase complexity and lead to less efficient options.
Many Linux commands have options that allow them to do the work of another command. For example, although the sort command exists, ls can sort its output. Learn which common pipelines have better alternatives and how to open them.
catch | toilet
The pipeline is generic, but grep has its own counter
This example is very common, partly because it is such a good demonstration of the toilet. Word counter toolwhich can also count lines and characters, is well-suited to many pipelines because you often care more about the number of results than what they actually are:
grep -E "^.+() {$" funcs.sh | wc -l
While wc perfectly counts the number of results, grep itself can do this using the -c or –count option.
grep --count -E "^.+() {$" funcs.sh
It seems that the -c option has been a feature of grep since its early days, so there was never any excuse for this model to be caught.
cat file | everything
A useless use of a cat that can be easily avoided
Another incredibly common use of a pipe that can be avoided. Cat command is useful because even though it means “to combine”, you don’t need to combine anything at all. By default, it writes a file to standard output, even if it’s a single file.
This means that it’s often tempting to reach for cat when you want to pass the contents of a file as input to another command:
cat file > wc -l
But a command like wc will read input from every file argument you pass it, so there’s no need to pipe a cat:
wc -l file
But what if the command doesn’t support file arguments? Then you have to use the cat, right? Actually, no: Linux has a built-in file redirection operator this means that even in these cases the cat is unnecessary.
tr ' ' '\n' It takes a little getting used to, especially when kitty is in your muscle memory, but learning to redirect effectively will open up many command line possibilities.
ls | to sort out
The built-in type brings new advantages
The sort command is another command that’s perfect for pipelines: it’s a filter that, as you might expect, sorts all the input it receives and sends it to standard output:
So it can be very useful for commands like ls that can produce output in a near tabular form. In fact, the sort -k feature, which specifies the number of fields to sort, makes it very versatile for this purpose.
Again, there’s no real need to use it, at least in the case of ls, which supports its own option to sort by file size:
ls -lrS
The -r option reverses the sorting direction to show the largest files at the bottom, which is probably what you want.
There is another small but important benefit of using the built-in sort in ls. When you sort the ls output, you lose any color formatting, but the built-in sort in ls keeps it.
head | the tail
Extract a range of lines with one command
Head and tail commands is great for extracting the first or last n input strings, respectively. Combined, they’re a neat trick for extracting a specific set of lines from a file:
It’s smart, but you can do the same thing with sed, a stream editor:
sed -n '102,105p' Note that both this command and the previous pipeline fetch lines 102-105 from the file, but the sed version requires less math.
grep | the head
Keep large result sets manageable and save extra work
Sometimes your grep has one result, sometimes hundreds or more. When you have a lot of grep output to manage, it’s natural to reach for the head command again to stop after a certain number:
grep firefox access_log | head -n 10
This is unnecessary thanks to grep’s -m (–max-count) option, which does the same thing:
grep --max-count=10 firefox access_log
sort | unique
A classic pipeline with a bit of team
Here’s another pipeline you’ll see often, and it’s the best, if not the only, way to sort and dedupe data lines:
sort But there is another way without using it awkwardly named unique in general. The sort command has uniqueness set as an option using -u (–unique):
sort --unique Always consider alternatives
Linux is a flexible system and has many small tools that you can combine to perform various tasks. This gives you a lot of flexibility, so make sure you try other ways of doing things, understand the differences, and be willing to learn new approaches.
