The hidden gems of moreutils
Revealed Dec 19, 2023
It appears irrespective of how lengthy I work with the command line, each from time to time
I discover useful utilities I’ve by no means encountered earlier than. Most individuals have heard
about the chicken coreutils, that is the place utilities equivalent to
echo, cat, and others come from. However do you know about moreutils?
ts
Say I need to hint each program invocation (~exec()) on the system. I can do
that with execsnoop from the BPF Compiler Collection.
# execsnoop
ls 1586 1401 0 /usr/bin/ls --color=auto -F
git 1587 1401 0 /usr/bin/git rev-parse ...
Apparently each time I contact a terminal, it runs Git to find out whether or not I am
inside a Git repository simply in order that it could present me the present department within the
immediate. Certainly that does not sluggish issues down. What it does not present me is the
time of the invocation although. I can simply add it with ts.
# execsnoop | ts
Dec 19 12:39:25 ls 1791 1401 0 /usr/bin/ls --color=auto -F
Dec 19 12:39:25 git 1792 1401 0 /usr/bin/git rev-parse ...
I now notice that execsnoop does have the -T flag which provides invocation
instances so there isn’t any want for ts on this case however I’ve already written these
examples so yeah. ts may convert time stamps into relative instances. That is
particularly useful when wanting by way of log recordsdata.
$ journalctl --since yesterday --priority emerg..warning
Dec 18 19:30:47 vm sudo[5068]: person: 3 incorrect password makes an attempt
$ journalctl --since yesterday --priority emerg..warning | ts -r
18h7m in the past vm sudo[5068]: person: 3 incorrect password makes an attempt
sponge
Have you ever ever needed to change a file, save the consequence into the identical file, and
bought shortly disillusioned with the consequence?
$ echo 1n3n2 > file.txt
$ type file.txt > file.txt
$ wc -l file.txt
0 file.txt # empty
Keep in mind, it’s the shell which is liable for redirecting the output, not
the person instructions. On this case, when the shell sees > file.txt, it
opens file.txt for writing (or creates it if essential). Crucially, it additionally
opens the file with the O_TRUNC flag which immediately truncates (empties) the
file. When type later opens the file to do the precise work, it finds the file
empty and exits.
The commonest workaround is to first redirect the output to a brief file
after which transfer it again to the unique title. And that is precisely what sponge
does behind the scenes for you.
echo 1n3n2 > file.txt
type file.txt | sponge file.txt
Apparently, a number of the instructions from coreutils have a “-o” flag. I might
have simply written type -o file.txt file.txt. Oh nicely.
vidir
That is in all probability what I take advantage of most frequently. Working vidir opens your $EDITOR or
$VISUAL with recordsdata / directories of the required listing (or the present
listing) and lets you edit them. Should you change the title, it would rename
it. Should you delete a file row, the file will get deleted. To delete a complete
listing with every part in it, delete its row and all sub entries.
vidir *.pdf
fd -t f | vidir -
What should you’ve made so many nice modifications that you simply need to simply stop the editor
with out making use of any of them? You simply must stop the editor with exit code
1. In Vim / Helix you do this with :cq. This by the way in which works in fairly
a lot all instances the place a command invokes your editor (e.g. when writing a commit
message).
In case your distribution does not present moreutils, there’s additionally qmv from
renameutils.
vipe
This can be a bit soiled however may be helpful generally. Think about you need to course of a
bunch of recordsdata however can not get the damned file title regex proper and so you may have
a number of further recordsdata within the output. You could possibly simply write the output to a file, edit
it, and proceed from there however there a extra fragile choice – edit the output
proper between the pipes!
fd | vipe | ...
On this case, the output from fd can be buffered by vipe and handed to your
configured textual content editor. You can also make any modifications you need and when you stop, the
information can be handed down the pipe if any. The draw back is that if you wish to
run the command once more, you have to to additionally edit output once more. It is in all probability a
higher thought to simply write it right into a file. At the very least on this state of affairs.
pee
I will point out this not as a result of I’ve an precise use case for this however
simply because it took me a second to appreciate what it really does. pee takes
its stdin and passes it to all instructions given as arguments. It then gathers
their output and sends that as its personal output. It runs the instructions utilizing
popen, so it is really passing them to /bin/sh -c (which on most programs is
a symlink to Bash).
$ echo "Alice" | pee "xargs echo Good day" "xargs echo Hello"
Good day Alice
Hello Alice
I believe these are all of the instructions I commonly use / I’ve used. There are extra
accessible, equivalent to continual (print output provided that the command failed) or ifne
(run command provided that there’s non empty stdin) however I have not wanted them but.
In addition to extra instructions in moreutils, there are much more of those command units equivalent to evenmoreutils or num-utils.