Shell scripting with Elixir – Michal (arathunku)
When easy bash scripts begin to change into unwieldy, you could think about reaching out for one thing “greater” degree, like Perl, Ruby or Python. I’m reaching out for Elixir. Possibly the startup occasions will not be good for each use case, however Elixir is extraordinarily versatile. It’s straightforward so as to add dependencies, debug, iterate and even write exams, all in a single file! I consider, on account of LiveBook, it actually hits its stride lately; the ecosystem leans closely into ergonomic developer expertise and nice out-of-the-box defaults (Req is superb!). In few traces of code, you possibly can hook up with Postgres, ship HTTP requests or begin HTTP server.
For my very own use circumstances I’ve written scripts for displaying climate(focus solely on rain, v. vital for canine walks when your canine hates rain) in i3status-rust, remodeling numerous CSV information from financial institution exports into beancount format, or creating Telegram bots.
A easy script could also be simply:
#!/usr/bin/env elixir
IO.places("Whats up world")
Effectively, if it had been that easy, I’d simply do echo "Whats up world" and skip Elixir, however then there’d be no level in penning this weblog put up, proper? So, I’ve one other, a bit extra concerned and what might appear to be over-complicated template. It’s a place to begin for extra complicated scripts, and I can both take away among the components I don’t want or begin extending it.
The template
#!/usr/bin/env -S ERL_FLAGS=+B elixir
Combine.set up([])
if System.get_env("DEPS_ONLY") == "true" do
System.halt(0)
Course of.sleep(:infinity)
finish
defmodule Whats up do
@moduledoc """
<!-- TODO -->
## Utilization
$ bin/good day --help
"""
@args [help: :boolean]
def principal(args) do
{parsed, args} = OptionParser.parse!(args, strict: @args)
cmd(parsed, args)
finish
defp cmd([help: true], _), do: IO.places(@moduledoc)
defp cmd(_parsed, _args) do
IO.places(@moduledoc)
System.cease(1)
finish
finish
Whats up.principal(System.argv())
To start with, the shebang is just not the standard factor you’d anticipate. It configures a further flag for erl. This flag, per docs, “De-activates the break handler for (SIGINT) ^C and ^ “. I’ll develop on it in Signals part.
Now, we’re executing, and there’s a ready-to-go Mix.install/2 assertion the place we are able to add further dependencies. To make HTTP requests, we are able to add battries-included HTTP shopper like {:req, "~> 0.4"}. Processing JSON? {:jason, "~> 1.4"}. We will seek for packages with combine hex.search [package-name] or lookup the newest model combine hex.data [package-name]
$ combine hex.data req
Req is a batteries-included HTTP shopper for Elixir.
Config: {:req, "~> 0.4.8"}
Releases: 0.4.8, 0.4.7, 0.4.6, 0.4.5, 0.4.4, 0.4.3, 0.4.2, 0.4.1, ...
Licenses: Apache-2.0
Hyperlinks:
Changelog: https://hexdocs.pm/req/changelog.html
GitHub: https://github.com/wojtekmach/req
One other uncommon block is if System.get_env("DEPS_ONLY") do ... finish. If the script doesn’t have any dependencies, I’d simply delete it or let or not it’s. It’s helpful in circumstances when the script has dependencies. We will use this block to cache dependencies and compilation of the script, skipping the execution of the remainder of the script. That is helpful for CI setups or when constructing container pictures. For CI, I’d additionally outline a listing the place the dependencies needs to be cached. for GitLab CI, a job might appears to be like like this:
print-hello:
# https://hub.docker.com/r/hexpm/elixir/tags?web page=1
picture: hexpm/elixir:1.16.1-erlang-26.2.2-debian-bookworm-20240130-slim
variables:
MIX_INSTALL_DIR: "$CI_PROJECT_DIR/.cache/combine"
cache:
- key: elixir-cache
paths:
- .cache
script:
- DEPS_ONLY=true bin/good day
- bin/good day
After that, there’s a module the place we’ll be documenting what the script is about, defining choices for parsing arguments and failing gracefully when invalid choices are handed, guaranteeing correct error exit code. That’s additionally the place we’d lengthen the script, earlier than final cmd/2. CLI argument parsing is completed with built-in OptionParser.
This construction could seem a bit verbose and will seem like loads of boilerplate, however once more, if it had been easy, we’d have simply stayed with the bash within the first place. Right here, this construction can simply develop with the script.
I used to outline inline capabilities like print_help = fn -> ... finish or process_args = fn (args) -> .. finish however ultimately, working inside a module is cleaner, and no have to look if given operate is nameless operate (.() name) or module’s operate.
With the template in place, we’re prepared so as to add some logic to it. Elixir can do fairly a bit simply with the usual library, however there are additionally some gotchas. Let’s undergo some frequent wants.
Output
IO will most likely be probably the most typically used module. It may be used to jot down stdout with capabilities like IO.write/1, IO.places/1, or to stderr with their equal 2-argument calls like IO.places(:stderr, "Error"). We will additionally learn inputs with IO.learn/2. Any writing or studying may be additionally dealt with as a stream with IO.stream/2.
IO.places("Whats up")
IO.places(:stderr, "Invalid argument")
IO.stream(:stdin, :line) # that is the default
|> Enum.map(&String.trim_trailing(&1, "n"))
|> Enum.map(&String.reverse/1)
|> Enum.map(&IO.places/1)
Colours
There’s no have to outline coloration codes manually. With IO.ANSI, we are able to add textual content and background colours simply.
iex(1)> IO.ANSI.blue_background()
"e[44m"
iex(2)> v <> "Example" <> IO.ANSI.reset()
"e[44mExamplee[0m"
iex(3)> v |> IO.puts()
Example
:ok
Exit code
Another quick one, there’s System.stop(exit_code) to gently shutdown VM, what may not be obvious is that it’s async process. Make sure to call Process.sleep(:infinity) after it to block the execution. This ensures that all the applications are taken down gently. There’s alternative of System.halt/1 and it forces immediate shutdown of Erlang runtime system.
Subprocesses
For one-off commands, System.cmd/3 is enough.
iex(1)> {output, 0} = System.cmd("git", ["rev-parse", "--show-toplevel"])
{"/dwelling/arathunku/code/github.com/arathunku/elixir-cli-template-examplen", 0}
With sample matching we guarantee speedy exit if there’s another exit code than the profitable one, and we are able to course of the output.
It will get extra difficult if you wish to create one other BEAM course of whereas persevering with with the remainder of the execution. If one thing goes mistaken or Erlang system crashes, OS course of would possibly get left behind. In these circumstances, as a substitute of reinveting a wheel of managing OS processes, it’s good event to make a use of this Combine.set up/2 originally and add MuonTrap. It would make sure the processes are, as described in README, well-behaved.
Combine.set up([{:muontrap, "~> 1.0"}])
defmodule Whats up do
...
defp cmd([], []) do
_pid = spawn_link(fn ->
MuonTrap.cmd("ping", ["-i", "5", "localhost"], into: IO.stream(:stdio, :line))
System.cease()
finish)
Course of.sleep(:infinity)
finish
finish
Alerts, some will not be just like the others
It’s difficult and it’ll most likely not behave as you’d anticipate primarily based in your expertise in different languages. Some indicators are dealt with by default by Erlang system, for extra particulars verify nice documentation in PR. For scripting… normally indicators don’t matter that a lot, at the least in my case. If you happen to spawn a GenServer, it’ll obtain terminate/2 for cleanup, assuming light shutdown.
We will nonetheless skip BREAK menu(^C) and exit instantly if we begin with ERL_FLAGS=+B elixir. That is why it’s within the template originally. Another indicators may be cough by swapping default erl_signal_server, however not all of them*. Within the instance above we’ll do exactly that, deal with what we’re it and defer relaxation to the default handler.
*At this second,
INTcan’t be trapped, see this issue
defmodule Alerts do
@behaviour :gen_event
def init(_), do: {:okay, nil}
def handle_event(:sigusr1, state) do
IO.places("USR1, proceed...")
{:okay, state}
finish
def handle_event(:sigterm, _state) do
IO.places("Okay, okay, let me take a second and exit...")
Course of.sleep(3)
System.cease()
finish
def handle_event(sign, state), do: :erl_signal_handler.handle_event(sign, state)
def handle_call(_, state), do: {:okay, :okay, state}
def terminate(motive, _state) do
IO.places("Goodbye! #{examine(motive)}")
finish
finish
with :okay <- :gen_event.swap_handler(
:erl_signal_server, {:erl_signal_handler, []}, {__MODULE__, []}
) do
IO.places("I am going to look ahead to indicators!")
Course of.sleep(:infinity)
else
err ->
IO.warn("One thing went mistaken. err=#{examine(err)}")
finish
Testing
In Rust, we are able to write exams subsequent to the code with #[cfg(test)], and these will run when cargo take a look at is executed. Do you know you are able to do form of an identical factor in Elixir scripts? There’s no magic right here, we have to create a take a look at module and set off ExUnit
if System.get_env("MIX_ENV") == "take a look at" do
ExUnit.begin()
defmodule HelloTest do
use ExUnit.Case, async: true
import ExUnit.CaptureIO
take a look at "prints a message when no arguments are handed" do
assert capture_io(fn -> Exams.principal([]) finish) == "Whats up Worldn"
finish
take a look at "prints assist for unknown arguments" do
assert capture_io(fn -> Exams.principal(["--help"]) finish) =~ "Instance of including ExUnit"
finish
finish
else
Whats up.principal(System.argv())
finish
Going past scripts
OptionParser is sweet. Possibly it doesn’t do all of the stuff that one thing like Rust clap does however it’s completely getting the job finished.
You’ll be able to transcend easy CLI scripts and construct full TUI apps. Progress bars? Charts? Textual content editor? Right here, Ratatouille involves the rescue. Simple TUI counter example or extra superior ones – more advanced examples.
If you happen to’d prefer to see much more examples with Mix.install/2, ensure that to verify mix_install_examples! There’re examples of HTTP servers, CSV parsing, net scraping, machine studying or full Phoenix LiveView file uploader(!!!), however at this level you could think about simply utilizing combine new ... and organising a correct Combine mission. After that, you should use burrito to ship a single Elixir CLI binary for end-users.
Is Elixir extra sophisticated than Ruby od Python as a bash alternative? It relies upon, after all it relies upon. If you happen to already know Python or Ruby effectively, you’ll most likely desire them however Elixir can completely be used too! It’s a bit on the sluggish aspect to begin up, will not be a finest option to implement PS1, but when the startup pace doesn’t matter that a lot and also you need go have very ergonomic language for scripting at your hand – it’s nice.
Closing observe
Writing this text was form of a wierd expertise. Chances are you’ll assume it’s a bit gentle on examples and particulars, and it’s finished on goal. There’s robust deal with documentation inside the neighborhood and I didn’t wish to repeat what’s already on the market in official docs and libraries. Simply take a look at this lovely System.cmd/3 documentation or IO.ANSI docs, and it’s all out there in iex with h/1.
When writing the article, I’ve dumped all my scripts and exams into this repository. Thanks for studying!