Now Reading
Writing Small CLI Applications in Widespread Lisp / Steve Losh

Writing Small CLI Applications in Widespread Lisp / Steve Losh

2023-09-05 20:40:12

Posted on March seventeenth, 2021.

I write plenty of command-line packages. For tiny packages I normally go along with the
typical UNIX method: throw collectively a half-assed shell script and transfer on.
For giant packages I make a full Widespread Lisp mission, with an ASDF system
definition and such. However there is a center floor of smallish packages that
do not warrant a full repository on their very own, however for which I nonetheless need an actual
interface with correct --help and error dealing with.

I’ve discovered Widespread Lisp to be a great language for writing these small command
line packages. However it may be a little bit intimidating to get began (particularly
for inexperienced persons) as a result of Widespread Lisp is a really versatile language and does not lock
you into a technique of working.

On this publish I am going to describe how I write small, stand-alone command line packages
in Widespread Lisp. It would give you the results you want, otherwise you would possibly wish to modify issues to
match your individual wants.

  1. Requirements
  2. Solution Skeleton
    1. Directory Structure
    2. Lisp Files
    3. Building Binaries
    4. Building Man Pages
    5. Makefile
  3. Case Study: A Batch Coloring Utility
    1. Libraries
    2. Package
    3. Configuration
    4. Errors
    5. Colorization
    6. Not-Quite-Top-Level Interface
    7. User Interface
    8. Top-Level Interface
  4. More Information

Once you’re writing packages in Widespread Lisp, you have received plenty of choices.
Laying out the necessities I’ve helped me determine on an method.

First: every new program needs to be one single file. A couple of different information for the
assortment as an entire (e.g. a Makefile) are okay, however as soon as every part is ready
up creating a brand new program ought to imply including one single file. For bigger
packages a full mission listing and ASDF system are nice, however for small
packages having one file per program reduces the psychological overhead fairly a bit.

The packages want to have the ability to be developed within the typical Widespread Lisp
interactive type (in my case: with Swank and VLIME). Interactive growth
is likely one of the finest elements of working in Widespread Lisp, and I am not prepared to present
it up. Specifically which means a shell-script type method, with
#!/path/to/sbcl --script and the highest and straight operating code on the high
degree within the file, does not work for 2 primary causes:

  • loading that file will fail as a result of shebang except you’ve got some ugly
    reader macros in your startup file.
  • This system might want to do issues like parsing command-line arguments and
    exiting with an error code, and calling exit would kill the Swank course of.

The packages want to have the ability to use libraries, so Quicklisp will should be
concerned. Widespread Lisp has plenty of good issues built-in, however there are some
libraries which might be simply too helpful to move up.

The packages might want to have correct person interfaces. Command line arguments
should be robustly parsed (e.g. collapsing -a -b -c foo -d into -abcfoo -d
ought to work as anticipated), malformed or unknown choices should be caught as an alternative of
dropping them on the ground, error messages needs to be significant, and the
--help needs to be completely and thoughtfully written so I can bear in mind the way to
use this system months later. A man web page is a pleasant bonus, however not required.

Counting on some primary conventions (e.g. a command foo is all the time in foo.lisp
and defines a bundle foo with a perform referred to as toplevel) is okay if it
makes my life simpler. These packages are only for me, so I haven’t got to fret
about individuals eager to create executables with areas within the title or one thing.

Portability between Widespread Lisp implementations is sweet to have, however not
required. If utilizing a little bit of SBCL-specific grease will let me keep away from a bunch of
additional dependencies, that is advantageous for these small private packages.

After making an attempt various completely different approaches I’ve settled on an answer that
I am fairly proud of. First I am going to describe the final method, then we’ll
take a look at one precise instance program in its entirety.

Directory Structure

I preserve all my small single-file Widespread Lisp packages in a lisp listing
inside my dotfiles repository. Its contents seem like this:

…/dotfiles/lisp/
    bin/
        foo
        bar
    man/
        man1/
            foo.1
            bar.1
    build-binary.sh
    build-manual.sh
    Makefile
    foo.lisp
    bar.lisp

The bin listing is the place the executable information find yourself. I’ve added it to my
$PATH so I haven’t got to symlink or copy the binaries anyplace.

man comprises the generated man pages. As a result of it is adjoining to bin (which
is on my path) the man program routinely finds the man pages as
anticipated.

build-binary.sh, build-manual.sh, and Makefile are some glue to make
constructing packages simpler.

The .lisp information are the packages. Every new program I wish to add solely
requires including the <programname>.lisp file on this listing and operating
make.

Lisp Files

My small Widespread Lisp packages observe a number of conventions that make constructing them
simpler. Let’s take a look at the skeleton of a foo.lisp file for instance. I am going to
present the whole file right here, after which step by way of it piece by piece.

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload '(:with-user-abort) :silent t))

(defpackage :foo
  (:use :cl)
  (:export :toplevel *ui*))

(in-package :foo)

(defparameter *no matter* 123)

(define-condition user-error (error) ())

(define-condition missing-foo (user-error) ()
  (:report "A foo is required, however none was provided."))

(defun foo (string))

(defun run (arguments)
  (map nil #'foo arguments))

(defmacro exit-on-ctrl-c (&physique physique)
  `(handler-case (with-user-abort:with-user-abort (progn ,@physique))
     (with-user-abort:user-abort () (sb-ext:exit :code 130))))

(defparameter *ui*
  (undertake:make-interface
    :title "foo"))

(defun toplevel ()
  (sb-ext:disable-debugger)
  (exit-on-ctrl-c
    (multiple-value-bind (arguments choices) (undertake:parse-options-or-exit *ui*)(handler-case (run arguments)
        (user-error (e) (undertake:print-error-and-exit e))))))

Let’s undergo every chunk of this.

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload '(:with-user-abort) :silent t))

First we quickload any mandatory libraries. We all the time wish to do that, even
when compiling the file, as a result of we want the suitable packages to exist when
we attempt to use their symbols later within the file.

with-user-abort is a library for simply dealing with control-c, which all of
these small packages will use.

(defpackage :foo
  (:use :cl)
  (:export :toplevel *ui*))

(in-package :foo)

Subsequent we outline a bundle foo and swap to it. The bundle is all the time named
the identical because the ensuing binary and the basename of the file, and all the time
exports the symbols toplevel and *ui*. These conventions make it straightforward to
construct every part routinely with make later.

(defparameter *no matter* 123)

Subsequent we outline any configuration variables. These can be set later after
parsing the command line arguments (once we run the command line program) or
on the REPL (when growing interactively).

(define-condition user-error (error) ())

(define-condition missing-foo (user-error) ()
  (:report "A foo is required, however none was provided."))

We outline a user-error situation, and any errors the person would possibly make will
inherit from it. This may make it straightforward to deal with person errors (e.g. passing
a mangled common expression like (foo+ as an argument) in another way from
programming errors (i.e. bugs). This makes it simpler to deal with these errors
in another way:

  • Bugs ought to print a backtrace or enter the debugger.
  • Anticipated person errors ought to print a useful error message with no backtrace or debugger.
(defun foo (string))

Subsequent now we have the precise performance of this system.

(defun run (arguments)
  (map nil #'foo arguments))

We outline a perform run that takes some arguments (as strings) and performs
the principle work of this system.

Importantly, run does not deal with command line argument parsing, and it does
not exit this system with an error code, which suggests we are able to safely name it to
say “run the entire program” once we’re growing interactively with out worrying
about it killing our Lisp course of.

Now we have to outline the command line interface.

(defmacro exit-on-ctrl-c (&physique physique)
  `(handler-case (with-user-abort:with-user-abort (progn ,@physique))
     (with-user-abort:user-abort () (undertake:exit 130))))

We’ll make a little bit macro round with-user-abort to make it much less wordy. We’ll
exit with a status of 130 if the
person presses ctrl-c. Perhaps some day I am going to pull this into Undertake so I haven’t got
to repeat these three traces all over the place.

(defparameter *ui*
  (undertake:make-interface
    :title "foo"))

Right here we outline the *ui* variable whose image we exported above. Adopt is
a command line argument parsing library I wrote. If you wish to use a unique
library, be at liberty.

(defun toplevel ()
  (sb-ext:disable-debugger)
  (exit-on-ctrl-c
    (multiple-value-bind (arguments choices) (undertake:parse-options-or-exit *ui*)(handler-case (run arguments)
        (user-error (e) (undertake:print-error-and-exit e))))))

And eventually we outline the toplevel perform. This may solely ever be referred to as
when this system is run as a standalone program, by no means interactively. It
handles all of the work past the principle guts of this system (that are dealt with by
the run perform), together with:

  • Disabling or enabling the debugger.
  • Exiting the method with an acceptable standing code on errors.
  • Parsing command line arguments.
  • Setting the values of the configuration parameters.
  • Calling run.

That is it for the construction of the .lisp information.

Building Binaries

build-binary.sh is a small script to construct the executable binaries from the
.lisp information. ./build-binary.sh foo.lisp will construct foo:

#!/usr/bin/env bash

set -euo pipefail

LISP=$1
NAME=$(basename "$1" .lisp)
shift

sbcl --load "$LISP" 
     --eval "(sb-ext:save-lisp-and-die "$NAME"
               :executable t
               :save-runtime-options t
               :toplevel '$NAME:toplevel)"

Right here we see the place the naming conventions have change into necessary — we all know that
the bundle is known as the identical because the binary and that it’ll have the image
toplevel exported, which all the time names the entry level for the binary.

Building Man Pages

build-manual.sh is analogous and builds the man pages utilizing Adopt‘s
built-in man web page technology. In case you do not care about constructing man pages
on your private packages you’ll be able to ignore this. I admit that producing man
pages for these packages is a little bit bit foolish as a result of they’re just for my very own
private use, however I get it at no cost with Undertake, so why not?

#!/usr/bin/env bash

set -euo pipefail

LISP=$1
NAME=$(basename "$LISP" .lisp)
OUT="$NAME.1"
shift

sbcl --load "$LISP" 
     --eval "(with-open-file (f "$OUT" :route :output :if-exists :supersede)
               (undertake:print-manual $NAME:*ui* :stream f))" 
     --quit

For this reason we all the time title the Undertake interface variable *ui* and export it
from the bundle.

Makefile

Lastly now we have a easy Makefile so we are able to run make to regenerate any
outdated binaries and man pages:

information := $(wildcard *.lisp)
names := $(information:.lisp=)

.PHONY: all clear $(names)

all: $(names)

$(names): %: bin/% man/man1/%.1

bin/%: %.lisp build-binary.sh Makefile
    mkdir -p bin
    ./build-binary.sh $<
    mv $(@F) bin/

man/man1/%.1: %.lisp build-manual.sh Makefile
    mkdir -p man/man1
    ./build-manual.sh $<
    mv $(@F) man/man1/

clear:
    rm -rf bin man

We use a wildcard to routinely discover the .lisp information so we do not have to
do something additional after including a brand new file once we wish to make a brand new program.

Probably the most notable line right here is $(names): %: bin/% man/man1/%.1 which makes use of
a static pattern rule
to routinely outline the phony guidelines for constructing every program. If
$(names) is foo bar this line successfully defines two phony guidelines:

foo: bin/foo man/man1/foo.1
bar: bin/bar man/man1/bar.1

This lets us run make foo to make each the binary and man web page for
foo.lisp.

Now that we have seen the skeleton, let us take a look at one in every of my precise packages that
I exploit on a regular basis. It is referred to as batchcolor and it is used to spotlight common
expression matches in textual content (normally log information) with a twist: every distinctive match
is highlighted in a separate shade, which makes it simpler to visually parse the
outcome.

For instance: suppose now we have some log information with traces of the shape <timestamp>
[<request ID>] <degree> <message>
the place request ID is a UUID, and messages would possibly
include different UUIDs for numerous issues. Such a log file would possibly look one thing
like this:

2021-01-02 14:01:45 [f788a624-8dcd-4c5e-b1e8-681d0a68a8d3] INFO Incoming request GET /customers/28b2d548-eff1-471c-b807-cc2bcee76b7d/issues/7ca6d8d2-5038-42bd-a559-b3ee0c8b7543/
2021-01-02 14:01:45 [f788a624-8dcd-4c5e-b1e8-681d0a68a8d3] INFO Factor 7ca6d8d2-5038-42bd-a559-b3ee0c8b7543 shouldn't be cached, retrieving...
2021-01-02 14:01:45 [f788a624-8dcd-4c5e-b1e8-681d0a68a8d3] WARN Person 28b2d548-eff1-471c-b807-cc2bcee76b7d doesn't have entry to factor 7ca6d8d2-5038-42bd-a559-b3ee0c8b7543, denying request.
2021-01-02 14:01:46 [f788a624-8dcd-4c5e-b1e8-681d0a68a8d3] INFO Returning HTTP 404.
2021-01-02 14:01:46 [bea6ae06-bd06-4d2a-ae35-3e83fea2edc7] INFO Incoming request GET /customers/28b2d548-eff1-471c-b807-cc2bcee76b7d/issues/7ca6d8d2-5038-42bd-a559-b3ee0c8d7543/
2021-01-02 14:01:46 [bea6ae06-bd06-4d2a-ae35-3e83fea2edc7] INFO Factor 7ca6d8d2-5038-42bd-a559-b3ee0c8d7543 shouldn't be cached, retrieving...
2021-01-02 14:01:46 [b04ced1d-1cfa-4315-aaa9-0e245ff9a8e1] INFO Incoming request POST /customers/sign-up/
2021-01-02 14:01:46 [bea6ae06-bd06-4d2a-ae35-3e83fea2edc7] INFO Returning HTTP 200.
2021-01-02 14:01:46 [b04ced1d-1cfa-4315-aaa9-0e245ff9a8e1] ERR Error operating SQL question: connection refused.
2021-01-02 14:01:47 [b04ced1d-1cfa-4315-aaa9-0e245ff9a8e1] ERR Returning HTTP 500.

If I attempt to simply learn this straight, it is easy for my eyes to glaze over except
I laboriously stroll line-by-line.

Screenshot of uncolored log output

I might use grep to spotlight the UUIDs:

grep -P 
    'b[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}b' 
    instance.log

Sadly that does not actually assist an excessive amount of as a result of all of the UUIDs are
highlighted the identical shade:

Screenshot of grep-colored log output

To get a extra readable model of the log, I exploit batchcolor:

batchcolor 
    'b[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}b' 
    instance.log

batchcolor additionally highlights matches, nevertheless it highlights every distinctive match in its
personal shade:

Screenshot of batchcolored log output

That is a lot simpler for me to visually parse. The interleaving of separate
request logs is now apparent from the colours of the IDs, and it is easy to match
up numerous person IDs and factor IDs at a look. Did you even discover that the 2
factor IDs have been completely different earlier than?

batchcolor has a number of different high quality of life options, like choosing express
colours for particular strings (e.g. purple for ERR):

Screenshot of fully batchcolored log output

I exploit this explicit batchcolor invocation so usually I’ve put it in its personal
tiny shell script. I exploit it to tail log information when growing regionally virtually
day by day, and it makes visually scanning the log output a lot simpler. It may
turn out to be useful for different kinds of textual content too, like highlighting nicknames in an IRC
log.

Let’s step by way of its code piece by piece.

Libraries

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload '(:undertake :cl-ppcre :with-user-abort) :silent t))

First we quickload libraries. We’ll use Adopt for command line argument
processing, cl-ppcre for normal expressions, and the previously-mentioned
with-user-abort to deal with control-c.

Package

(defpackage :batchcolor
  (:use :cl)
  (:export :toplevel :*ui*))

(in-package :batchcolor)

We outline and swap to the appropriately-named bundle. Nothing particular right here.

Configuration

(defparameter *begin* 0)
(defparameter *darkish* t)

Subsequent we defparameter some variables to carry some settings. *begin* can be
used later when randomizing colours, don’t fret about it for now.

Errors

(define-condition user-error (error) ())

(define-condition missing-regex (user-error) ()
  (:report "An everyday expression is required."))

(define-condition malformed-regex (user-error)
  ((underlying-error :initarg :underlying-error))
  (:report (lambda (c s)
             (format s "Invalid regex: ~A" (slot-value c 'underlying-error)))))

(define-condition overlapping-groups (user-error) ()
  (:report "Invalid regex: appears to include overlapping capturing teams."))

(define-condition malformed-explicit (user-error)
  ((spec :initarg :spec))
  (:report
    (lambda (c s)
      (format s "Invalid express spec ~S, should be of the shape "R,G,B:string" with colours being 0-5."
              (slot-value c 'spec)))))

Right here we outline the person errors. A few of these are self-explanatory, whereas
others will make extra sense later as soon as we see them in motion. The particular
particulars aren’t as necessary as the general concept: for person errors we all know would possibly
occur, show a useful error message as an alternative of simply spewing a backtrace at
the person.

Colorization

Subsequent now we have the precise meat of this system. Clearly that is going to be
fully completely different for each program, so be at liberty to skip this when you do not
care about this particular drawback.

(defun rgb-code (r g b)
      (+ (* r 36)
     (* g 6)
     (* b 1)
     16))

We will spotlight completely different matches with completely different colours. We’ll want
an affordable quantity of colours to make this handy, so utilizing the fundamental 8/16 ANSI
colours is not sufficient. Full 24-bit truecolor is overkill, however the 8-bit ANSI
colours will work properly. If we ignore the bottom colours, we basically have
6 x 6 x 6 = 216 colours to work with. rgb-code will take the purple, inexperienced, and
blue values from 0 to 5 and return the colour code. See Wikipedia
for extra data.

(defun make-colors (excludep)
  (let ((outcome (make-array 256 :fill-pointer 0)))
    (dotimes (r 6)
      (dotimes (g 6)
        (dotimes (b 6)
          (except (funcall excludep (+ r g b))
            (vector-push-extend (rgb-code r g b) outcome)))))
    outcome))

(defparameter *dark-colors*  (make-colors (lambda (v) (< v 3))))
(defparameter *light-colors* (make-colors (lambda (v) (> v 11))))

Now we are able to construct some arrays of colours. We might use any of the 216 out there
colours, however in follow we in all probability do not wish to, as a result of the darkest colours
can be too darkish to learn on a darkish terminal, and vice versa for gentle terminals.
In a concession to practicality we’ll generate two separate arrays of colours,
one which excludes colours whose complete worth is simply too darkish and one excluding these
which might be too gentle.

(Discover that *dark-colors* is “the array of colours that are appropriate to be used
on darkish terminals” and never “the array of colours that are themselves darkish”.
Naming issues is tough.)

Be aware that these arrays can be generated when the batchcolor.lisp file is
loaded, which is once we construct the binary. They will not be recomputed each
time you run the ensuing binary. On this case it does not actually matter (the
arrays are small) nevertheless it’s value remembering in case you ever have some information you
need (or don’t desire) to compute at construct time as an alternative of run time.

(defparameter *explicits* (make-hash-table :take a look at #'equal))

Right here we make a hash desk to retailer the strings and colours for strings we wish to
explicitly shade (e.g. ERR needs to be purple, INFO cyan). The keys would be the
strings and values the RGB codes.

(defun djb2 (string)
    (cut back (lambda (hash c)
            (mod (+ (* 33 hash) c) (expt 2 64)))
          string
          :initial-value 5381
          :key #'char-code))

(defun find-color (string)
  (gethash string *explicits*
           (let ((colours (if *darkish* *dark-colors* *light-colors*)))
             (aref colours
                   (mod (+ (djb2 string) *begin*)
                        (size colours))))))

For strings that we wish to explicitly shade, we simply lookup the suitable
code in *explicits* and return it.

In any other case, we wish to spotlight distinctive matches in several colours. There are
various other ways we might do that, for instance: we might randomly decide
a shade the primary time we see a string and retailer it in a hash desk for
subsequent encounters. However this may imply we would develop that hash desk over time,
and one of many issues I usually use this utility for is tail -fing long-running
processes when growing regionally, so the reminiscence utilization would develop and develop till
the batchcolor course of was restarted, which is not ultimate.

As a substitute, we’ll hash every string with a easy DJB hash and use it to
index into the suitable array of colours. This ensures that similar matches
get similar colours, and avoids having to retailer each match we have ever seen.

There can be some collisions, however there’s not a lot we are able to do about that with
solely ~200 colours to work with. We might have used 16-bit colours like
I discussed earlier than, however then we would have to fret about choosing colours completely different
sufficient for people to simply inform aside, and for this easy utility I did not
wish to hassle.

We’ll speak about *begin* later, ignore it for now (it is 0 by default).

See Also

(defun ansi-color-start (shade)
  (format nil "~C[38;5;~Dm" #Escape color))

(defun ansi-color-end ()
  (format nil "~C[0m" #Escape))

(defun print-colorized (string)
  (format *standard-output* "~A~A~A"
          (ansi-color-start (find-color string))
          string
          (ansi-color-end)))

Next we have some functions to output the appropriate ANSI escapes to highlight
our matches. We could use a library for this but it’s only two lines. It’s
not worth it
.

And now we have the beating heart of the program:

(defun colorize-line (scanner line &aux (start 0))
  (ppcre:do-scans (ms me rs re scanner line)
            (let* ((regs? (plusp (length rs)))
           (starts (if regs? (remove nil rs) (list ms)))
           (ends   (if regs? (remove nil re) (list me))))
      (map nil (lambda (word-start word-end)
                 (unless (<= start word-start)
                   (error 'overlapping-groups))
                 (write-string line *standard-output* :start start :end word-start)
                 (print-colorized (subseq line word-start word-end))
                 (setf start word-end))
           starts ends)))
  (write-line line *standard-output* :start start))

colorize-line takes a CL-PPCRE scanner and a line, and outputs the line with
any of the desired matches colorized appropriately. There are a few things to
note here.

First: if the regular expression contains any capturing groups, we’ll only
colorize those parts of the match. For example: if you run batchcolor
'^<(w+)> '
to colorize the nicks in an IRC log, only the nicknames themselves
will be highlighted, not the surrounding angle brackets. Otherwise, if there
are no capturing groups in the regular expression, we’ll highlight the entire
match (as if there were one big capturing group around the whole thing).

Second: overlapping capturing groups are explicitly disallowed and
a user-error signaled if we notice any. It’s not clear what do to in this
case — if we match ((f)oo|(b)oo) against foo, what should the output be?
Highlight f and oo in the same color? In different colors? Should the oo
be a different color than the oo in boo? There’s too many options with no
clear winner, so we’ll just tell the user to be more clear.

To do the actual work we iterate over each match and print the non-highlighted
text before the match, then print the highlighted match. Finally we print any
remaining text after the last match.

Not-Quite-Top-Level Interface

(defun run% (scanner stream)
  (loop :for line = (read-line stream nil)
        :while line
        :do (colorize-line scanner line)))

(defun run (pattern paths)
  (let ((scanner (handler-case (ppcre:create-scanner pattern)
                   (ppcre:ppcre-syntax-error (c)
                     (error 'malformed-regex :underlying-error c))))
        (paths (or paths '("-"))))
    (dolist (path paths)
      (if (string= "-" path)
        (run% scanner *standard-input*)
        (with-open-file (stream path :direction :input)
          (run% scanner stream))))))

Here we have the not-quite-top-level interface to the program. run takes
a pattern string and a list of paths and runs the colorization on each path.
This is safe to call interactively from the REPL, e.g. (run "<(w+)>"
"foo.txt")
, so we can test without worrying about killing the Lisp process.

User Interface

In the last chunk of the file we have the user interface. There are a couple of
things to note here.

I’m using a command line argument parsing library I wrote myself: Adopt.
I won’t go over exactly what all the various Adopt functions do. Most of them
should be fairly easy to understand, but check out the Adopt
documentation
for the full story if you’re curious.

If you prefer another library (and there are quite a few around) feel free
to use it — it should be pretty easy to adapt this setup to a different library.
The only things you’d need to change would be the toplevel function and the
build-manual.sh script (if you even care about building man pages at all).

You might also notice that the user interface for the program is almost as much
code as the entire rest of the program. This may seem strange, but I think it
makes a certain kind of sense. When you’re writing code to interface with an
external system, a messier and more complicated external system will usually
require more code than a cleaner and simpler external system. A human brain is
probably the messiest and most complicated external system you’ll ever have to
deal with, so it’s worth taking the extra time and code to be especially careful
when writing an interface to it.

First we’ll define a typical -h/--help option:

(defparameter *option-help*
  (adopt:make-option 'help
    :help "Display help and exit."
    :long "help"
    :short #h
    :reduce (constantly t)))

Next we’ll define a pair of options for enabling/disabling the Lisp debugger:

(adopt:defparameters (*option-debug* *option-no-debug*)
  (adopt:make-boolean-options 'debug
    :long "debug"
    :short #d
    :help "Enable the Lisp debugger."
    :help-no "Disable the Lisp debugger (the default)."))

By default the debugger will be off, so any unexpected error will print
a backtrace to standard error and exit with a nonzero exit code. This is the
default because if I add a batchcolor somewhere in a shell script, I probably
don’t want to suddenly hang the entire script if something breaks. But we still
want to be able to get into the debugger manually if something goes wrong.
This is Common Lisp — we don’t have to settle for a stack trace or core dump, we
can have a real interactive debugger in the final binary.

Note how Adopt’s make-boolean-options function creates two options here:

  • -d/--debug will enable the debugger.
  • -D/--no-debug will disable the debugger.

Even though disabled is the default, it’s still important to have both
switches for boolean options like this. If someone wants the debugger to be
enabled by default instead (along with some other configuration options), they
might have a shell alias like this:

alias bcolor="batchcolor --debug --foo --bar"

But sometimes they might want to temporarily disable the debugger for a single
run. Without a --no-debug option, they would have to run the vanilla
batchcolor and retype all the other options. But having the --no-debug
option allows them to just say:

bcolor --no-debug

This would expand to:

batchcolor --debug --foo --bar --no-debug

The later option wins, and the user gets the behavior they expect.

Next we’ll define some color-related options. First an option to randomize the
colors each run, instead of always picking the same color for a particular
string, and then a toggle for choosing colors that work for dark or light
terminals:

(adopt:defparameters (*option-randomize* *option-no-randomize*)
  (adopt:make-boolean-options 'randomize
    :help "Randomize the choice of color each run."
    :help-no "Do not randomize the choice of color each run (the default)."
    :long "randomize"
    :short #r))

(adopt:defparameters (*option-dark* *option-light*)
  (adopt:make-boolean-options 'dark
    :name-no 'light
    :long "dark"
    :long-no "light"
    :help "Optimize for dark terminals (the default)."
    :help-no "Optimize for light terminals."
    :initial-value t))

The last option we’ll define is -e/--explicit, to allow the user to select
an explicit color for a particular string:

(defun parse-explicit (spec)
  (ppcre:register-groups-bind
      ((#'parse-integer r g b) string)
      ("^([0-5]),([0-5]),([0-5]):(.+)$" spec)
    (return-from parse-explicit (cons string (rgb-code r g b))))
  (error 'malformed-explicit :spec spec))

(defparameter *option-explicit*
  (undertake:make-option 'express
    :parameter "R,G,B:STRING"
    :assist "Spotlight STRING in an express shade.  Could also be given a number of instances."
    :guide (format nil "~
      Spotlight STRING in an express shade as an alternative of randomly selecting one.  ~
      R, G, and B should be 0-5.  STRING is handled as literal string, not a regex.  ~
      Be aware that this does not routinely add STRING to the general regex, you ~
      should do this your self!  This can be a recognized bug that could be fastened sooner or later.")
    :lengthy "express"
    :quick #e
    :key #'parse-explicit
    :cut back #'undertake:gather))

Discover how we sign a malformed-explicit situation if the person offers us
mangled textual content. This can be a subtype of user-error, so this system will print the
error and exit even when the debugger is enabled. We additionally embrace a barely extra
verbose description within the man web page than the terse one within the --help textual content.

Subsequent we write the principle assist and guide textual content, in addition to some real-world
examples:

(undertake:define-string *help-text*
  "batchcolor takes an everyday expression and matches it in opposition to normal ~
   enter one line at a time.  Every distinctive match is highlighted in its personal shade.~@
   ~@
   If the common expression comprises any capturing teams, solely these elements of ~
   the matches can be highlighted.  In any other case the whole match can be ~
   highlighted.  Overlapping capturing teams usually are not supported.")

(undertake:define-string *extra-manual-text*
  "If no FILEs are given, normal enter can be used.  A file of - stands for ~
   normal enter as effectively.~@
   ~@
   Overlapping capturing teams usually are not supported as a result of it is not clear what ~
   the outcome needs to be.  For instance: what ought to ((f)oo|(b)oo) spotlight when ~
   matched in opposition to 'foo'?  Ought to it spotlight 'foo' in a single shade?  The 'f' in ~
   one shade and 'oo' in one other shade?  Ought to that 'oo' be the identical shade as ~
   the 'oo' in 'boo' despite the fact that the general match was completely different?  There are too ~
   many potential behaviors and no clear winner, so batchcolor disallows ~
   overlapping capturing teams totally.")

(defparameter *examples*
  '(("Colorize IRC nicknames in a chat log:"
     . "cat channel.log | batchcolor '<(w+)>'")
    ("Colorize UUIDs in a request log:"
     . "tail -f /var/log/foo | batchcolor '[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}'")
    ("Colorize some key phrases explicitly and IPv4 addresses randomly (be aware that the key phrases should be in the principle regex too, not simply within the -e choices):"
     . "batchcolor 'WARN|INFO|ERR|(?:[0-9]{1,3}.){3}[0-9]{1,3}' -e '5,0,0:ERR' -e '5,4,0:WARN' -e '2,2,5:INFO' foo.log")
    ("Colorize earmuffed symbols in a Lisp file:"
     . "batchcolor '(?:^|[^*])([*][-a-zA-Z0-9]+[*])(?:$|[^*])' checks/take a look at.lisp")))

Lastly we are able to wire every part collectively in the principle Undertake interface:

(defparameter *ui*
  (undertake:make-interface
    :title "batchcolor"
    :utilization "[OPTIONS] REGEX [FILE...]"
    :abstract "colorize regex matches in batches"
    :assist *help-text*
    :guide (format nil "~A~2%~A" *help-text* *extra-manual-text*)
    :examples *examples*
    :contents (record
                *option-help*
                *option-debug*
                *option-no-debug*
                (undertake:make-group 'color-options
                                  :title "Shade Choices"
                                  :choices (record *option-randomize*
                                                 *option-no-randomize*
                                                 *option-dark*
                                                 *option-light*
                                                 *option-explicit*)))))

All that is left to do is the top-level perform that can be referred to as when the
binary is executed.

Top-Level Interface

Earlier than we write toplevel we have a few helpers:

(defmacro exit-on-ctrl-c (&physique physique)
  `(handler-case (with-user-abort:with-user-abort (progn ,@physique))
     (with-user-abort:user-abort () (undertake:exit 130))))

(defun configure (choices)
  (loop :for (string . rgb) :in (gethash 'express choices)
        :do (setf (gethash string *explicits*) rgb))
  (setf *begin* (if (gethash 'randomize choices)
                  (random 256 (make-random-state t))
                  0)
        *darkish* (gethash 'darkish choices)))

Our toplevel perform appears very like the one within the skeleton, however fleshed out
a bit extra:

(defun toplevel ()
  (sb-ext:disable-debugger)
  (exit-on-ctrl-c
    (multiple-value-bind (arguments choices) (undertake:parse-options-or-exit *ui*)
      (when (gethash 'debug choices)
        (sb-ext:enable-debugger))
      (handler-case
          (cond
            ((gethash 'assist choices) (undertake:print-help-and-exit *ui*))
            ((null arguments) (error 'missing-regex))
            (t (destructuring-bind (sample . information) arguments
                 (configure choices)
                 (run sample information))))
        (user-error (e) (undertake:print-error-and-exit e))))))

This toplevel has a number of additional bits past the skeletal instance.

First, we disable the debugger instantly, after which re-enable it later if the
person asks us to. We wish to preserve it disabled till after argument parsing
as a result of we won’t know whether or not the person needs it or not till we parse the
arguments.

As a substitute of simply blindly operating run, we examine for --help and print it if
desired. We additionally validate that the person passes the right amount of arguments,
signaling a subtype of user-error if they do not. Assuming every part appears
good we deal with the configuration, name run, and that is it!

Operating make generates bin/batchcolor and man/man1/batchcolor.1, and we
can view our log information in stunning shade.

I hope this overview was useful. This has labored for me, however Widespread Lisp is
a versatile language, so if you wish to use this structure as a place to begin and
modify it on your personal wants, go for it!

If you wish to see some extra examples you’ll find them in my dotfiles
repository
. A few of the extra
enjoyable ones embrace:

  • climate for displaying the climate over the subsequent few hours so I can inform if
    I want a jacket or umbrella earlier than I am going out for a stroll.
  • retry to retry shell instructions in the event that they fail, with choices for what number of instances
    to retry, methods for ready/backing off on failure, and so forth.
  • decide to interactively filter the output of 1 command into one other
    (impressed by the decide program in “The UNIX Programming Setting” however with
    extra choices).

The method I specified by this publish works effectively for small, single-file packages.
In case you’re creating a bigger program you will in all probability wish to transfer to a full ASDF
system in its personal listing/repository. My pal Ian wrote a post about
that
which you
would possibly discover attention-grabbing.

Source Link

What's Your Reaction?
Excited
0
Happy
0
In Love
0
Not Sure
0
Silly
0
View Comments (0)

Leave a Reply

Your email address will not be published.

2022 Blinking Robots.
WordPress by Doejo

Scroll To Top