a contemporary different to Makefile

I meant to put in writing this put up for some time now, however I by no means thought many individuals would learn it. I am writing it now for these few folks courageous sufficient to attempt one thing new round automation instruments.

After being sad with Makefile for years now, final yr I made a decision that I had sufficient and I began trying round for alternate options. Not one of the initiatives I discovered appeared to suit my wants till I found Taskfile. After utilizing it efficiently for the previous yr, I am glad to say that I discovered a greater approach to write automation scripts than Makefile and I am not going again.

On this put up, I want to introduce Taskfile and some different instruments that mixed make a pleasant automation framework. My intention will not be for this put up to be an exhaustive information to these instruments. Individuals can learn the official documentation if they’re .

I simply need to introduce some primary options that I exploit day-after-day and possibly spark the reader’s curiosity. Simply use 20% of these instruments’ options to reap 80% of the advantages. Thanks, Pareto to your principle.

That is simply my first put up on this matter. I am planning to put in writing quickly a extra in-depth one the place I describe my present use circumstances.

What I do not get is how we make a lot progress in AI, knowledge science, and machine studying, however we’re nonetheless caught previously at any time when it involves automation and developer instruments. How can we innovate quick sufficient in AI (or every other fields) if know-how hasn’t caught up with our fashionable wants for developer instruments?

As anybody that writes or interacts with software program nowadays, I must run automation scripts day-after-day however I’m nonetheless caught with a instrument that (in line with Wikipedia) has been written 47 years in the past. Do not get me unsuitable, Makefile was actually helpful 20 years in the past however I feel we will do higher than that in 2023.

My tackle why persons are nonetheless utilizing Makefile?

Everybody bought used to the truth that Makefile is the one different out there for the standard (not language-specific like Grunt or Gulp) job runner and there’s nothing they will do about it.

Nicely, not everybody. Somebody determined that that they had sufficient of Makefile and wrote another in Golang known as Taskfile.

Like everybody right here, I used to simply write makefiles or easy bash scripts after which prolonged Readme in Markdown to elucidate the way to run these scripts or the way to set up all of the required instruments vital.

It was a protracted and excruciating, however vital, course of not only for different folks however principally for myself. I’ve a really unhealthy reminiscence, and I are likely to neglect what was that command argument or the script that I run final week. Issues do not get higher with age, I needed to discover a higher different.

Additionally, as an engineer, I hate writing documentation:

• It’s a tedious course of.

• It will get outdated the minute that you just end writing it.

• There aren’t any refactoring instruments for a Readme in Markdown that hold your documentation updated together with your code.

• Documentation will not be executable. I imply there are instruments now that will let you run code from Markdown documentation however they aren’t a widespread observe.

As an alternative of writing documentation as an after-math, if I run a shell command that I feel I would use sooner or later, I add it to my now massive listing of duties, make it reusable, and fix a one-line remark to the code, if I want it. My automation framework makes it very straightforward to select it up subsequent time after I must run that command once more.

In an effort to make these automation scripts reusable, I separate the state (the half which may change at every use case) from the reusable code (the half that’s mounted each time).

How do I try this? I exploit a mix of three totally different instruments: Taskfile, Direnv, Devenv.

My documentation is far shorter now, I nonetheless embed notes on the way to use these scripts however I have a tendency to put in writing them subsequent to the code as feedback. The proximity of documentation and code makes the shelf lifetime of my docs loads longer.

I simply need to level out, that this isn’t a easy setup. In an effort to make this occur, I’m utilizing three totally different instruments and I needed to do fairly some experimentation. Throughout this time, I ran into many bugs (a few of which at the moment are mounted) and have lacking as a result of new nature of these instruments and the very fact they aren’t broadly used.

Having stated that, I see the event of these instruments getting into the best path and the variety of Github stars rising exponentially. I do know, GitHub stars as a metric for adoption will not be the whole lot, however that is what we’re caught with.

Additionally, you do not have to undergo the identical ache. You’ll be able to simply reap the good thing about my expertise.

Moreover, the educational curve of these instruments is kind of light and you may simply begin with the fundamentals and add extra stuff over time. I am not even saying that it’s best to undertake my full framework. When you do not feel snug you possibly can simply begin enjoying with every single instrument in isolation.

I wish to assume that by combining these three someway small instruments I’m following the instructing of the Unix Philosophy, which I wish to paraphrase to

And this described right here is strictly it, a really highly effective framework, that when appropriately setup, can will let you construct a fancy pipeline of duties

Till now, I have been calling it Taskfile, however in actuality, that is the identify of the config file utilized by the instrument Process. Given how generic is the phrase job, I imagine that is a greater approach to determine the instrument. I imply, it’s sort of the identical story with Make the instrument and its format Makefile.

The official definition of Process from taskfile.dev is the next:

Process is a job runner / construct instrument that goals to be easier and simpler to make use of than, for instance, GNU Make.

Some fascinating details about Process:

• Written in Go

• Single binary with no different dependencies

• Taskfile is only a dialect of YAML format with a particular syntax

• 8k stars on GitHub on the time of writing

Between the options of this instrument we will discover:

• You’ll be able to construct a pipeline of duties in parallel or in sequence, name different duties from different Taskfiles, and have duties that run as dependencies.

• Duties will be uncovered (aka public) or inner (aka non-public)

• You’ll be able to run a job’s cleanup utilizing Go’s defer command

• Stop pointless work (just like how Makefile works)

• Reference setting variables already outlined within the shell setting

• Duties will be templated utilizing Go’s template engine

• Dry run mode

The listing of options goes on and on. Extra on the official docs.

Right here it’s a easy Taskfile taken from taskfile.dev

model: '3'

duties:
  construct:
    cmds:
      - go construct -v -i most important.go

  belongings:
    cmds:
      - esbuild --bundle --minify css/index.css > public/bundle.css

Please check with the official documentation for the way to set up the instrument and to see pattern use of these options.

The right one-liner to explain Direnv will be discovered of their official documentation direnv.net

direnv – unclutter your .profile

after which:

direnv is an extension to your shell. It augments current shells with a brand new characteristic that may load and unload setting variables relying on the present listing

Some noticeable details about Direnv:

• Single binary

• Written in Golang

• Integrates together with your shell (eg. bash, zsh, fish, and extra)

• It helps 12 factor apps the place you retailer your configs in setting variables or config recordsdata

• 10k begins on GitHub on the time of writing

I principally use it to outline setting variables on .env recordsdata after which load them into my shell routinely after I cd right into a listing with a .envrc file.

For instance, if I’ve a .envrc with the next content material

dotenv

dotenv ../golang/.env

I can load setting variables each from a .env within the present listing and likewise from a file at ../golang/.env.

This fashion of splitting setting variables into totally different recordsdata permits for higher reusability over the long run. When you solely want Golang in your subsequent mission, you possibly can simply deliver with you golang/.env file.

Devenv might be probably the most difficult of those three instruments but additionally probably the most highly effective. With “solely” 2.2k stars on GitHub it’s nonetheless within the early phases of its life however it’s already very highly effective.

The one-liner from the official documentation states

See Also

Are compostable espresso cups really any higher for the setting?

Use a easy unified configuration to configure packages, processes, providers, scripts, git hooks, integrations.

Devenv is predicated on Nix a robust bundle supervisor and system configuration instrument, that comes with its personal language. I am not certified to elucidate what Nix is, and this isn’t even the best place for it. Suffice it to say, I do not know the way to write Nix code and I did not must be taught it to date. I am solely utilizing it right here by way of a higher-level abstraction.

Devenv deserves a complete put up to explain all its options and use circumstances. Right here I am solely utilizing it to explain dependencies (within the form of command line instruments) that I want to have put in on my laptop computer.

{ pkgs, ... }:

{
	packages = [ 
	    pkgs.git 
	    pkgs.govulncheck
	    pkgs.gofumpt 
	    pkgs.go-swag
	    pkgs.gocyclo
	];
}

On this instance, I exploit to put in a couple of command line instruments that I exploit to put in writing Golang functions.

This file is sufficient to inform what dependencies I’ve in my mission, Devenv will ensure that these are put in in an remoted and reproducible setting.

It’s lastly time to place these three initiatives collectively and clarify how I exploit them in my framework.

Construction of the mission

Right here Taskfile.yml is simply the entry level to all different taskfiles. Duties are cut up into a number of recordsdata to attain a pleasant separation of considerations:

model: '3'

consists of:
  docker: taskfiles/docker.taskfile.yml
  go: taskfiles/go.taskfile.yml

Right here we’ve got a golang.taskfile.yml used to construct a Golang utility

model: '3'

duties:  
  construct:
    cmds:
      - GOOS={{.CMD_GOOS}} GOARCH={{.CMD_GOARCH}} go construct -o construct/hiya cmd/hiya.go

and a docker.taskfile.yml used to construct a Docker picture from the Golang binary and a Dockerfile

model: '3'

consists of:
  go: taskfiles/go.taskfile.yml

duties:  
  construct:
    deps:
      - go:construct
    cmds:
      - docker construct -t {{.DOCKER_IMAGE}}:{{.DOCKER_TAG}} -f Dockerfile .

Right here we’ve got a file envs/golang/.env used to outline the setting variable for the Golang utility

CMD_GOOS=linux
CMD_GOARCH=arm64

and one other file envs/docker/.env used as an alternative for Docker setting variables

DOCKER_IMAGE=hiya
DOCKER_TAG=v1.0

Moreover we’ve got a file envs/docker/.envrc that tides collectively the 2 .env recordsdata and cargo setting variables from each

dotenv

dotenv ../golang/.env

Lastly a file devenv.nix used to put in Taskfile and Golang

{ pkgs, ... }:

{
	packages = [ 
	    pkgs.go-task
	];

	languages.go.allow = true;
}

Devenv comes with extra recordsdata than simply devenv.nix. Right here we have not mentioned about them since they do not actually need human intervention.

To ensure that this framework to work, there are a few assumptions:

• Devenv and Direnv must be put in

• Taskfile might be put in by way of devenv however alternatively you possibly can set up it by way of brew

• We assume that you’ve already put in Docker in your laptop computer.

• No want to put in Golang

If you wish to construct the docker picture you solely must run the next instructions

cd envs/docker
job docker:construct

Altering the listing will instruct Direnv to load the setting variables from the relative .env file, whereas the second command will construct the docker picture after having constructed the binary from the supply code.

It’s that straightforward!

Now you will have a reusable framework to construct container pictures to your Golang functions. You’ll be able to copy the content material of those recordsdata to your Golang utility and simply change the content material of the .env recordsdata with out touching anything.

I do know this can be a small synthetic mission, however I hope you possibly can see the facility of this framework and broaden it to your use case.

When you spend sufficient time with this framework, you’re going to have the ability to run automation scripts that may prevent hours of handbook toil.

I am sorry for the very lengthy put up however I could not make it shorter. There was loads to unpack and I could not miss the chance so as to add these memes.

I am going to most likely write a extra elaborate put up the place I present some samples of how I’m utilizing this automation framework for actual use circumstances.

I hope that you’ll undertake a type of instruments or all of them. When you discover any of these helpful, please star them on GitHub to drive adoption.