A proposal for a graphy pipe-based construct system

But one other construct system?

Hear me out! This one is straightforward!! No want so as to add separate help for various languages, we simply write what instructions we need to execute.

Yep! However what if we add mappings and collections?

This submit is only a proposal and a few concepts that I brainstormed to make a construct system that’s straightforward to make use of for any language, with out having to be taught something sophisticated.

The identify I bear in mind for that is Piper.

Motivation

I used to be lately experimenting with a software that transformed one file into one other, after which that new file needed to be aggregated with recordsdata of the identical kind within the arguments of one more command.

It appears sophisticated, but it surely’s solved simply with a bash script:

#!/bin/bash
shopt -s nullglob  # Prevents terrible globbing behaviour if no recordsdata match
for file in *.ext; do
  echo "Processing $file"
  ./convert -o "${file%.ext}.new" "$file"
achieved
echo "Aggregating recordsdata"
./last -o last *.new

So why do we want one thing new?

There are a couple of downsides of rawdogging it with bash:

• No parallelism until you strive actually actually laborious. I’m not bothered to do it whereas writing this instance.
• You’d be insane so as to add incremental compilation.
• No solution to visualise the construct graph.
• It’s not cross-platform.
• It’s laborious to learn.

These are issues that may be dealt with for you very simply with a syntax similar to Make — with out its pain-points.

Observations

We will see that there are two sorts of transformations:

• Mappings: 1-1 transformations, like changing a file to a different file.
• Collections: N-1 transformations, like aggregating recordsdata into one.

Whereas it’s true that there is usually a 1-N transformation in a single command, I’ve discovered that it isn’t quite common and this is usually a later addition. I’ll reserve one thing alongside the strains of Turbines for these transformations.

Alas, let’s prototype some syntax!

Syntax

Instructions

Like Make, we’ll make ident: signify a runnable command. Nonetheless, in contrast to Make, we is not going to be placing different instructions in physique. As a substitute, we’ll declare the collection of mappings and collections to carry out on teams of recordsdata.

E.g:

construct:
  .c --[CC]-> .o --{AR}-> lib.a

Observe:

• We use [ident] to signify a mapping and {ident} to signify a group.
• We use .ext as a shorthand for glob *.ext.

It’s essential to additionally present a method for the person to specify customized glob patterns, however we’re in 2023 so lets do it with regexes. The above is equal to:

construct:
  r(.*.c) --[CC]-> r(.*.o) --{AR}-> lib.a

As a result of CC is a mapping, it auto infers that the output filename is similar, simply with a special extension. A full filename is required for the output of collections.

Mapping & Assortment Definitions

[ident]: and {ident}: respectively:

[CC]:
  cc -o $_OUT $_IN

{AR}:
  ar rcs $_OUT $_IN

Observe:

• Variables start with $, extra on that within the subsequent part.
• Particular “inserted” variables are prefixed with $_.

These are the one inserted guidelines:

• $_OUT: The output filename, together with file extension.
• $_IN: The enter filename(s), together with file extension.
  • For collections, it is a house separated listing of filenames.
• $_{0..9999}: Regex group matches, if the enter is a regex.
  • $_0 is all the match, at all times equal to $_IN. You’ll be able to consider this as due to the outer brackets within the regex expressions above.
  • In collections, every variable is an area separated listing of the respective group.

Variables

Variables start with a $. They’re outlined as such:

$A = it is a literal
$B = "it is a literal"

What about variables which might be the output of a command?

We will use the $(command) syntax:

$C = $(echo "it is a literal")

$A, $B and $C are similar.

Observe: Variables can solely be substituted. In essence, they’re lazily evaluated. For those who reference $C twice, will probably be executed twice.

Variables are scoped. For those who outline a variable in a command, it should solely reside in the course of that command.

You should use variables to additionally outline file sorts, e.g.

$SRC = .c
$OBJ = r(.*.o)

I’m nonetheless debating internally if $A needs to be allowed as there can probably be a battle with $SRC.

Conclusion

I imagine this is usually a quite simple, however highly effective, construct description language.

• Parallel jobs might be simply decided by each the person and the software.
• Incremental compilation is trivial resulting from every mapping having a single enter and output, with the output file identify being recognized earlier than it’s generated.
• It’s intuitive, and simple to learn.
• It may possibly most undoubtedly be cross-platform
  • Though, it is a little bit of a moot level, all of it is dependent upon the construct instructions used. Solely pointing this out as a result of bash is just not cross-platform.
• No want for a software to generate a graph – it’s proper there! Though the suitable software program can nonetheless be written.

That is only a “proposal”, I haven’t written any software program for this, and my opinions could change. I haven’t spent a lot time on this in any respect and I’m positive there are lots of issues I’ve missed, and plenty of issues that might be improved. I’d love to listen to your ideas!

— Matthew

Feedback