Is it Crucial to be Declarative?
Just lately, in Container Options’ engineering Slack channel, a heated argument ensued amongst our engineers after a Pulumi-related story was posted. I received’t recount the lots of of posts within the thread, however the first response was “I nonetheless don’t know why we nonetheless use Terraform”, adopted by a still-unresolved ping-pong debate about whether or not Pulumi is declarative or crucial, adopted by one other debate about whether or not any of this crucial vs declarative stuff actually issues in any respect, and why can’t we simply use Pulumi please?
This text is my try and show that I used to be proper and everybody else was mistaken calmly lay out a few of the points and enable you perceive each what’s occurring and the way to answer your benefit when somebody says your favoured device just isn’t declarative and due to this fact verboten.
What does declarative imply, precisely?
Answering this query is tougher than it seems, because the formal use of the time period can differ from the casual use throughout the trade. So we have to unpick first the formal definition, then have a look at how the time period is utilized in observe.
The formal definition
Let’s begin with the Wikipedia definition of declarative:
“In pc science, declarative programming is a programming paradigm—a method of constructing the construction and components of pc packages—that expresses the logic of a computation with out describing its management circulation.”
This may be lowered to:
“Declarative programming expresses the logic of a computation with out describing its management circulation.”
This instantly begs the query: ‘what’s management circulation?’ Back to Wikipedia:
“In pc science, management circulation (or circulation of management) is the order through which particular person statements, directions or perform calls of an crucial program are executed or evaluated. Inside an crucial programming language, a management circulation assertion is an announcement that ends in a selection being made as to which of two or extra paths to comply with.”
This may be lowered to:
“Crucial packages make a selection about what code is to be run.”
In keeping with Wikipedia, examples of management circulation embrace if statements, loops, and certainly every other assemble that enables adjustments which assertion is to be carried out subsequent (e.g. jumps, subroutines, coroutines, continuations, halts).
Casual utilization and definitions
In debates round tooling, individuals not often stick intently to the formal definitions of declarative and crucial code. Essentially the most generally heard casual definition saw heard is: “Declarative code tells you what to do, crucial code says the way to do it”. It sounds definitive, however dialogue about it shortly devolves into definitions of what ‘what’ means and what ‘how’ means.
Any program tells you ‘what’ to do, in order that’s doubtlessly deceptive, however one interpretation of that’s that it describes the state you wish to obtain.
For instance, by that definition, is that this pseudo-code declarative or crucial?
if exists(ec2_instance_1):
create(ec2_instance_2)
create(ec2_instance_1)Firstly, strictly talking, it’s positively not declarative in line with a proper definition, because the second line might or might not run, so there’s management circulation there.
It’s positively not idempotent, as working as soon as doesn’t essentially lead to the identical final result as working twice. However an argument put to me was: “The result doesn’t change as a result of somebody presses the button a number of occasions”, some kind of ‘ultimately idempotent’ idea. Certainly, a later clarification was: “Declarative means for me: state ultimately constant”.
It’s not simply engineers within the area who don’t cling to the formal definition. This Jenkinsfile documentation describes the usage of conditional constructs while calling itself declarative.
To this point we are able to say that:
- The formal definitions of crucial vs declarative are fairly clear
- In observe and basic dialogue, individuals get a bit confused about what it means and/or don’t care concerning the formal definition
Are there levels of declarativeness?
In concept, no. In observe, sure. Let me clarify.
What’s the most declarative programming language you’ll be able to consider? Whichever one it’s, it’s doubtless that both there’s a method to make it (technically) crucial, or it’s usually described as “not a programming language”.
HTML is so declarative {that a}) individuals usually deride it as “not a programming language in any respect”, and b) we needed to create the JavaScript monster and the SCRIPT tag to ‘escape’ it and make it helpful for extra than simply markup. This is applicable to all pure markup languages. One other oft-cited instance is Prolog, which has loops, situations, and a halt command, so is technically not declarative in any respect.
SQL is to many a canonical declarative language: you describe what knowledge you need, and the database administration system (DBMS) determines how that knowledge is retrieved. However even with SQL you’ll be able to assemble conditionals:
insert into table1
the place exists (
choose 1
from table2
the place "some worth" == table2.column1
)The insert to table1 will solely run conditionally, i.e. if there’s a row in desk two that matches the textual content “some worth”. You would possibly assume that it is a contrived instance, and I received’t disagree. However in a approach this backs up my central argument: regardless of the technical definition of declarative is, the distinction between most languages on this respect is how straightforward or pure it’s to show them into crucial languages.
Now contemplate this YAML, yanked from the web:
job:
script: "echo Whats up, Guidelines!"
guidelines:
- if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "grasp"'
when: at all times
- if: '$VAR =~ /sample/'
when: guide
- when: on_successThat is clearly successfully crucial code. It runs in an order from high to backside, and has conditionals. It might run completely different directions at completely different occasions, relying on the context it’s run in. Nonetheless, YAML itself remains to be declarative. And since YAML is declarative, we’ve the hell of Helm, kustomize, and completely different devops pipeline languages that declare to be declarative (however clearly aren’t) to cope with, as a result of we want crucial, dynamic, conditional, branching methods to precise what we wish to occur.
It’s this stress between the declarative nature of the core device and our instant wants to unravel issues that creates the perverse outcomes we hate a lot as engineers, the place we wish to ‘escape’ of the declarative device as a way to get the issues we wish finished in the way in which that we wish it finished.
Terraform and Pulumi
Which brings us neatly to the unique topic of the Slack dialogue we had at Container Options.
Anybody who has used Terraform for any size of time within the area has most likely gone via two phases. First, they marvel at how the declarative nature of it makes it in some ways simpler to keep up and motive about. And second, after a while utilizing it, and as complexity within the use case builds and builds, they more and more wish they could have access to imperative constructs.
It wasn’t lengthy earlier than Hashicorp responded to those calls for and launched the ‘count’ meta-argument, which successfully gave us some type of loop idea, and hideous bits of code like this abound to offer us if statements by the again door:
depend = var.something_to_do ? 1 : 0
There’s additionally ‘for’ and ‘for_each’ constructs, and the ‘local-exec’ provisioner, which lets you escape any declarative shackles fully and simply drop to the (decidedly non-declarative) shell as soon as the useful resource is provisioned.
It’s usually argued that Pulumi just isn’t declarative, and despite protestations to the contrary, if you’re utilizing it for its major promoting level (that you need to use your most well-liked crucial language to declare your required state), then Pulumi is successfully an crucial device. If you happen to discuss to the declarative engine below Pulumi’s hood in YAML, then you might be declarative all the way in which down (and extra declarative than Terraform, for certain).
The purpose right here is that not being purely declarative isn’t any unhealthy factor, as it could be that your use case calls for a extra crucial language to generate a state illustration. Underneath the hood, that state illustration describes the ‘what’ you wish to do, and the Pulumi engine figures out the way to obtain that for you.
A few of us at Container Options labored some years in the past at a serious establishment that constructed a large-scale venture in Terraform. For numerous causes, Terraform was ditched in favour of a python-based boto3 resolution, and a type of causes was that the restrictions of a extra declarative language produced extra friction than the advantages gained. In different phrases, extra management over the circulation was wanted. It might be that Pulumi was the device we would have liked: A ‘Goldilocks’ device that was the proper mix of crucial and declarative for the job at hand. It might have saved us writing a variety of boto3 code, for certain.
How to answer ‘nevertheless it’s not declarative!’ arguments
Hopefully studying this text has helped make clear the fog round declarative vs crucial arguments. First, we are able to recognise that purely declarative languages are uncommon, and even people who exist are sometimes contorted into successfully crucial tooling. Second, the variations between these instruments is how straightforward or pure they make that contortion.
There are good causes to make it troublesome for individuals to be crucial. Establishing easy Kubernetes clusters generally is a extra repeatable and moveable course of as a consequence of its declarative configuration. When issues get extra complicated, it’s a must to attain for instruments like Helm and kustomize which can make you are feeling like your life has been made tougher.
WIth this extra nuanced understanding, subsequent time somebody makes use of the “nevertheless it’s not declarative” argument to close you down, you’ll be able to inform them two issues: That that assertion just isn’t sufficient to win the controversy; and that their recommended various is probably going both not declarative, or not helpful. The vital query just isn’t: “Is it declarative?” however relatively: ‘How declarative do we want it to be?”
