A Codebase That Makes Codebases

Could 10, 2023

I’ve spent the final 4 years constructing and sustaining a venture referred to as SaaS Pegasus.
Pegasus is a SaaS boilerplate or SaaS starter kit.
It supplies a beginning codebase for brand spanking new SaaS tasks with a bunch of options like person accounts,
groups, and billing already included.

Most boilerplates are common codebases.
You clone a repository from Github, observe the getting began directions, and also you’re up and working.
What makes Pegasus completely different is that each venture’s codebase is exclusive.
You enter details about your venture and the tech stack you need to use, and Pegasus makes a codebase only for you.
In different phrases, Pegasus is a codebase that makes different codebases.

Making—and extra importantly, sustaining—a codebase-making codebase has posed some distinctive challenges,
and I’ve needed to get fairly inventive to resolve them.
On this put up I am going to define how Pegasus is constructed, the issues I’ve run into, and the tooling that holds all of it collectively.

However first, I would like to handle the apparent query.

Why generate codebases in any respect?

I discussed that almost all boilerplates simply present beginning code and do not trouble with any code-generation enterprise.
Chances are you’ll be considering…would not that be a lot simpler? Why tackle all this further complexity?

The issue is my inside perfectionist.

Once I began engaged on Pegasus, I had two essential design objectives:

The primary was that it must be versatile.
I knew each venture can be completely different and wished to make Pegasus versatile sufficient to deal with a variety of completely different use circumstances.

The second was that I wished builders to find it irresistible.
Pegasus is a product geared in the direction of builders—notoriously an opinionated and finicky bunch of individuals.
For Pegasus to achieve success, I wanted it to create codebases that builders can be completely satisfied to make use of.
For this constraint, I used myself as a litmus take a look at.
Something that I personally would not use was an immediate no-go.

Anyway—let’s first concentrate on flexibility.
To ensure that a boilerplate to be versatile it needs to be configurable.

For instance, many SaaS tasks need to acquire month-to-month funds from their customers.
However many others—inner instruments or passion tasks—will not.
To facilitate each of those use circumstances you’ll want to have some piece of configuration that tells the app
whether or not billing is enabled or not, after which have all the things behave accordingly.

If the boilerplate is “only a codebase”, this configuration needs to be a runtime setting.
You’d set USE_BILLING = True in your settings file, and that might dynamically allow the billing module, UI, and so forth.
That is sometimes how configurable open-source libraries work, and it really works fairly properly.

if settings.USE_BILLING:
    show_billing_module()

Sadly, from a developer expertise perspective—this sucks.

On this world, all generated codebases embody the billing code (and all its dependencies)—even the codebases that by no means use it.
It additionally means plenty of “if USE_BILLING” statements sprinkled all through the codebase that—for
any particular person venture—solely add pointless complexity.
It might be exhausting for builders engaged on the code to know what may safely be deleted, what libraries have been by no means used, and so forth.
This is perhaps acceptable for libraries—since they’re largely handled as a black containers—however
for the beginning code for a brand new app it felt…sloppy.

So I made a decision runtime configuration was out. The purist in me would not permit it.

What shortly grew to become clear was that if Pegasus was going to generate code that happy builders like me,
the generated codebase itself wanted to be configurable.
Then, by the point a developer obtained their fingers on the code, all the additional cruft and branching logic can be gone
and so they may begin with precisely what they wanted.

So that is what I did.

Here is a bit of snippet from the Pegasus codebase creator:

I do know it appears easy, however every of those checkboxes controls an enormous quantity of logic—from
the UI that’s created all the way in which all the way down to the information fashions and required packages.
A few of these even have interdependencies.
^{And Pegasus—the codebase creator—takes on all this complexity in order that the builders utilizing Pegasus are shielded from it.}

Upkeep of this venture has been… fascinating. However over time I’ve managed to provide you with options for many of the massive issues I’ve run into.

With that out of the way in which, let’s peek underneath the hood.

The key to dynamic code era: cookiecutter

At its core, SaaS Pegasus is a cookiecutter venture.
Cookiecutter is an superior little utility that allows you to create tasks from templates.
In my case, Pegasus is the template, and the codebases it produces are the tasks.

Cookiecutter has two major elements:

1. A set of configuration variables. These are the equal of USE_BILLING = True within the instance above.
2. A logic/templating engine that sits on high of these variables (written in Jinja2).
   That is the equal of all of the if USE_BILLING logic, above.

The examples from the venture enhancing UI above produce a cookiecutter configuration dictionary that appears like this:

cookiecutter = {
    "use_teams": "y",
    "use_teams_example": "n",
    "use_translations": "y",
    "use_wagtail": "n",
    "use_openai": "y",
}

Then, you possibly can add Jinja2 markup to any file in your venture to put in writing logic that is dependent upon these variables.

For instance, in Pegasus, I need to embody or exclude sure URL paths relying on whether or not a function is turned on:

urlpatterns = [
    # some urls are always included
    path('admin/', admin.site.urls),
    # but many are turned on/off depending on your configuration
{% if cookiecutter.use_translations == 'y' %}
    path('i18n/', include('django.conf.urls.i18n')),
{% endif %}
{% if cookiecutter.use_teams == 'y' %}
    path('teams/', include('apps.teams.urls')),
{% endif %}
{% if cookiecutter.use_openai == 'y' %}
    path('openai/', include('apps.openai_example.urls')),
{% endif %}
    # and so on...
]

Every of those {% if cookiecutter.use_thing == 'y' %} statements runs in opposition to the generated cookiecutter dictionary
and both retains or leaves out that content material from the venture.

Now, every generated venture may have the proper set of URLs primarily based on the chosen configuration.
The identical trick will be utilized to information fashions, enterprise logic, the UI, dependencies, and so forth—and
no further logic or code will likely be included within the ultimate output!

Dealing with bigger adjustments with cookiecutter hooks

The syntax above works nice for adjustments inside a file, however typically you’ll want to do bigger operations,
e.g. embody or exclude whole information or directories.
For these larger items, cookiecutter supplies hooks that run earlier than/after venture creation.
Cookiecutter’s hooks are Python scripts which have entry to the generated venture listing.
So, for instance, if you wish to delete whole information or directories primarily based on a variable you are able to do that like this:

using_translations="{{ cookiecutter.use_translations }}" == 'y'
if not using_translations:
    # take away locale information and middleware if we aren't utilizing translations
    take away(os.path.be a part of(project_dir, 'locale'))
    take away(os.path.be a part of('apps', 'internet', 'locale_middleware.py'))

Jinja templating and hooks can get you fairly far—virtually all the things I’ve needed to configure in Pegasus will be finished with one in all these two patterns.

How do you template a template?

SaaS Pegasus generates Django tasks—which embody HTML templates with their very own template language that appears similar to Jinja2:

<nav>
{% if person.is_authenticated %}
  <a category="navbar-item" href="{% url 'pegasus_examples:examples_home' %}">
    Examples Gallery
  </a>
{% endif %}
</nav>

This template snippet provides a navigation hyperlink to the instance gallery provided that the person is authenticated.
However what if we have to layer on some cookiecutter logic?
What if we solely need to embody this code within the template if the venture was constructed with examples enabled?

A primary try may look one thing like this:

<nav>
{% if cookiecutter.use_examples == 'y' %} <!-- cookiecutter examine -->
{% if person.is_authenticated %}            <!-- Django template examine -->
  <a category="navbar-item" href="{% url 'pegasus_examples:examples_home' %}">
    Examples Gallery
  </a>
{% endif %}                               <!-- finish Django template examine -->
{% endif %}                               <!-- finish cookiecutter examine -->
</nav>

However there’s an issue.
When cookiecutter is constructing this venture, Jinja2 cannot inform the distinction between the cookiecutter
choices ({% if cookiecutter.use_examples == 'y' %}) and the Django ones ({% if person.is_authenticated %}).
It tries to judge the entire file, however chokes as quickly because it runs into one thing it would not perceive
(on this case, the reference to person.is_authenticated).

With the intention to make it work now we have to inform cookiecutter’s Jinja syntax to disregard the similar-looking Django template syntax.
We will do that with the {% raw %} tag—which stops analysis on all the things inside it.
Here is the up to date instance:

<nav>
{% if cookiecutter.use_examples == 'y' %} <!-- cookiecutter examine -->
{% uncooked %}                                 <!-- pause cookiecutter parsing -->
{% if person.is_authenticated %}            <!-- Django template examine -->
  <a category="navbar-item" href="{% url 'pegasus_examples:examples_home' %}">
    Examples Gallery
  </a>
{% endif %}                               <!-- finish Django template examine -->
{% endraw %}                              <!-- unpause cookiecutter parsing -->
{% endif %}                               <!-- finish cookiecutter examine -->

Now cookiecutter will ignore the troublesome {% if person.is_authenticated %} assertion, and all the things works appropriately.

These templates can get fairly unwieldy when there’s a variety of intermingled cookiecutter and Django template logic!
For kicks, attempt to make sense of the instance under, taken straight from Pegasus’s supply code.

<physique{% endraw %}{% if use_multiple %}{% uncooked %}{% if pg_is_material_bootstrap %} class="g-sidenav-show  bg-gray-200"{% endif %}{% endraw %}{% endif %}{% if use_material %} class="g-sidenav-show  bg-gray-200"{% endif %}{% if using_htmx %}{% uncooked %} hx-headers="{"X-CSRFToken": "{{ csrf_token }}"}"{% endraw %}{% endif %}{% uncooked %}>

It ain’t fairly but it surely works…

Testing code that makes code

Testing a cookiecutter venture presents its personal challenges. The primary one being, merely: how do you take a look at it?

Importantly, cookiecutter tasks themselves aren’t legitimate code. Keep in mind our urls.py file?

urlpatterns = [
    # some urls are always included
    path('admin/', admin.site.urls),
    # but many are turned on/off depending on your configuration
{% if cookiecutter.use_translations == 'y' %}
    path('i18n/', include('django.conf.urls.i18n')),
{% endif %}
    # and so on...
]

This file will crash a Python interpreter the second a {% %} tag is reached.
Sure, when you run cookiecutter it is best to (hopefully!) get legitimate code, however the cookiecutter venture—Pegasus itself—is meaningless nonsense.

So I can not run a take a look at suite on the venture itself.
As an alternative I’ve to run the take a look at suite on a generated venture.
And checks change into a two-stage take a look at course of:

1. Generate a venture codebase from the Pegasus template.
2. Run the checks on the generated venture.

That is simple sufficient to script, however raises a brand new query: What venture(s) must be examined?

Making an attempt to get respectable take a look at protection

There are presently 25 configurable variables you should use in Pegasus.
This implies there are about 2²⁵—or ~33 million—other ways to configure a Pegasus venture. That is a variety of combos!

A lot of the bug experiences I get for Pegasus inevitably find yourself being tied to some distinctive mixture of variables—e.g.
“constructing with out groups or subscriptions, however with API keys enabled causes profile image uploads to fail”.
It is loads to handle.

Within the early days, I would repair the bug, then add an merchandise to my launch guidelines to verify this explicit mixture
of variables nonetheless labored the subsequent time round.
The discharge guidelines began getting fairly lengthy!

Ultimately, because the variety of choices (and customers) grew, so did the variety of combos I needed to take a look at.
And shortly I discovered myself spending whole days simply banging on completely different combos of variables on the lookout for bugs.
I wanted one thing extra sustainable.

Enter: automation. With the assistance of a pal, I turned to Github Actions,
and we managed to arrange a collection of jobs that:

1. Generate an inventory of configurations to check.
2. Generate new tasks primarily based on every configuration.
3. Set up and run the take a look at suite (and anything) on every generated venture.

Now each change to Pegasus will get run by way of ~25 hand-picked construct configurations, every of which ensures {that a} explicit mixture
of options nonetheless works correctly.

And now at any time when I discover a new bug involving a selected mixture of flags—I add a line to the config file to check that exact mixture,
and I can relaxation assured it will not occur once more!

Beneath the hood this works through Github’s matrix support.
The primary job of the workflow creates the matrix of configurations as a JSON output, and the subsequent job makes use of it to setup the venture and run CI on every one.

Testing protection downside (principally) solved!

Automating away complexity with post-processing

So that is the underlying code and testing infrastructure. However, there are nonetheless issues which might be exhausting to do in cookiecutter.

Code formatting, for one, is a nightmare.
All of the cookiecutter Jinja syntax makes it very troublesome to manage whitespace within the generated output.

Additionally compiled information are a large number.
If a cookiecutter variable impacts JavaScript code, managing how that results in a compiled JavaScript bundle is unattainable to do by hand.
Equally, managing a compiled necessities.txt file primarily based off a cookiecuttered
requirements.in file is a large problem.

To unravel these points, I’ve launched a customized post-processing step to the pipeline.
After cookiecutter generates the venture, however earlier than the developer downloads it,
I run quite a lot of issues server-side to place the ending touches on the venture.
Formatting the code with black, working isort, constructing the ultimate necessities.txt file, compiling and bundling the entrance finish, and so forth.

Doing it this fashion helps keep away from having to handle loopy, advanced cookiecutter logic on something that may be automated.

Wrapping up: complexity all the way in which down

On the finish of the day, constructing Pegasus is—like most software program tasks—an train in managing complexity.

My beginning objective was to take away as a lot complexity as attainable from the end-product of Pegasus: the generated codebases.
The less complicated the generated codebases are, the higher Pegasus the product will likely be.

However—the extra complexity I defend from the end-user, the extra I tackle myself, and the more durable the venture is to keep up.
This ultimately slows me down and hurts the product too—if in a much less direct method.

So I additionally should take away complexity for myself—as I’ve finished with cookiecutter, automated testing, post-processing and so forth.

You may name what I am doing complexity-driven growth (CDD) or complexity juggling.
As some space of Pegasus will get too advanced, I prioritize developing with a method to make it simpler—both for my end-users or for myself.
As soon as issues are streamlined, I transfer onto one thing else.

Pegasus nonetheless is not good, however the charge of complexity progress has stabilized.
There are nonetheless a variety of balls within the air, however thus far I am managing to maintain them afloat.

Because of Michael Lynch and Rowena Luk for offering suggestions on a draft of this.

If you wish to discover out after I publish new content material you possibly can subscribe under, or observe me on Twitter.

And if you wish to take an enormous shortcut to launch your subsequent venture, I hope you may think about SaaS Pegasus.
Hopefully this put up has a minimum of satisfied you that it is one thing I work fairly exhausting on.

Notes