Structure diagrams must be code

For the previous few years I’ve been probably the most senior developer on my groups in Atlassian, in each place (Principal Engineer) and time (nearly 9 years) – this implies I often tackle the duty of managing our software program structure.

Structure is the relationships between programs, which might be pretty tough to speak about. In all probability the perfect type of communication is a diagram, with packing containers representing programs (or parts) and features representing relationships between them. This may nonetheless have points.

When my earlier engineering supervisor joined the Atlassian Market workforce, he requested everybody to attract an structure diagram. Every got here out extraordinarily totally different. The folks centered on frontend had issues like Jira, Embedded Market, Commerce, Provisioning, Atlassian Join, then an arrow to an enormous field – simply labelled Market backend.

The folks centered on backend had the reverse – an enormous field simply labelled frontend!

Precise relationships do exist between programs (e.g. community calls, shared storage) however an structure diagram can’t give all particulars with out turning into the code it’s meant to symbolize. Which means all structure diagrams are views into an abstraction. It’s not fallacious to have an enormous field representing your backend; it’s only one means of viewing an abstraction.

Some groups and tasks could have a diagram and level to it as “right here is the structure” – nevertheless it’s not the structure, it’s only one explicit view.

Ideally, we’d have dozens of structure diagrams – from varied views, from varied proposals, from varied groups. We don’t see this very a lot and I believe a part of the issue is that structure diagrams might be expensive.

Most diagramming instruments require you to make use of the mouse, pointing and clicking and dragging and drawing. They are often very fiddly and even buggy; e.g. there’s one structure diagram for Atlassian Market with an previous outdated field that may’t be deleted utilizing the software program it was diagrammed in, it simply received’t enable that, for some purpose.

If somebody provides a relationship to a brand new system, will they even bear in mind to go to the Confluence web page to click on and drag and draw over the structure diagrams?

Perhaps we should always write structure diagrams utilizing code as a substitute. With code, we are able to replace structure diagrams inside a pull request, model them and shortly modify a lot of them directly.

We will additionally add some formality. As an alternative of modelling issues with traces and packing containers, we are able to immediately mannequin programs and relationships.

The C4 Model formalises structure into 4 abstractions: system context, containers, parts and code. I solely use system context and containers; parts and code are too detailed. A container is one thing which runs, corresponding to an utility server, database, filesystem, and many others. A system represents some group of software program, and it may have containers which that software program is deployed inside.

There are a bunch of instruments for utilizing C4. There are two which you might need already used for different issues, corresponding to PlantUML and Mermaid. One of many above pictures was generated from this PlantUML code:

@startuml
!embody  https://uncooked.githubusercontent.com/plantuml-stdlib/C4-PlantUML/grasp/C4_Container.puml

title Market from backend view

AddElementTag("spotlight", $bgColor="orange")

System(market, "Market")
System(commerce, "Commerce")
System(hams, "HAMS")
System(identification, "Atlassian Identification")
System(pako, "Folks Atlassian Is aware of Of")
System(frontend, "Market FRONTEND", $tags="spotlight")
System(algolia, "Algolia")
System(upm, "Common Plugin Supervisor")

Rel(market, commerce, "Fetches pricing")
Rel(market, hams, "Fetches legacy pricing")
Rel(market, identification, "Fetches customers")
Rel(market, pako, "Queries experiences")
Rel(market, algolia, "Searches")
Rel(frontend, market, "Queries")
Rel(upm, market, "Queries")

SHOW_LEGEND()

@enduml

However as a substitute of writing PlantUML, I take advantage of Haskell to mannequin structure. A easy instance would appear to be:

data AtlassianSystem
  = Marketplace
  | Identity
  | Pako
  deriving (Eq, Ord, Show)

data AtlassianTechnology
  = REST
  | SQL
  deriving (Eq, Ord, Show)

marketplaceSystem ::
  SoftwareSystem
    AtlassianSystem
    ()
    AtlassianTechnology
    ()
marketplaceSystem =
  softwareSystem
    & relationships . at Identity ?~ relationship REST
    & relationships . at Pako ?~ relationship SQL

marketplace ::
  Model
    ()
    ()
    AtlassianSystem
    ()
    AtlassianTechnology
    ()
marketplace =
  model
    & softwareSystems . at Marketplace ?~ marketplaceSystem
    & softwareSystems . at Identity ?~ softwareSystem
    & softwareSystems . at Pako ?~ softwareSystem

We can also generate a model from running tests. Each test can generate a sequence diagram using something like DataFlow. Gathering the programs utilized in all assessments can generate a C4 system context and container diagram for our entire system. Right here’s an instance sequence diagram generated by operating a single check:

I’m fairly satisfied that we must be utilizing code to generate structure diagrams. Code permits making fast adjustments. Structure code might be versioned with the code which implements it. We will write algorithms to examine our structure. I’d wish to see extra instruments obtainable to explain structure as a part of code, permitting us to generate as many diagrams as we would like, for correct and simple communication.

Do you employ code to diagram structure? Do you do one thing different than simply write PlantUML or Mermaid, by hand? I’d love to listen to extra!