← All notes

Architecture is the decisions that are hard to change

Published3 min readarchitecturedesignreliability

Most code can be rewritten next quarter. A few decisions cannot, because data, other teams or customers have grown around them. I treat those as the architecture and spend my time there.

Where state lives

The first question for any feature is who owns the data and where it is written. A booking reserves an item for a few minutes while the customer pays. I could lock a database row for that time, or keep a hold in Redis with an expiry. Redis is cheap to create and release, and an abandoned checkout expires on its own. The cost is that the hold is soft: a payment that lands just after expiry needs explicit handling. I chose the soft hold because locking rows during a human payment flow scales badly. Either choice is defensible. The point is to make it on purpose.

What is allowed to be late

Some work must finish before the user sees a response, and some can finish a moment after. Sending a confirmation email can be late. Marking inventory as held cannot. Pushing a catalog change to search can be late, as long as everyone accepts that a new listing takes a moment to appear. Each late step needs a way to retry, a limit, and a place where permanent failures wait for a person.

What can disagree

If two systems can disagree, decide which one wins and how you will find out. A payment provider and your ledger will drift sometimes. A cached balance and the entries behind it will drift sometimes. The design is not the absence of drift. It is a reconciliation job that finds it, and a rule for who is right.

What must be unrepeatable

Networks repeat messages and users repeat taps. Anything that moves money or changes a lifecycle needs an idempotency story: a key, a deduplication table, or a state machine that ignores a transition it has already made. I add this on the first version, because retrofitting it after a double charge is a bad week.

What the schema promises

A table that other services read becomes an API. A column name in an event payload becomes a contract. I keep events small and versioned, and I keep other services out of my tables. The more boundaries are data you own completely, the more freedom you keep later.

What goes in code and what goes in config

Rates, brackets and rules that change by law belong in reviewed code, with tests, so every change has a history and a result you can reproduce. Values that operations needs to change in an incident, such as a timeout or a feature switch, belong in config. Mixing the two is how a tax rate ends up edited in a dashboard at midnight.

A short checklist

Before I call a design done I can answer these:

  • Where is each piece of state written, and by whom?
  • What happens if this step fails halfway?
  • What happens if this message arrives twice?
  • What happens if it never arrives?
  • How will I know that two systems disagree?
  • Who gets paged, and what do they see?

If I cannot answer one, that is where the next piece of design work goes. The code is the easy part. The architecture is the answers.

Ayomide Ilesanmi, Lagoscommit feec46d · built 2026-11-13 07:18 UTC