Five Books, One System: Architecture from Decisions to Reliable Production

ARCCLOUD / THE ARCHITECT'S WORKSHOP / 02

Everything worked on Thursday. On Friday, two people got the same time slot.

The team added an inventory integration, improved the form, and shipped a release. The pipeline was green. In the morning, an advisor accepted two work orders for the same crew. One already had a part reserved; the other had already sent the customer a confirmation. Every piece of code did something reasonable. The process as a whole no longer added up.

At moments like this, architecture stops being a diagram of boxes. It becomes an answer to specific questions: who can confirm an appointment, who owns the reservation, and what have we promised the person on the other side of the screen?

I have chosen five books that help us work through these questions in the right order. We will start with requirements, look at the code, bring an existing system into shape, and finally check whether we can keep it running after deployment. Each book offers a different perspective. What they share is responsibility for the consequences of our decisions.

Requirements · Patterns in code · Refactoring · Boundaries and events · Reliability · Test your decisions

01 / FUNDAMENTALS OF SOFTWARE ARCHITECTURE

Good design starts with a problem we can name.

Mark Richards and Neal Ford bring structure to architectural styles and system quality attributes. Their starting point is simple: an architectural decision comes at a cost. Greater deployment independence may make faults harder to trace. Simpler transactions may mean bundling some changes into a single deployment. Read this book as practice in explaining why you made a choice. The publisher's description of the second edition.

At North Service, three people develop a single application. Most support tickets concern appointments and parts reservations. That is reason enough to start with the rules of the process. Knowing only that the company wants “modern architecture” does not yet tell us which solution will help.

Only now does a discussion of architectural styles become meaningful. For this company, we choose to start with a modular monolith: one deployment with explicit boundaries of responsibility. We gain simpler startup and the ability to use local transactions. We also accept the cost of a shared release and the need to keep dependencies between modules under control.

This choice fits the conditions described here. We will revisit it when a separate team starts developing scheduling or its workload differs substantially from the rest of the application. A good architecture document also states what should trigger a fresh review.

02 / PATTERNS OF ENTERPRISE APPLICATION ARCHITECTURE

A pattern helps only when it matches the complexity of the rules.

Martin Fowler, together with his coauthors, describes recurring problems in business applications: organizing logic, persisting data, and updating information concurrently. The book's age does not diminish the value of its questions. In practice, it helps to distinguish a simple Transaction Script from a richer Domain Model and to understand the boundary of a Unit of Work. The author's book page.

Suppose an administrator changes the description of a service bay. Check permissions, validate the length, save: a simple case can remain a simple transaction script. We do not need to build an elaborate model around it.

Confirming a repair is harder. We need an approved quote, parts, and a crew. If these conditions are checked separately in a controller, an overnight job, and an integration, a few months later we will have three different definitions of a “confirmed work order”. It is worth bringing the rule together in a single model.

public enum WorkOrderState { Draft, Confirmed }

public sealed record ConfirmationEvidence(
    bool QuoteAccepted,
    bool PartsReserved,
    bool TechnicianReserved);

public sealed class WorkOrder
{
    public WorkOrderState State { get; private set; }

    public void Confirm(ConfirmationEvidence evidence)
    {
        if (State == WorkOrderState.Confirmed) return;
        if (!evidence.QuoteAccepted || !evidence.PartsReserved ||
            !evidence.TechnicianReserved)
            throw new InvalidOperationException("Confirmation is incomplete.");

        State = WorkOrderState.Confirmed;
    }
}

This snippet illustrates one rule, rather than a complete production workflow. In a running application, reservation evidence must come from a trusted contract and refer to the correct work order and version. We must also handle reservation expiry, permissions, and concurrent writes. Three Boolean values sent by a browser cannot serve as that evidence.

A unit test verifies that the model rejects incomplete conditions. An integration test runs two concurrent writes and checks the database's concurrency mechanism. These provide two different kinds of assurance. Testing the class alone will not expose a double booking made by two processes.

Unit of Work helps group the changes that belong to a single write operation. It does not automatically extend the transaction to an external inventory system, a broker, and a mail server. It is useful to draw that boundary before the code starts crossing it unintentionally. Brief pattern descriptions: Transaction Script, Domain Model and Unit of Work.

03 / DOMAIN-DRIVEN REFACTORING

Untangle the meanings first. Then separate the code.

Alessandro Colla and Alberto Acerbis show how to apply DDD to a system that already exists: from domain language and contexts to changes in structure and the database. The direction of the work matters: improve boundaries gradually while keeping the product working. A modular system can be a good outcome of refactoring; extracting a microservice needs its own justification. The book on the publisher's website.

At our company, “ready” means something different to three people. The advisor means an agreed price, the warehouse worker means a part set aside, and the technician means a completed repair. A single Status column does not remove that difference. It hides it until the first unusual case.

We start defining boundaries by talking to the people doing the work. We ask about a specific customer complaint: what happened, who made the decision, and what fact was missing at the time? A useful model grows out of these conversations. Folder names should then reflect that model.

How do you change an old system without a major rewrite?

1. Choose one painful scenario. For example, confirming an appointment. Write down the rule with the person responsible for the process.
2. Protect the behavior you need with tests. Establish which current outcomes are correct. Capturing an old bug in a test does not make it a requirement.
3. Introduce a single access path. The work order code requests a reservation from scheduling through a contract. It does not change scheduling tables directly.
4. Transfer responsibility for writes. Define a cutover point and a single owner. Two independent mechanisms writing the same data will make discrepancies harder to detect.
5. Compare the results and remove the old path. Check historical data, edge cases, and the rollback plan. Only then remove unnecessary dependencies.

A shared database can remain at this stage. Data ownership means responsibility for changes, rules, and the contract; it does not automatically mean a separate database server. In a modular monolith, you can start with schemas, type visibility, and dependency tests. Where stronger isolation is needed, add appropriate permissions and execution boundaries.

04 / SOFTWARE ARCHITECTURE: THE HARD PARTS

When a module becomes a service, the network enters the conversation.

Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Dehghani examine difficult decisions about service size, data, contracts, and distributed processes. Reach for this book when several reasonable solutions come with different, significant costs. It teaches you to compare tradeoffs in the context of a specific system. The publisher's description of the book.

Suppose scheduling now has its own team and integrates with several partners. A separate service may give that team deployment freedom. At the same time, an ordinary method call becomes an operation that may be delayed, execute twice, or end without a response. The business rule still applies, but the conditions for enforcing it have changed.

Outbox closes a specific gap. The full process still needs several more decisions.

If we save a work order and then send an event, the process may terminate between those operations. The record remains; the message never arrives. The transactional outbox pattern saves the business change and the intent to publish in the same local transaction. A separate mechanism publishes the event later. The broker or publisher can still deliver a duplicate, so the consumer must be prepared for it. Outbox explained in the Azure Architecture Center.

At the consumer, recording the handled event's identifier and making the business change should form a single local transaction, with a uniqueness constraint on the consumer–EventId pair. Simply checking “have I seen this before?” ahead of the write is not enough when messages arrive concurrently. If the effect is a call to an external API, safe retries need a separate solution, such as an idempotency key supported by the provider.

We also need an ordering policy. An event with version 8 must not silently roll back a projection that already knows version 9. We can check versions and handle gaps, or use an ordering mechanism for each work order. The choice depends on the broker, the processing approach, and whether ordering is required for a single object or a wider group. Globally ordering everything often costs more than it delivers.

Sagas and compensation: what can we really undo?

Once inventory and scheduling have been extracted, we no longer have a single transaction covering the whole process. We can coordinate the local steps with a saga. If reserving a crew fails, we carry out the agreed recovery actions: release the part, record the outcome, and inform the operations staff. Compensation can itself fail, so it needs durable state, retries, and a path for manual intervention. The Saga pattern in Microsoft documentation.

We cannot turn back time or unread a message. “Cancelled” must have a meaning agreed with the business. If the state of an external reservation is unknown after a timeout, we do not immediately mark it as failed. First, we reconcile the outcome. The interface should accurately show that the request is pending or needs review, rather than promise completion.

05 / SITE RELIABILITY ENGINEERING

Architecture must also help the person who answers the phone after an incident.

The book edited by Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy shows how to connect product development with responsibility for keeping it running. SLIs, SLOs, and error budgets offer a useful starting point: what we measure, what level we consider sufficient, and how we respond when we fall below it. The official free edition of the book.

For North Service, we set an example objective: in a rolling 28-day window, at least 99.5% of properly accepted work orders receive a correct, durable outcome within 120 seconds. The outcome may be a confirmation or a justified rejection. Calling an infrastructure failure a “rejection” does not make it a correct outcome. We agree on the definition before we measure.

This budget counts operations, rather than minutes of downtime. Retries of the same operation should not increase the denominator. We must also decide how to treat operations accepted just before the end of the observation period that have not yet had the full 120 seconds to complete. Without that definition, two teams can honestly calculate different results. SLIs and SLOs according to Google SRE.

A green health endpoint alone cannot verify this objective. We need events for the start and end of the process, with matching identifiers. As supporting indicators, we monitor the age of the oldest outbox message, consumer lag, the number of states requiring review, and failed compensations. These signals should help us find the cause, rather than replace measurement of the user experience.

We tie the budget to a response agreed in advance. If the error rate suggests the budget will soon be exhausted, the team restricts risky releases, investigates the cause, and improves reliability. Emergency fixes and security changes receive a separate risk assessment. The specific policy belongs to the product and its team; a useful starting point is the example policy in The Site Reliability Workbook.

What should be kept alongside the code?

• Ownership map. Who leads an incident and who knows each part of the process.
• Runbook. How to find a work order by its identifier, recognize a safe retry, and route a case for review.
• Recovery plan. A database backup and a rehearsed procedure for restoring it and reconciling integration state. A “backup completed” status alone is not a recovery test.
• Deployment conditions. Contract and migration compatibility, a controlled rollout, and a measurable condition for halting the release.
• Lessons from the incident. What made diagnosis difficult, which safeguard failed, and how we will check whether the fix works.

Returning to a previous application version does not automatically reverse data changes or effects in an external system. The contingency plan therefore needs to be tested against a scenario with a realistic sequence of operations, rather than solely against an empty test database.

WORKSHOP / FROM UNDERSTANDING TO DECISIONS

Arrange the flow. Then defend five decisions.

In the first exercise, select the steps in the correct order. We assume a single local write of the work order with an outbox and a separate consumer. Messages may be redelivered. The exercise result stays in this browser tab.

After reading, choose one decision to improve in your system.

You do not need to implement every pattern at once. Describe one process: what it accepts, where it records the truth, how it handles a retry, and how you recognize completion. Mark the places where the answer is currently a guess. A page like this often leads to a better work plan than a list of new technologies.

Reading order: Fundamentals will help you name requirements and tradeoffs. Fowler will help you check which patterns fit the logic. Domain-Driven Refactoring is useful for organizing existing code, The Hard Parts for separating services and data, and SRE for turning promises into measurement and everyday practice. Return to the relevant chapter when a specific decision arises.

Books and sources for further work

This original guide draws on topics covered in the listed publications and documentation. The links lead to authors, publishers, and official materials. You do not need to buy the books to complete the exercises.

1. Fundamentals of Software Architecture, 2nd Edition — Mark Richards, Neal Ford.
2. Patterns of Enterprise Application Architecture — Martin Fowler, with the coauthors listed on the book's page.
3. Domain-Driven Refactoring — Alessandro Colla, Alberto Acerbis.
4. Software Architecture: The Hard Parts — Neal Ford, Mark Richards, Pramod Sadalage, Zhamak Dehghani.
5. Site Reliability Engineering — edited by Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy.

Also: Saga, Transactional Outbox, Service Level Objectives and Error Budget Policy.