System architecture without chaos: decisions, teams and evidence that change makes sense

A practical architecture workshop: when it is needed, who should lead it, how to decide, document boundaries and take over an existing system. Includes a mind map, code, questions and a decision game.

The full article is available below. Enable JavaScript to use quizzes and simulations.

An architecture workshop without theater

Good architecture begins when we stop guessing.

Before choosing a diagram, framework or microservices, answer a simple but difficult question: what will change frequently, what must remain stable and what must not break in the second deployment?

Architecture works when a company can change its product without panic and the team knows where responsibility lies. Payments, forms, customer panels, email and reports should not be one tangled mass nobody wants to touch before the weekend.

What architecture does

Defines boundaries, records decisions, reduces change costs and tells the team what should happen when something fails.

What architecture does not do

It does not replace thinking, testing, responsibility or business conversations. Documentation alone will not keep production running.

When it becomes urgent

When one fix touches many modules and nobody can predict the side effects.

A definition that works in practice

Architecture consists of decisions that are difficult to reverse.

You can change a framework, button color or endpoint name. It is much harder to change the data model, responsibility boundaries, payment integration, authorization policy, deployment pipeline or observability approach.

If reversing a decision is expensive, treat it as architectural. If not, avoid a major committee meeting. This rule protects projects from both rule-free chaos and bureaucracy without value.

Mind map A good architectural decision
ProblemWhat actually hurts the business?
DataWhat is the source of truth?
BoundaryWhere does the module's responsibility end?
FailureWhat do we do when the provider goes silent?
EvidenceHow will we verify it works?
RecoveryHow do we return to a stable state?

What is needed

Five things that turn architecture from a drawing into a working approach.

business goal process boundaries decision owner evidence of behavior recorded reason for the choice

The missing ingredient is often responsibility rather than technology. A team knows it “needs an integration”, but nobody records who owns retries, what happens on duplicates, where errors go, who accepts the result or when the work is complete.

Architecture should therefore be brief, readable and tied to a real flow. If a customer buys an audit, the system must know what payment means, when to run the measurement, how to save the report, how to notify the customer and what to show if a stage fails.

When is light structure enough, and when is formal architecture needed?

Situation Enough to begin with Must not be omitted
Marketing website Component standards, SEO, forms and analytics. Content backups, form security and error monitoring.
Customer panel Clear roles, sessions, logs and change auditing. Permissions, data retention, password reset and E2E tests.
Checkout and payments A separate ordering and delivery flow. Webhook idempotency, signatures, refunds and payment status.
ERP/API integrations Contracts, queues, retries and data mapping. Duplicate handling, DLQ, data consistency and versioning.
Regulated system ADRs, security review, access control and audit trails. GDPR, data processing arrangements, retention and incident procedures.

People and responsibility

Who should introduce architecture, and how many people should participate?

One poor model is a person inventing the architecture and handing it to the team as unquestionable truth. Another is a twelve-person meeting where everyone has an opinion and nobody owns the decision.

Most projects need a small group: the business-goal owner, the technical owner of backend and data, someone responsible for frontend or user experience, an operations/security representative and, where needed, a domain expert in sales, accounting, logistics or customer service.

3–4 peoplesmall application, one domain, quick decisions
5–6 peoplecustomer panel, payments, integrations, several roles
6–8 peoplesensitive data, multiple systems, auditing and formal acceptance

More people do not necessarily improve architecture. They often slow decisions and blur responsibility. A small group that can decide, record the reason and verify the result is better.

Challenge 01

A customer pays for an audit, but analysis may take several minutes. What do you do?

Run the entire audit in the Stripe webhook.

Save the payment and record an event in the outbox.

Show “report coming soon” without a queue.

Choose a decision and see its consequence.

A production example

A small feature can become expensive when its boundaries are wrong.

Imagine a simple flow: the customer provides a website address, pays for an audit, the system analyzes it, stores the result, sends an email and shows a report in the panel. Easy on a slide. Production raises immediate questions: what if Stripe sends the webhook twice, email fails, analysis takes longer, the customer refreshes after payment or a report is generated but the database write fails?

Architecture starts with how the system behaves in the real world, before library selection.

Code that signals trouble

Even short code can do too much.

This example looks harmless but combines payment, analysis, reporting, email and an external provider in one place. After a failure, it is difficult to tell what already happened.

public async Task CompleteOrder(Guid orderId)
{
    var order = await db.Orders.FindAsync(orderId);
    order.Status = "Paid";

    var audit = await crawler.Run(order.TargetUrl);
    order.ReportJson = JsonSerializer.Serialize(audit);

    await email.Send(order.Email, "Raport gotowy", audit.Summary);
    await stripe.MarkProcessed(order.PaymentIntentId);
    await db.SaveChangesAsync();
}

A better boundary

First the decision, then the side effects.

The transaction should save the fact and the intention to continue work. Auditing, email and integrations can run separately with retries, monitoring and a clear customer status.

public sealed record ConfirmAuditPayment(
    Guid OrderId,
    string PaymentIntentId,
    DateTimeOffset ConfirmedAt);

public sealed class ConfirmAuditPaymentHandler
{
    public async Task Handle(ConfirmAuditPayment command)
    {
        var order = await orders.Get(command.OrderId);

        order.MarkAsPaid(command.PaymentIntentId, command.ConfirmedAt);
        outbox.Add(new AuditRequested(order.Id, order.TargetUrl));
        outbox.Add(new CustomerNotificationRequested(order.Id));

        await unitOfWork.SaveChanges();
    }
}

SOLID without dogma

Principles should help rather than conceal missing decisions.

Single Responsibility says that if an email change breaks payment, responsibilities are mixed. Open/Closed says that if every new provider requires changing half the system, a contract is missing. Dependency Inversion says that if the domain depends on a technical detail, that detail starts controlling the product.

In practice, begin with simple contracts that tell the truth:

public interface AuditReportStore
{
    Task Save(AuditReport report, CancellationToken cancellationToken);
}

public interface AuditNotifier
{
    Task NotifyReportReady(Guid orderId, CancellationToken cancellationToken);
}

public sealed class CompleteAuditHandler
{
    public async Task Handle(AuditCompleted completed, CancellationToken ct)
    {
        await reports.Save(completed.Report, ct);
        await notifier.NotifyReportReady(completed.OrderId, ct);
    }
}

Mini-game

Arrange the architectural decision flow.

0/6

Failure and retry plan

Business goal

Tests, monitoring and rollback

Data model and source of truth

Process boundaries

Integration contract

Put the steps in order. Start with the goal, not the technology.

Clear answers

Taking over someone else's system

Do not start with refactoring. First understand what must not break.

Taking over code is detective work, not blame-finding or hunting ugly classes. Find critical flows, source data, integrations, decision owners and places where a missing test could cost the customer money.

Run the application locally Find login, payments, forms and integrations Draw a data map Add a characterization test Agree on acceptance criteria before changing code

A characterization test does not say the old code is beautiful. It says, “This is how the system behaves today, and we must not accidentally change it.”

[Fact]
public async Task Paid_order_requests_audit_only_once_for_duplicate_webhook()
{
    var paymentIntentId = "pi_123";

    await app.HandleStripeWebhook(paymentIntentId);
    await app.HandleStripeWebhook(paymentIntentId);

    var order = await app.FindOrderByPayment(paymentIntentId);

    Assert.Equal(OrderStatus.Paid, order.Status);
    Assert.Single(await app.Outbox.EventsOfType<AuditRequested>(order.Id));
}
Challenge 02

You are taking over a module without tests. What do you do on day one?

Immediately refactor the ugliest class.

Write a characterization test for the critical flow.

Read the README and assume it is current.

Choose an answer and see what it means for maintenance.

ADRs in practice

A decision needs a reason, consequences and evidence.

We record decisions so that three months later nobody has to guess why the system behaves this way, not to produce documentation for its own sake.

ADR-004: Run the audit asynchronously after payment

Context:
A website audit can take tens of seconds to several minutes.
Checkout must finish quickly.

Decision:
After the Stripe webhook, mark the order as paid
and record an AuditRequested event in the outbox.

Consequences:
The panel displays "queued".
The worker can retry an audit after a failure.
Queue and stuck-job monitoring are required.

Evidence:
Duplicate-webhook test, worker-restart test,
orderId correlation log.

Run decision preflight

Ready to check. Click to see whether the decision has a complete trail.

Review questions

If you cannot answer these, the architecture is not ready yet.

01Which business event starts the process?
02Which data is the source of truth?
03Who can approve a costly decision?
04What happens when a request is duplicated?
05How will the customer see progress?
06Which test completes acceptance?

Conclusion

Good architecture is most visible when something goes wrong.

If the system can show status clearly, retry safely, preserve data, alert someone, roll back a change and put a decision in the right person's hands, the architecture starts paying for itself.

It reduces the cost of chaos and lets the team deliver changes without fear.

Your decisions in this workshop 0/2

Start with a decision. Good architecture should stand up in a real scenario, not just on a diagram.

Reset workshop