2.7 Architecture foundation

Before we start defining our architecture foundation, let’s make sure we’re ready. As with each chapter, we’ll do that by reviewing our definition of ready. This makes sure we don’t jump ahead prematurely.

Before starting

Your architecture foundation establishes system design by laying out components based on domain modeling, functional and non-functional considerations. It’s expressed using two models: a high-level architecture model (conceptual, technology-agnostic) and an architecture realization (concrete, named components). This activity is one of three that make up your North Star — the others are 2.6 Target state architecture and 2.8 Delivery processes & tools.

The North Star

You may have noticed the Delivery Playbook subway map has “North Star” written above chapters 2.6, 2.7 and 2.8. Your North Star is your navigational reference point — always there to get your bearings, answering the question, “What do I do next? Where does this go?” It consists of three essential parts:

1. Your target state architecture. This is largely embodied in your service catalog. It tells us what tools and technologies we can use.
2. Your architectural foundation. Your core blueprint, a top-level guide that your team can use to determine where all the pieces go and how they connect with each other.
3. Your delivery processes and tools. Your “ways of working” — how you’ll build and deliver software, what your delivery pipeline looks like, and how each activity is carried out.

Guide to architecture foundation

Defining your architecture foundation is likely going to be an iterative exercise. You might have to go back to your service catalog and add (or remove) some technologies.

It’s chiefly a technical design activity, although you’ll need input from your business team at various stages. Depending on the size and complexity of your system, it could take multiple iterations to reach a technical design that you feel is “good enough” to proceed with. It doesn’t have to be perfect and complete. It’s actually better to iterate, trying out ideas, proving and disproving them.

Also, strive to be technology agnostic. Your architecture foundation should describe patterns and implementation principles. For example, in modeling an event streaming architecture you’ll want to identify what it is: An “event stream.” Avoid identifying how it is built, for instance, labeling it with “Kafka” or “AWS EventBridge.” There are a couple reasons for this. First, we don’t want to lock in a specific technology. We want flexibility, and if we start labeling with specific product names, that becomes rote. We use Kafka because we said we would use Kafka, not because it was actually the best choice. Second, we want a general, high-level design. It should be conceptual to keep the design pure. Kafka is a very specific thing, and while it can be used for an event stream, it does so with certain limitations. Our design should simply indicate that we are using an event stream.

“But, wait… the service catalog already talks about specific technologies, aren’t we supposed to use them?”

It’s a bit of an ostrich or egg conundrum. It’s hard to discuss a future state technology solution without talking about tools and products. When sitting around the table discussing “using an event stream,” we’ll naturally explore whether Kafka, SQS, or EventBridge meets our needs. We’ve even put some in our service catalog as candidate technologies.

But the service catalog will change. Today’s favored Kafka will turn into tomorrow’s EventBridge for various technical reasons. The service catalog is intended to be a mutable catalog of tools.

Contrasted against that, our architecture foundation should be immutable. It’s the bedrock on which everything is built. If it changes, then presumably everything changes. Avoid “baking in” product choices at an architectural level. Kafka is a product choice that we can use for event streaming, but an “event stream” is the architectural stereotype.

An example architecture

Let’s continue to build on our example: An online transaction system that supports Sally’s use case of buying a Peloton bike.

In considering the overall requirements for our system, a few stand out that push us in a particular direction:

• Extreme security and reliability (it’s banking, after all).
• Irrefutable auditability (again, banking).
• Integrating a lot of external systems (such as fraud checking, payment processing, merchant systems, and more).

These requirements are leading candidates for an event driven system. Loose coupling, resilience through isolation, high fault tolerance, and a complete audit trail (embodied in the event stream). A conceptual model of such a system describes the event stream as core to the architecture — internal and external systems interact almost exclusively through the event stream (individual domain services don’t connect directly to each other). APIs are clearly positioned for end users and systems to interact with the overall system, connecting to specific services and gateways.

This high-level model is still very abstract. There are a lot of different ways it could be realized and still be accurate. Especially for anyone uninitiated into this kind of architecture, exactly how to build it is going to be very opaque. For that reason, we distill an architecture realization — a specific example to help close the gap.

The architecture realization is exactly the same shape as the high-level model, but drawn as a concrete implementation. Instead of abstract lambda functions and generic domains, we name specifics. For example, a “Customer” domain and a “Merchant” domain, both representing specific services, each with their own data models. Services message each other over the event log. External connectivity is routed through a common authentication and gateway layer. Adapters flow information between the event log and legacy banking systems, keeping the event driven model pure.

This takes us one step closer to a complete technical design. It fills in the big picture, bringing clarity to how major components interact, where data lives and breathes, and how we’ll access the system.

Goals and impact

Inputs

• Participation from team leads, architectural leadership, and engineering team, as well as some availability of business team.
• Domain definitions and context map covering the current increment.
• Service catalog providing current candidate technologies.
• Event maps expressing the business processes in the current increment.
• Non-functional requirements (cloud vs. on-prem, regulatory, availability, performance expectations, etc).

Process

This will be a series of collaborative workshops. Depending on the complexity of your system, it may take days and many sessions to establish a foundation that everyone is comfortable adopting.

As is often the case, you may need to run a spike or breakout session to do additional analysis. As candidate designs are fleshed out, those designs can be brought back to the team for wider discussion.

For a particularly large system, you may want to use your component map (from chapter 2.6) to organize each workshop around specific components of your overall project. You may also choose to divide into parallel teams to get more done.

While the heart of your architecture foundation is likely going to address software architecture, don’t ignore your other architectural components. Just as the product needs an architecture, so does every other part of the system.

Don’t ignore new requirements that crop up. It’s very common for discussions to lead to new requirements regarding performance, security, and the like. These should be documented and reduced to clear functional requirements so that you have specific objectives (and you can measure success). In a well-defined system, there is no such thing as an unverifiable requirement.

Prepare

1. Organize your working sessions appropriately, based on the scale of your system. Use your architectural component map as an aid in coordinating topics and outcomes.
2. Share material for each working session with everyone who will be participating ahead of time. Scope your workshops accordingly and try to establish some clear goals (for instance, “decide on whether we want to use an event driven architecture, or take a more traditional object-relational persistence approach”).
3. Schedule accordingly, knowing that some of the larger technical decisions will require a significant amount of time. I like to plan between 2 and 4 hours for a workshop, giving the team plenty of time to work independently on breakout tasks.
4. Make sure everyone has access to your design tool. I like to use Miro, a good, general purpose tool that supports excellent collaboration. Set expectations that everyone should be “hands on” and contribute to the design.
5. Perhaps most important, make sure everyone brings their candidate solutions to the workshop. This is everyone’s opportunity to voice their opinion about the target state architecture.

Execute

1. Modeling activity
   1. Integration. Build a high-level view of the interaction points that each domain will have with any new or existing systems, gateways, or UI’s. How do your services interact with the world?
   2. Pattern definition. Define implementation patterns used to build the new system (for example, event streaming, serverless, microservice, CQRS).
   3. Iterate (internal). You may have more than one candidate solution in play. That’s fine. Document each, adding fidelity until one emerges as a clear winner architecturally. This is the perfect time to have further conversations about patterns, architecture principles, and key assumptions.
2. Finalize
   1. High-level model. Distill the final solution into an agreed-upon high-level architecture model. This becomes the part of your North Star that shows how all the pieces fit together at a conceptual level.
   2. Architecture realization model. Create a specific example that shows how the high-level architecture model will be realized in your system. This is where abstractions are removed, and specifics are inserted. The goal is to create a simple path to follow, an example that shows how each abstraction (e.g., “event stream”) is realized in the currently designed system (e.g., “event log” with connections to actual named services).
3. Document. As you proceed with the design activity, make sure that new discoveries (requirements, changes, constraints) are adequately captured. I try to delegate immediate updates right in the working session. “Great idea Jonas — could you pull up the event map and make that change for me while I jot down this other thing?”
4. Iterate (external). Solicit feedback from the architects within your organization to gain alignment and further insight.
5. A few tips for making the workshop as successful as possible:
   1. Make sure everyone has an opportunity to put forward their ideas regarding the architecture foundation. This can be done by segmenting your drawing workspace for different candidate solutions.
   2. Avoid deleting drawings wholesale, or completely replacing the team’s work product. Instead, make a copy, and segment off the original. Tools like Miro are great for this since the drawing space is unlimited. This makes it easy to go back and look at previous iterations.
   3. Drawings can get quite large, and often have a lot of cross-references. If your tool supports it, make heavy use of linking within the drawings (Miro does this).
   4. Significant constraints should be documented in your design (for instance, “AWS-only required,” or “customer firewalled & on-prem,” anything that dramatically affects the design).

Outputs

• High-level architecture model. A generic, simplified view of the target state. It sets the context of how components fit into the ecosystem and what patterns you’ll use to glue it all together. It’s a guardrail to protect your architecture.
• Architecture realization. A concrete implementation of the high-level architecture model, intended to demonstrate how the abstract model will be specifically implemented. Typically not drawn to encompass the entire system, it is usually a subset to establish an implementation pattern.

Together, the above collection of models and any other supporting drawings you deem necessary become your architecture foundation.

I’ve seen fairly complete architecture foundations that only needed two or three drawings. I’ve also seen incomplete ones that have closer to 10 drawings, adding important process flows, interaction diagrams, and “blow ups” for specific sections. It’s a matter of scale, and also what the team needs.

Craft your architecture foundation to suit your needs. Start with the high-level architecture model and a specific implementation — these two create a clear path forward. They will either reveal new questions that need answering (and perhaps some added technical drawings) or everyone will nod and agree and have clarity. The most important part is that clarity. Everyone at the table needs to understand where you end up.

In the next chapter, 2.8 Delivery processes & tools, we’ll focus on the build and delivery chain, how we’ll tackle verification and validation, and what deployment and operations will look like. All of this is part of your North Star, and part of your architecture. It’s as vital as the architecture foundation.

Next activity

• 2.8 Delivery processes & tools