Microservices & Event-Driven Architecture

Today’s businesses are more reliant than ever on software. This means key determinants of business success (e.g., agility, speed) are tied to a company’s ability to implement better software, faster. Complexity is the enemy of agility and speed. Within a business’s software ecosystem, the bulk of the complexity (i.e. cost, effort, risk) is in the connections within and between application components. Thus, this maze of fragile and tightly coupled integrations between application components is often a key roadblock to business agility and speed.

This is not a new problem. Software architects have been trying to solve this issue for decades.  It’s given rise to Enterprise Application Integration (EAI), Extract, Transform & Load (ETL), Service-Oriented Architecture (SOA), Message Oriented Middleware (MOM), and many other patterns.

Each of these classic approaches has benefits and drawbacks. For example, classic ETL implementations are typically batch-oriented, which means current data arrives too late for typical business cases. While it enables more asynchronous processing, message-oriented middleware has traditionally suffered from suboptimal message storage capabilities, out-of-order message-processing challenges, and limitations for big data scenarios.  As a result, new architecture paradigms (and enabling technologies) have emerged. Microservices and Event-Driven Architecture are two examples of patterns that are exploding in popularity.

Microservices

The Microservices pattern is an evolution of SOA that enhances business agility by enabling independent deployability of fine-grained business services. Legacy architecture too often focuses on separating components based on their underlying technology. In reality, business requirements evolve and change far more frequently than do technical requirements.  Therefore, Microservices align software components with business domains and business contexts instead of with technology domains. This is done at a fine-grained level to minimize impact on other components when specific business-requirement changes are implemented and deployed, providing great potential to increase agility and speed.

The Microservices “Data Problem”

One of the biggest challenges with implementing Microservices is decoupling business data that traditionally lives in a monolithic database into smaller, independently deployable datasets that live with each Microservice. Breaking a large relational database into more-finely grained data repositories can be quite challenging due to rigid entity relationships. Even when decomposition can be achieved, the need to join data across physical databases often results. Traditional SQL database technology is not well-suited for this approach.

Event-Driven to the Rescue

The classic messaging pattern of producing and consuming messages in a loosely coupled asynchronous manner is being fundamentally transformed by modern technology. Innovations in distributed commit logs and big data storage have enabled real-time messaging platforms (e.g., Apache Kafka) to emerge as foundations for implementing extremely performative event-driven architectures at massive scale.

The capability to continuously process huge volumes of data in real-time has nearly limitless utility and business use cases.  That’s why companies such as Confluent are growing like the veritable wildflower.

So, how does this relate to Microservices? Imagine you’re designing a Microservices-based architecture for your organization, in adherence with proper domain-driven design. This means each Microservice controls the business data needed for its specific business context (i.e., the “bounded context”).  Inevitably, you will need to assemble a full business entity, such as a customer, which has been divided across Microservices.

This is where event-driven architecture comes into play. Event streams can provide the data communication mechanism for sharing data across bounded contexts. You can implement an event-driven architecture to ensure all changes (from any user, system or Microservice) related to a customer profile are published to a central event hub. Interested parties, such as other Microservices (i.e., consumers) can subscribe to those events to maintain the real-time aggregate view of a customer, when needed.

In a nutshell, event-driven architecture is the solution to the “data problem” in Microservices.

Power Couples Make Each Other Better

Businesses continue to search for ways to achieve more agility and speed. Technology innovation is foundational to these efforts. New design patterns are emerging that open new doors for solving longstanding challenges in software design. Two such design patterns: Microservices and event-driven architecture offer significant improvements over classic patterns. By pairing these approaches together, we can achieve independent deployability of software components and real-time data flow across the enterprise. These breakthroughs will help deliver the agility and performance modern businesses need to create and sustain competitive advantage.

Lastly, for more information on all the ways your organization can benefit from Microservices, please check out this recent episode of the Modern Digital Enterprise podcast: Trends in Modern Enterprise Architecture or reach out to us with any questions. We’d love to help you get started.