#Microservices#SAGA
The SAGA pattern is a failure management pattern that brings consistency in distributed applications and coordinates transactions between various microservices to maintain consistency. Command and Query Responsibility Segregation or CQRS is a pattern that separates read and update operations for a data store. Both CQRS and saga are used to decouple read and write workloads from each other for better performance, security and scalability. Find out how services communicate with each other using event messages and how they enable loosely coupled structures and asynchronous events.
May 06, 2022
Transactional microservices or the use of microservice architecture (MSA) for building applications has gained popularity over time. Distributed systems come with a range of benefits over monolithic systems. But there are often new challenges and complexities that crop up with unique business processes and scaling up of applications.
For one, distributed transactions span over multiple physical systems or a number of computers over the network. The transaction itself is distributed to multiple services called in a sequential order to complete the transaction. While this has obvious advantages and benefits, it becomes increasingly challenging to ensure data consistency, accuracy and safety when it comes to scaling up transactional apps in a microservice architecture.
Distributed transactions are by nature complex processes with a number of moving parts that can fail. Many of these parts or processes run on different machines and often with different data centres. As a result, the process of committing the transaction can become lengthy or unreliable. This can impact the user experience and system bandwidth.
MSAs are often ideal for large and complex apps because they are modular, each service responding to a business capability. An eCommerce store, for instance, would have various services that include orders, payments, customer service, catalogue, support etc.
However, with this modularity comes the challenge of decomposing the domain model. The Domain Model pattern which is useful to implement complex business logic is difficult to decompose because many of these services or classes often reference each other or share dependencies. For example, the domain model for an online store typically has classes like Order, OrderLineItem, customer, product etc. In terms of MSA, the Order and OrderLineItem classes belong to the Order Service, while the Customer class is part of the Customer Service and so on.
This is especially important because transactions will span services. In the case of a traditional application or website, the system can rely on atomicity, consistency, isolation, and durability or ACID transactions to implement business rules. Consider the example of customers whose credit limits need to be tested before creating a new order.
The application needs to ensure that multiple concurrent attempts to place the order must not exceed the credit limit. In an ideal scenario, Orders and Customers would reside in the same database and it would be easy to verify this with an ACID transaction. However, in the distributed transactions scenario, this is not possible.
The solution to the challenges outlined lies in event-driven microservice architecture where services communicate with each other using event messages. Producers publish these messages when business events occur and other services consume them through event listeners. The main advantage of event-driven microservices is loosely coupled structures and asynchronous events.
The saga design pattern is a unique solution to managing data consistency across microservices in distributed transaction scenarios. Using SAGA, multiple compensatory transactions can be used at different stages ensuring the ability to roll back when required.
In short, saga is a set of operations on microservices and relevant compensation processes. If and when an operation fails, the compensation operation and all previous compensation operations are called out in reverse order to roll back the system
There are benefits and drawbacks of either of these approaches and the saga pattern in itself may be challenging initially, as it demands a new way of thinking on transaction management and data consistency.
It is ideal for the following scenarios: To ensure data consistency in a distributed system without tight coupling To roll back or compensate in case an operation or sequence fails
Let’s look at an example from an online food delivery flow: When a user places an order, this is the sequence that could happen:
This is of course a very basic example of saga in use. Saga can be used in multiple ways to handle complex business transactions. In CQRS referenced below, sagas are leveraged to manage the BASE transactions or Basic Availability, Soft State, Eventual Consistency transactions. In this case, they respond to Events and may dispatch Commands, invoke external applications etc.
Another approach to managing the challenge of concurrency and locking in transactional database management is the Command-Query Responsibility Segregation. CQRS is an architectural pattern that advocates a strict split within an application and suggests isolating the query components from one another.
The concept of CQRS uses messages to describe commands, events, queries to deal with the Command Model and sync changes and extracts data from the Query model. This is hugely beneficial in enabling microservices and taking a monolith first approach with your application. The idea is to create the business functionality first and implement the non-functional aspects later.
CQRS segregates reads and writes into different models. It uses commands to update data and queries to read the data.
An operation can either read data or it can write data but it cannot do both. This separation makes it easier to manage data. The read and write operations are much more manageable because their functions are focused. Secondly, these operations can be developed individually by different teams and optimized separately, evolving independently over time following user requirements.
The complete CQRS pattern leverages the APIs for accessing data and the models for the management of data. The database itself is segregated into a read/write database that is write-only and one or more read-only databases.
CQRS is typically used in conjunction with Event Sourcing. Here, the write model of the system manages the persistence of the event while acting as the source of information for the read side. Whereas the read model offers materialized views of the data as highly denormalized views.
Transactional microservices need to find workarounds for some of the more challenging scenarios to ensure data consistency, user experience, security and independent scaling. The open-source frameworks like Axon and Eventuate help us building applications using domain-driven design techniques to implement CQRS, saga and event-sourcing patterns. For more information on implementing microservices for distributed transactions, please get in touch with our team
