Asynchronous messaging microservice design pattern

One of the most important things to consider in a distributed system is state. Although highly powerful REST APIs, it has a very primitive flaw of being synchronous and thus blocking. This pattern is about achieving a non-blocking state and asynchronicity to maintain the same state across the whole application reliably, avoid data corruption, and allow a faster rate of change across the application:

• Problem: Speaking contextually, if we go with the principle of single responsibility, a model or an entity in the application can mean something different to different microservices. So, whenever any change occurs, we need to ensure that different models are in sync with those changes. This pattern helps to solve this issue with the help of asynchronous messaging. In order to ensure data integrity throughout, there is a need to replicate the state of key business data and business events between microservices or datastores.
• Solution: Since it's asynchronous communication, the client or the caller assumes that the message won't be received immediately, carries on and attaches a callback to the service. The callback is for when the response is received what further operation to be carried on. A lightweight message broker (not to be confused with orchestrators used in SOA) is preferably used. The message broker is dumb, that is, they are ignorant of the application state. They communicate to services handling events, but they never handle events. Some of the widely adopted examples include RabbitMQ, the Azure bus, and so on. Instagram's feed is powered by this simple RabbitMQ. Based on the complexity of the project, you can introduce either a single receiver or multiple receivers. While a single receiver is good, soon it can be the single point of failure. A better approach is going reactive and introducing the publish-subscribe pattern of communication. That way the communication from the sender will be available to subscriber microservices in one go. Practically, when we consider a routine scenario, an update in any of the models will trigger an event to all its subscribers, which may further trigger the change in their own models. To avoid this, event bus is generally introduced in such type of a pattern that can fulfill the role of inter micro service communication and act as the message broker. Some of the commonly available libraries are AMQP, RabbitMQ, NserviceBus, MassTransit, and so on for scalable architecture.

Here is an example using AMQP:  https://gist.github.com/parthghiya/114a6e19208d0adca7bda6744c6de23e.

• Take care of: To successfully implement this design, the following aspects should be considered:
• When you need high scalability, or your current domain is already a message-based domain, then preference should be given to message-based commands over HTTP.
• Publishing events across microservices, as well as changing the state in the original microservices.
• Make sure that events are communicated across; mimicking the event would be a very bad design pattern.
• Maintain the position of the subscriber's consumer to scale up performance.
• When to make a rest call and when to use a messaging call. As HTTP is a synchronous call, it should be used only when needed.

• When to use: This is one of the most commonly used patterns. Based on the following use cases, you can use this pattern or its variants as per your requirements:
• When you want to use real-time streaming, use the Event Firehouse pattern, which has KAFKA as one of its key components.
• When your complex system is orchestrated in various services, one of the variants of this system, RabbitMQ, is extremely helpful.
• Often, instead of subscribing to services, directly subscribing to the datastore is advantageous. In such a case use, GemFire or Apache GeoCode following this pattern is helpful.

• When not to use: In the following scenarios, this pattern is less recommended:
• When you have heavy database operations during event transmission, as database calls are synchronous
• When your services are coupled
• When you don't have standard ways defined to handle data conflict situations