The practical need for eventual consistencyHere is how the problem can happen in practice. We have a task to synchronise orders and customers between two complex systems like ERP and online shop.
- Customers register in shop and place their orders.
- Each order contains a reference to a customer, so that no order could be created in ERP before customer is created.
- Process A synchronises customers and
- Process B synchronises the corresponsing orders.
The theory behind the eventual consistencyIn theoretical computer science we are faced with the uncertainty of three states which are the Consistency, Availability and Partition tolerance (the CAP theorem). For a distributed computer system it is impossible to guarantee all three states simultaneously. One or two of these states would need to be sacrificed if one would try to seek an absolution in one state only. Instead, synergy or partial solutions can be devised. It is not the scope of this article to present all the solutions to this dilemma – Werner Vogels, the CTO of Amazon.com has written more on this. Here we will talk about one of the widely accepted and used solutions in the distributed computer systems called eventual consistency:
Eventual consistency is a consistency model used in distributed computing to achieve high availability that informally guarantees that, if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value.In all its glory, eventual consistency is not a flawless solution partly because eventual consistency is a liveness guarantee. This means it can keep the system alive (free from deadlock) and progress further despite the errors. What we would need to do for safeguarding is use sets of concurrent systems to reschedule or send messages back for reprocessing. This kind of safeguarding is called the bounded bypass which is implemented at elastic.io to reach the eventual consistency for integration processes.
– Wikipedia, Eventual Consistency
Reaching eventual consistency through Rebound at elastic.ioComing back to the example above, when we try to synchronise an order in ERP while the customer data for that order is not yet in place, we’ll simply postpone the processing of that order for a while. By doing that we give the parallel process more time to synchronise customers, and hope that the corresponding customer information would eventually be synched with the ERP. This is a clear example of eventual consistency application in the integration processes. To reach the eventual consistency we use one of the built-in features of elastic.io integration platform – the Rebound.
- Rebound is a feature that adds a possibility to bounce back and reprocess the incoming messages when the system is not ready to process them at that particular instance.