Data integration best practices: Request-Reply Integration vs Asynchronous Integration

Jacob Horbulyk Integration best practices, know-how

Request-Reply vs Asynchronous Integration

Moving on to the next chapter of our blog series, namely about different types of integration, in this article we’re going to explore the main two types of integrations: asynchronous integration and synchronous integration.

Whether you need a fast response of the systems involved or the results to be delivered further in the future; both, request-replay or asynchronous integration, offer their own pros and cons and could be used for different purposes.

Request-Reply (synchronous)

Request-Reply integration requires interactions with an external system to occur before an operation is complete. In general, we need these types of integrations for rendering information in front of a user in real time.

For example, a system needs to display a list of courses to a volunteer based on their profile while the list of courses is stored in a different system. In order for this to happen, the system requiring information must request information from the system containing the information. And the system containing information must reply to that request.

A common protocol for this exchange is HTTP(S) but other protocols are possible. These requests may be ‘read’ operations that do not have any so called side effects – that is to say “has an observable effect besides returning a value”. However, they may also be ‘write’ operations or other operations with side effects.

In order for the requesting system to operate, the system where the requests go to must be up and available.

When building request-reply integrations, there are two strategies as follows, each with their own pros and cons

Direct Requests:

As the name suggests, these are requests from one system to another without any intermediary system.

Pros:

  • Data exchange happens fast
  • It is reasonably simple to set up
  • There are fewer failure points as there are only two systems in play

Cons:

  • Error tracing is difficult because you wouldn’t usually have a visual interface or built-in monitoring tools
  • Credential management is difficult too

Requests through an integration layer:

In this case, we have a third system between the requesting and the receiving system, which plays a role of intermediary. This can be a third-party or in-house solution.

Pros:

  • This third system can act as mediator by modifying or enriching requests and responses
  • It can also act as a caching mechanism when there is no built-in one
  • It can also have a system health tool role providing system monitoring. After all, you have two systems talking to each other directly, it’s hard to know if there is a problem with that “conversation”. With this third, intermediate layer, you can check if this communication is OK or not.
  • It can provide centralized logging, error management and credential management
  • Additionally, it can act as an abstraction layer. For instance, to collect information from several similar sources such as email providers where the actual source is not relevant but the information is.

Cons:

  • With one additional system now in play, this adds yet another failure point
  • Has some performance overhead since the third system requires some time to process and sort the information it receives and sends

Asynchronous Integration

Asynchronous Integration is integration where the data does not have to be moved immediately but can be moved at a later point in time. This means that the system sending a request doesn’t have to wait for a reply in order to continue operating.

This type of integration is especially useful when we have large volumes of data to process or when we don’t expect any immediate response. For instance, when we set up a regular data sync between a CRM and ERP systems. If a sales employee enters a new account, the CRM system would spot the change and push it, e.g. into a queue on an integration middleware. In doing so, CRM has done its part of the deal. From there, it will be at some point picked up by or pushed to the ERP system. It doesn’t really matter either for the employee or for the CRM when the update occurs – immediately or in an hour -, as long as it occurs eventually.

Pros:

  • Asynchronous integration is considered to be more reliable than synchronous integrations. Due to the fact that systems don’t wait for each other, none of them will have a timeout.
  • Derived from the previous point, asynchronous integration lead to higher services availability as asynchronous integration can wait for systems to recover
  • More efficient use of machine resources
  • The system can offload data when it is not busy
  • Batching becomes possible

Cons:

  • The fire and forget model means that the system may forget these events
  • Immediate feedback is not possible
  • Lag may lead users to assume that the system is broken

So, as you can see, it really depends on your integration scenario which type of integration you would need. Both synchronous integration (request-reply) and asynchronous integration have their specific application cases, and their pros and cons, too. Sometimes, however, you have no choice but to choose one over the other. That’s why you can consider the cons listed above as something to keep in mind rather than something to try to avoid. To learn a little bit more about different types of these integrations, check out our blog article “3 Message Exchange Patterns in Application Integration You Should Know About (with Examples)”.

In our next article, we’ll have a look at different systems that enable these synchronous and asynchronous integrations in the first place. In other words, we’ll check out the systems that move data. Stay tuned by following us on Twitter and LinkedIn!


About the Author

Jacob Horbulyk

Twitter

Pre-sales & Professional Services Engineer at elastic.io. Casual board gamer, language learner, loves a good weekend road trip.


You might want to check out also these posts