Innovation in Workflow Engines
- August 27, 2019
- 0 Comments
Camunda continues to advance the state of the art for distributed workflow with the first production release of Zeebe.io a few weeks ago. The most interesting part of this product (to me) is that Camunda has taken what they learned about process and workflow engines that have a more traditional architecture, and then applied it to a very different use case: microservices workflows with really high through put and low latency.
The insight that developers at Camunda had was that while a collection of atomic microservices will scale to very large volumes, and might well keep each component well defined and simple, the interactions between these microservices are both complex, and may require transparency.
This is because microservices based application architectures largely rely on messaging (pub/sub) models to communicate and coordinate their actions, which makes it difficult for developers and operators to be sure of the flow of the interactions – because they are often asynchronous, open-ended, fast-moving, and there’s no limit on the complexity of the interactions.
Zeebe.io itself is an event sourced workflow engine, so it plays nicely and natively to the target audiences (architectures).
It turns out that there’s an analogous problem in Robotics. You build a series of automations (bots), which might be chained together in a linear fashion, or might rely on a human to initiate (and digest) the results. Eventually, if you tackle enough individual task automations, you’re going to want a way to understand how they all play in the bigger picture or in the end-to-end process.
This is part of what makes the software business fascinating – there’s a constant tension between trends of atomizing or decomposing into separate functions, and the trend of bundling and consolidating and ordering previously chaotic or unwieldy interactions.With the right combinations of atomic automation (bot), and loosely coupled coordination (process), we can achieve dramatically better value for our clients.
It looks to me like Zeebe.io brings the same potential to those building scale systems with atomic microservices that really need scale.