Buffering of received messages

Hello,

camunda BPM has a somewhat unpleasant property that, if a process has an intermediate receive message activity, the process must have arrived at this activity in order to be able to receive that message. If a sender wants to send the message before the process has arrived at the receive activity, it (the sender) will fail.

This is a pity because, quite often, we can not control when the external events occur and when the messages are sent to processes. As a result, we have to specially take care of such cases and ensure that the process arrives at the receive step “in time”. This makes process models more complex.

ZeeBe, the new BPM kid, behaves differently in this regard, as described e.g. here: Zeebe | Camunda Platform 8 Docs messages sent to a process are buffered and “wait” for the process to arrive. When the process arrives at the activity, the message gets processed. This is like a parallel join: one branch is the process, the other branch is the message. When both arrive, the processing continues. This is a very useful and intuitive behaviour.

My question is: is it possible to implement this behaviour (message buffering) in camunda BPM? It would be cool if it would be possible to implement it in a generic way, i.e. not individually for each case where a message must be received.

Thank you for any hints.

Hi Fml2,

One generic way is to use middle-ware as an intermediate buffer and thus create a re-delivery policy in the middle-ware message buffer…

Conceptually you could implement a process which catches a generic message type and thus all messages. You could then use message ‘content based routing’ to create a new type so that it correlates to a specific message type. If you cant correlate, then you could use the async task retry policy to periodically retry…

So its possible, but not necessarily going to scale well and could consume a lot of resource of the engine which may be better spend in middleware.

regards

Rob

Thank you for the quick reply. I see it’s possible, but it would be very complicated. I’d write something like ZeeBe from scratch :slight_smile:

I’m a bit surprised this issue is not discussed much in the forums. IMO it’s a problem that shoud occur quite often. Are there other generic solutions (besides message buffering) to cope with it?

It is discussed fairly frequently and in fact the most recent discussion is probably worth reading for you

2 Likes

@Niall thaynk you for the hint! I somehow missed that.

Hello,

I had a look at the “plugin” (I use " because, as I see, this code is not dedicated to message buffering; message buffering is just a small part of a larger project).

Essentially, it boils down to having a buffer with submitted messages and a thread (similar to the job executor) that tries to deliver the messages after they have been submitted (but could not be delivered immediately). The buffer should be persisted of course so that submitted messages survive an engine restart.

It would be VERY cool if camunda could implement such feature as part of the core product.

He-he, I’ve just had a thought (after I wrote “job executor”): it should be possible to implement this feature using only the existing camunda features. Just have a process that has an infinite loop. In the loop, it would receive a message and maybe start another small process whose subject would be to deliver the message. The looping process would then immediately go to the “ready” position and would be able to receive another message.

The only problem is that it would not be the “native” camunda API.

1 Like

For all who are interested in “Message Queueing” getting into Camunda Platform, let’s upvote this feature request.
https://jira.camunda.com/browse/CAM-13320

1 Like