Camunda & kafka topic with multiple type events

Hello camunda community,

I am trying camunda to supervise kafka topics.
So, i have 1 kafka topic with multiple type events (account created, updated, etc.). And asking myself, if it is possible to filter consummed events with a kafka message field.

kafka messages :

{“id”:1,“state”:4}
{“id”:2,“state”:1}
{“id”:1,“state”:1}
{“id”:1,“state”:3}
{“id”:2,“state”:2}
{“id”:1,“state”:2}

  • only 3 states interesting me to visualize workflow for this test, but there are more than 3 types of messages on this topic.
  • i need to start the workflow instance by waiting kafka event with state 1
  • Exclusive gateway with conditional expression, could be the solution for my case ? But its looks complicated for start event and timer event to do this, nope ?

Thanks

example with exclusive gateway :

Hello @jerome ,

This is an example to solve this problem without expanding the usage of “waiting event with state 2”.

The Non-interrupting start event of the event-sub-process can be used to catch events with other states than 1, 2 or 3 and still, they will be tracked inside the engine.

The bottom construct means almost the same than yours, one point here is the join of the parallel flows after both events arrived.

I hope this helps

Jonathan

1 Like

Thanks @jonathan.lukas

i’m ok for the parallel and event based gateway :+1:

Nonetheless, i am reading about event sub process, but, how can we make the link between the event and the subprocess ? I need 1 subprocess by event ? And i need implement an exclusive gateway (filtering on state) in each subprocess ? :confused:

Thanks

Hello @jerome ,

as I see it, your PoC kafka stream delivers objects containing 2 attributes: an Id and a state.

I would map the id to a business key to identify the process instance and to be able to map incoming events to the correct process instance.

For the state, I would suggest the following:
When the event arrives, there are 4 possible messages that will be sent to the engine: 1,2,3 and rest. So, any other state than 1, 2 or 3 will be sent as a message called rest that triggers the event-subprocess inside the process instance with the correct business key. As the event-subprocess can be triggered as often as you need (while the process instance is active), you can map n different events on it. This is because of the “non-interrupting message start event” in the event-subprocess.

You can also read about this behaviour here: BPMN 2.0 Symbols - A complete guide with examples. - Camunda
There is a section called Event Subprocess that explains the behaviour of it.

A quick scenario based on your kafka messages:
1 arrives: will not be correlated as no instance with business key “1” exists
2 arrives: process instance with business key “2” is created
3 arrives: process instance with business key “1” is created
4 arrives: message “state 3” is correlated to process instance with business key “1”, interrupting the timeout
5 arrives: message “state 2” is correlated to process instance with business key “2”, this instance is then only waiting for message “state 3”.
6 arrives: message “state 2” is correlated to process instance with business key “1”, this instance will be finished afterwards

1 Like

This scenario is exactly what we wait.

The event sub-process is compatible with kafka connect zeebe out of the box ? :confused:

Hello @jerome ,

here is the blog article about our kafka connector:

The event subprocess is able to catch messages that are correlated to the engine, so yes, it is compatible.

Right now, I am not sure whether event subprocess is yet supported by zeebe engine.

Hope this helps

Jonathan