FetchAndLock API Best Practice


#1

Hello,

I am using C# and HTTP requests to communicate with the Camunda engine.
I based my logic on - https://blog.bernd-ruecker.com/use-camunda-without-touching-java-and-get-an-easy-to-use-rest-based-orchestration-and-workflow-7bdf25ac198e

By now I got quite a few different BPMN’s with many shared topic names between them (for my external tasks).

At the moment I am using the FetchAndLock mechanism in a 1:1 ratio in the following way -
I created a C# worker for each unique topicName in the BPMN’s I got, and each such worker constantly sends FetchAndLock requests for that specific topicName and redirects the responses to an executable c# code. When the code execution ends the worker sends a ‘completed’ or ‘error’ HTTP request back to the camunda engine…

This solution is working but there is one problem. Slowly I am adding new BPMN’s and more unique topic names continue to pop up as well which also means the creation of more c# workers that needs to constantly send fetchAndLock requests to the camunda engine api.

If I look forward it seems that at some point I might have something like 1000 topic names and 1000 c# workers which constantly send FetchAndLock requests - 1 worker per 1 topicName.
It seems that on the long run it is not very practical and it will overload the server. I was wondering if there are some best practices I can learn from to reduce the heavy resources consumption.

I think I probably need to group up some of those workers and stop having a 1:1 ratio for each topicName (separated fetchAndLock request for each topic name) but I am not quite sure how to approach this.


#2

Hi @Ivgi,

this ploblem may occur when you use java delegates as well. 1000 different task types may need 1000 different service implementations.

But there are some options to sort them out.

The fetch-and-lock request can carry multiple topics. And in the official external task client for java and javascript (https://docs.camunda.org/manual/7.10/user-guide/ext-client/) you can register multiple handlers for different topics in the same client.

This may reduce the amount of clients and offers a way to structure your service execution.

Maybe you can follow this approach in your c# client as well?

Hope this helps, Ingo