Hello,
I am using C# and HTTP requests to communicate with the Camunda engine.
I based my logic on - Use Camunda as an easy-to-use REST-based orchestration and workflow engine (without touching Java) | by Bernd Rücker | berndruecker
By now I got quite a few different BPMN’s with many shared topic names between them (for my external tasks).
At the moment I am using the FetchAndLock mechanism in a 1:1 ratio in the following way -
I created a C# worker for each unique topicName in the BPMN’s I got, and each such worker constantly sends FetchAndLock requests for that specific topicName and redirects the responses to an executable c# code. When the code execution ends the worker sends a ‘completed’ or ‘error’ HTTP request back to the camunda engine…
This solution is working but there is one problem. Slowly I am adding new BPMN’s and more unique topic names continue to pop up as well which also means the creation of more c# workers that needs to constantly send fetchAndLock requests to the camunda engine api.
If I look forward it seems that at some point I might have something like 1000 topic names and 1000 c# workers which constantly send FetchAndLock requests - 1 worker per 1 topicName.
It seems that on the long run it is not very practical and it will overload the server. I was wondering if there are some best practices I can learn from to reduce the heavy resources consumption.
I think I probably need to group up some of those workers and stop having a 1:1 ratio for each topicName (separated fetchAndLock request for each topic name) but I am not quite sure how to approach this.