External Task multithreading

Camunda 7 Topics Discussion & Questions
1
  1. While using external task client for communication with camunda can we open a new thread always to perform or execute the actual action?

client.subscribe(“xxx”) .handler((externalTask, externalTaskService) → {
// open a new thread and execute the task in another thread

// continue for the next task without waiting
});
Is there any problems with this approach??

  1. Or How can we can run multiple external task clients on the same machine in order ro process tasks with more throughput?
1 Like
2

Hi @Nishant-sehgal,

I would avoid multi threading inside an external worker. The operation system should be better with this.

Have a look at this and the following posts: Multiple External Task Client instances

Hope this helps, Ingo

3

@Ingo_Richtsmeier What issues will it lead to if i follow the multithreading approach? Why it is not used?

Can i do something like below?
Create multiple ExternalTaskClient objects on server boot up something as below:

// say based on config will create n ExternalTaskClient
List < ExternalTaskClient > externalTaskClients;

public void enableWorkers(){
// loop over the list of client and call below code for each client
client.subscribe(“xxx”) .handler((externalTask, externalTaskService) -> {
// execute an action
});
}

4

Hi @Nishant-sehgal,

multi thread programming is more complicated than single threading. Especially when it comes to joining your started threads.

And there is no guarantee that it will be executed faster, because it depends highly on the hardware where you execute your code. At the end, multithreading is just time slicing if you start more threads than your processor has cores.

I would avoid complexity and scale the worker with the help of the operation system or kubernetes, as I have tools to control the environment and resource consumption.

This is very easy if you wrap your external task client into a spring boot application and start several java processes on the command line.

But this is just my personal opinion.

Hope this helps, Ingo

5

@Ingo_Richtsmeier
we can use a framework that can handle that for us like rx java.can that solve the issues?

another question is instead of launching worker on multilple machine can’t we use hybrid approach of launching multiple machine and per machine multiple instances of worker.

Like u can have 3 instances of my application and each running 5 instance of worker.

// say based on config will create n ExternalTaskClient lets say 5 per machine
List < ExternalTaskClient > externalTaskClients;

public void enableWorkers(){
// loop over the list of client and call below code for each client
client.subscribe(“xxx”) .handler((externalTask, externalTaskService) -> {
// execute an action
});
}

6

@Ingo_Richtsmeier can you share your point of view on the hybrid approach.

7

Hi @Nishant-sehgal,

I haven’t used these reactive frameworks by myself, but from the articles I’ve read, my impression is:

  • yes, they help to do multi theading in your code.
  • the business logic gets spread around in your code and will become more difficult to maintain.

In the end it is important which improvement you want to reach: Higher throughput of process instances? Less latency between service task creation and service task completion? Where are the bottlenecks?

I would start measuring the current state with one simple task worker and then try to improve in simple steps and compare the results.

Your hybrid approach may work.

Hope this helps, Ingo

8

Check this simple…Multi-threaded socket programming in Java

9