Hi All,
I have a problem starting Camunda process which is supposed to be starting 5000 new process instances using multi-instance call activities.
This is already taking more than two hours which is unacceptable performance, since this number will only grow.
I am using Camunda with spring boot, version of Camunda is 7.8, version of MySQL is 14.14.
Any suggestion would be great.
Thank you in advance
Can you describe what your process is doing and what the multi instance is doing.
Multi instance is calling a subprocess with 5 queries in a service task. Queries are simple and return maximum of 3 rows but must be done after process starts.
Here are the screenshots of demo processes with descriptions:
What are the response times of each of the queries? And when you save your variables, what are the types ?
Likely want to be increasing your job executor pool
All queries take aprox. 50 ms. Variable types are java classes representing one row from database (so saved as bytearrays), but still I think this is better than doing a query each time I need column other than the id, which is all the time.
Like I mentioned, screenshot is just a demo with service task which represents the problem. In real bpmn I have a lot of conditional events to start event subprocesses which use variables I saved in the service task.
I am afraid I didn’t understand your last sentence, could you explain what you ment?
so basically the multi-instance would be executed by your thread (assuming it’s sync). If you set your multi-instance to async before Then the engine will create the 5000 jobs, and then use the job executor. Using the job executor should let you use the parallel functionally. So then your issue is execution counts. If you look at docs for Job Executor and tweak the configs for a larger pool and larger number of active jobs, you should be able to dramatically increase your active instances.
Also remember about variable scope. If you have variables being returned to the parent process (the process that has the call activity), and you have multiple instances from the call activity completing at the same time and returning variables to the parent, you are going to start to get optimistic locking exceptions.
I changed Multi instance options to multi instance asynchronous before and after 30 minutes still not finished starting process. I will try to change job executor configuration and notify what I managed to do. Thanks for help!
How many did it add to the database?
More than 20 000 rows in ACT_RU_EXECUTION table. This demo version of process ended successfully after 35 minutes. Now I am trying to check what time would real, more complex version of process end. Setting multi instance asynchronous before definitely is the right way to solving this issue. I will try further to ajust job executor pool configuration.
It will not influence performance, but for this kind of requirement you might want to have a look at th custom bath extension .
Hi,
we had a similar issue. Multi-instance with Camunda does not work well when it comes to thousands of instances even if the task has no variables and just prints “Hello World”. See Scaling multi-instance tasks, true parallel-execution and play around with https://github.com/marsewe/camunda-external-task-test .
Scaling with Camunda though works from my experience when you scale via the number of process instances, a 100k running instances work smoothly.
Since parallel multi-instance involves transactions that make a lot of inserts and deletions (linear with the number of iterations), using a database that benefits from SQL statement batching could improve performance as well (see for example https://blog.camunda.com/post/2017/11/camunda-bpm-780-released/batch-chart-throughput.png from https://blog.camunda.com/post/2017/11/camunda-bpm-780-released).
Cheers,
thorben
