Long running flows : how to properly avoid endless loop in jobexecutor

Camunda 7 Topics Discussion & Questions
1

Hello,

I’m facing this issue since a long time. I digged a lot in forum, samples… to find a proper solution without success.

I want to run asynchronously several workflows on the same embedded process engine.
To not lock main thread, I need asynchronous on all “start event”.
These workflows call external service executing long jobs (until 30 min).

I implement AbstractActivityBehavior service task.
The method calls the remote service and polls it until end of job.
Then signal is called.

But I see a lot of OptimisticLockingException

org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of 'DELETE MessageEntity[1fae0b35-7f93-11e8-bb97-0050569273c5]' failed. Entity was updated by another transaction concurrently.
        at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.concurrentUpdateDbEntityException(EnginePersistenceLogger.java:132)

And the workflow finally ends, if see BatchExecutorException

org.camunda.bpm.engine.ProcessEngineException: ENGINE-03083 Exception while executing Batch Database Operations with message '
### Error flushing statements.  Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #3) failed. 2 p
rior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: The DELETE statement conflicted with the REFERENCE constraint "ACT_FK_JOB_EXCEPTION". The conflict occurred in
database "Camunda", table "db_owner.ACT_RU_JOB", column 'EXCEPTION_STACK_ID_'.

And finally, the job executor seems to run anyway endless, probably because he reached the timeout “lockTimeInMillis”.

Thanks to lead me to a proper solution to manage async+long flows.

2

After reading this post, it reminds me that the fact to poll/wait on the main thread could “hurts” the job executor.

I made a workaround by polling in @Async method to isolate and it seems solve my issues.

May be a better way would be to send a url callback to the job service, so that when the job is done, a callback request can be done and fire (on another thread) the signal.

3

@ludoo0d0a can you use the external task pattern? https://docs.camunda.org/manual/7.9/user-guide/process-engine/external-tasks/

4