Optimistic Locking Exception thrown during parallel tasks execution

Camunda 7 Topics Discussion & Questions
2

Hi Senthil,

Could you please share an example of your process.

Best regards,
Yana

3

Hi Yana,

PFB the bpmn file for parallel execution.
loan-approval.bpmn (9.5 KB)

I am trying with sequential execution but still facing the same issue with MariaDB 10.
loan-approval.bpmn (3.8 KB)

Please provide your suggestions here, we are totally blocked…

Thanks
Senthil

4

Hi Senthil,

Your process is working for me.
Could you please mark the service tasks as “async before” and “exclusive” and try again?

Best regards,
Yana

5

Hi Yana,

Actually, this process is working fine with MariaDB 5.x. But not working with MariaDB 10.2.x.
Have you tried with MariaDB 10.2.x.

Thanks
Senthil

6

Hi Senthil,

And what about h2, have you tried with it?
Also I did ask you which camunda version do you use?

Best regards,
Yana

7

Hi Yana,

It seems the sequential work flow is working fine with h2 db.
I have not tested with parallel gateway.

The issue exist with both 7.8.0 and 7.9.0 release of camunda

It will be great if we can get some quick suggestions on this. Because we are totally blocked and none of the workflow is executing with MariaDB 10.2.x release.

Thanks
Senthil

8

Hi Yana,

All workflows are getting failed with the error - Optimistic Locking exception when integrating camunda engine with MariaDB 10.x release. Any sequential / parallel execution is failing.

Error during workflow execution :

An error happened while submitting the task form :
Cannot submit task form 8190f28c-6eeb-11e8-b622-fa163e5b9340: ENGINE-03005 Execution of ‘UPDATE ExecutionEntity[818ea893-6eeb-11e8-b622-fa163e5b9340]’ failed. Entity was updated by another transaction concurrently.

It is throwing optimistic locking Exception.

The same workflow is working fine till camunda release 7.8.0-alpha5 with Maria DB 10.x
And it is working fine with camunda 7.8.0 / 7.9.0 release and h2 database.

But it is not working from camunda 7.8.0-alpha6 release and Maria DB 10.x.

I have created a bug
https://app.camunda.com/jira/browse/CAM-9145

I have done a basic debugging with Camunda 7.9.0 engine.

The below statement returns the updateCount as “-2”. The update count should be “1” for avoiding optimistic locking exception.

flushResult = persistenceSession.flushOperations();
Method – protected void flushDbOperations(List operationsToFlush)
File – org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager

Really it is a blocking issue.

Transaction Log shows there is a try to apply the lock on ACT_GE_PROPERTY twice

2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
MySQL thread id 3508, OS thread handle 140480266315520, query id 56760 demo cleaning up
Trx read view will not see trx with id >= 10270, sees < 10269
TABLE LOCK table db03.ACT_GE_PROPERTY trx id 10270 lock mode IX
RECORD LOCKS space id 224 page no 3 n bits 80 index PRIMARY of table db03.ACT_GE_PROPERTY trx id 10270 lock_mode X locks rec but not gap
Record lock, heap no 14 PHYSICAL RECORD: n_fields 5; compact format; info bits 0
0: len 9; hex 6e6578742e64626964; asc next.dbid;;
1: len 6; hex 00000000281e; asc ( ;;
2: len 7; hex 4e0000021628f0; asc N ( ;;
3: len 3; hex 323031; asc 201;;
4: len 4; hex 80000003; asc ;;

—TRANSACTION 10269, ACTIVE 2328 sec
2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 3504, OS thread handle 140481065899776, query id 56744 demo cleaning up
TABLE LOCK table db03.ACT_GE_PROPERTY trx id 10269 lock mode IX
RECORD LOCKS space id 224 page no 3 n bits 80 index PRIMARY of table db03.ACT_GE_PROPERTY trx id 10269 lock_mode X locks rec but not gap
Record lock, heap no 5 PHYSICAL RECORD: n_fields 5; compact format; info bits 0
0: len 15; hex 6465706c6f796d656e742e6c6f636b; asc deployment.lock;;
1: len 6; hex 000000001f17; asc ;;
2: len 7; hex f6000001dd0150; asc P;;
3: len 1; hex 30; asc 0;;
4: len 4; hex 80000001; asc ;;

9

Hi Senthil,

If 7.8.0-alpha5 is working, this problem could be related to the batch processing.
What is your db configuration? Is it jdbcBatchProcessing disabled?
Could you please try to disable the jdbcBatchProcessing [1].

[1] https://docs.camunda.org/manual/7.8/user-guide/process-engine/database/#jdbc-batch-processing

Best regards,
Yana

10

Hi Yana,

After disabling jdbcBatchProcessing, the workflow is executed properly.
But Why jdbcBatchProcessing (with value ‘true’) configuration is not working here ?

As per my understanding, this issue still needs to be addressed.

Thanks
Senthil

11

Hi Senthil,

My colleague pointed me that such issue is known with some maria db driver versions:
https://app.camunda.com/jira/browse/CAM-8891
What is your driver version, could it be the same problem?

Best regards,
Yana

12

Hi Yana,

Thanks for the update.
Yes, It is a maria db driver issue.
As you specified it is same as https://app.camunda.com/jira/browse/CAM-8891.

Camunda 7.9.0 - Maria DB 10.2.x - Maria db driver above 2.0.3 is not working (Meaning till Maria DB java client 2.0.3 no issues)

Thanks
Senthil

13

Hi Yana,

We have received the following from MariaDB engineers:

The “Entity was updated by another transaction concurrently” error message is internal to Camunda.
I might be wrong, but my guess would be the new batching implementation introduced in connector 2.1.0 when the maria db server is >=10.2.

Example :to simplify, executing 2 updates in batch :
Implementation before connector 2.1.0 : Query was send one after the other :
• => send query “UPDATE myTable SET someField = ‘newValue’, optimisticField = optimisticField +1 WHERE id = 1 and optimisticField = 100”
• => send query “UPDATE myTable SET someField = ‘newValue2’, optimisticField = optimisticField +1 WHERE id = 2 and optimisticField = 50”
• <= server send back data (one row changed, and other infos)
• <= server send back data (one row changed, and other infos)

Implementation by default since connector 2.1.0 :
• => send query “UPDATE myTable SET someField = ?, optimisticField = optimisticField +1 WHERE id = ? and optimisticField = ?” and parameters : ‘newValue’, 1, 100 / ‘newValue2’, 2, 50
• <= server send back data (2 row changed, and other infos)

Speed in new implementation is largely better, BUT there is one limitation: connector doesn’t know the row changed unitary.
So in the previous example, statement.executeBatch in the previous implementation would be [1, 1]. In the new implementation, the result will be [-2,-2] (-2 stands for Statement.SUCCESS_NO_INFO) meaning that batch succeeded, but without knowing the result of each execution. (if an exception is thrown, no change has been done)

That kind of result follows the JDBC standard.

The problem here must be that optimistic updates probably check that value to ensure that update has been successful.

So the conclusion is, to avoid such kind of optimistic locking exception, camunda should handle such updateCount result (-2 - Statement.SUCCESS_NO_INFO) as well. As of now, it is verifying whether the updateCount result is “1” or not, if 1 it is success otherwise it will throw Optimistic locking exception.

Thanks
Senthil

1 Like
14

Any update for the above issue ?

Thanks
Senthil

15

Hi Senthil,

The state of the ticket https://app.camunda.com/jira/browse/CAM-8891 reflects our progress on it. You can add yourself as a watcher to be notified whenever state changes. Thanks for your analysis by the way, this will probably help us when we address the issue.

Cheers,
Thorben

16

Hi Thorben,

Any update on this issue.
https://app.camunda.com/jira/browse/CAM-8891
The above issue is still open. Can you provide any tentative date when the fix is available.

Thanks
Senthil

17

Hi Senthil,

This issue is currently not scheduled and I cannot provide a fix date. Should you be a Camunda enterprise customer, I recommend you raise a support ticket and request that we schedule this ticket.

Best regards,
Thorben

18

Hi Thorben,

This issue is still open
https://app.camunda.com/jira/browse/CAM-8891

I want to become a contributor and fix this bug.
Is it ok to start this issue.

Thanks
Senthil

19

Hi Senthil,

That’s great. If you already have an idea how to achieve this, feel free to hack away and raise a pull request over at https://github.com/camunda/camunda-bpm-platform. Even if it is not complete, a pull request makes discussing the solution idea easier.

If you would like to discuss things upfront (e.g. to validate your understanding), feel free to do this via a discussion here on the forum, best in a separate thread.

Cheers,
Thorben

1 Like
20

Thanks for the update Thorben.
I will dig into more and make pull request.

Thanks
Senthil

21