Foreign key constraint fails

jamesfwood · November 18, 2018, 5:14pm

Hi everyone,

I’m getting a bunch of Incidents (40+ sometimes) created when running a model that has a Call Activity that calls another BPMN model basically 6000+ times in parallel. In the model parallel Call Activity I selected the “Multi Instance Async Before, Multi Instance Async After, Multi Instance Excluse” options. It doesn’t seem to fail if there are less than 2000 calls, but when it gets to around 5000 then it starts to fail with this error:

org.camunda.bpm.engine.ProcessEngineException: ENGINE-03083 Exception while executing Batch Database Operations with message ’

Error flushing statements. Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))

Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
java.sql.BatchUpdateException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
java.sql.SQLIntegrityConstraintViolationException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
java.sql.SQLException: Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
Query is: delete from ACT_GE_BYTEARRAY where ID_ = ‘08677313-eace-11e8-9658-0242d103390e’
'. Flush summary:
[
INSERT HistoricProcessInstanceEventEntity[08d354ac-eace-11e8-9658-0242d103390e]
INSERT HistoricActivityInstanceEventEntity[Task_1irwi18:08d21c2b-eace-11e8-9658-0242d103390e]
INSERT ExecutionEntity[08d354ac-eace-11e8-9658-0242d103390e]
INSERT VariableInstanceEntity[08d354ad-eace-11e8-9658-0242d103390e]
INSERT VariableInstanceEntity[08d354ae-eace-11e8-9658-0242d103390e]
INSERT VariableInstanceEntity[08d354af-eace-11e8-9658-0242d103390e]
INSERT MessageEntity[08d3a2d0-eace-11e8-9658-0242d103390e]
DELETE MessageEntity[4fca8ea9-eacd-11e8-9658-0242d103390e]
DELETE_BULK deleteByteArrayNoRevisionCheck 08677313-eace-11e8-9658-0242d103390e
UPDATE ExecutionEntity[4f86f64b-eacd-11e8-9658-0242d103390e]
]
at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.flushDbOperationsException(EnginePersistenceLogger.java:689)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:358)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperationManager(DbEntityManager.java:321)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:293)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:204)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:133)
at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:113)
at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:66)
at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:47)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:40)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:88)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:57)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ibatis.exceptions.PersistenceException:

Error flushing statements. Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))

Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:255)
at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.flushOperations(DbSqlSession.java:88)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:353)
… 14 more
Caused by: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:146)
at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:129)
at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:122)
at org.apache.ibatis.executor.CachingExecutor.flushStatements(CachingExecutor.java:114)
at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:253)
… 16 more
Caused by: java.sql.BatchUpdateException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.mariadb.jdbc.MariaDbStatement.executeBatchExceptionEpilogue(MariaDbStatement.java:282)
at org.mariadb.jdbc.MariaDbPreparedStatementClient.executeBatch(MariaDbPreparedStatementClient.java:294)
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114)
at com.sun.proxy.$Proxy4.executeBatch(Unknown Source)
at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:122)
… 20 more
Caused by: java.sql.SQLIntegrityConstraintViolationException: (conn=3558) Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:171)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:110)
at org.mariadb.jdbc.MariaDbStatement.executeBatchExceptionEpilogue(MariaDbStatement.java:279)
… 27 more
Caused by: java.sql.SQLException: Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
Query is: delete from ACT_GE_BYTEARRAY where ID_ = ‘08677313-eace-11e8-9658-0242d103390e’
at org.mariadb.jdbc.internal.util.LogQueryTool.exceptionWithQuery(LogQueryTool.java:119)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol$1.handleResultException(AbstractQueryProtocol.java:567)
at org.mariadb.jdbc.internal.protocol.AsyncMultiRead.call(AsyncMultiRead.java:140)
at org.mariadb.jdbc.internal.protocol.AsyncMultiRead.call(AsyncMultiRead.java:67)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
… 3 more

I can’t really figure out what is causing this or how to fix it. I’m using MariaDb version 10.1. I had this same problem in Camunda v7.7 and now also in Camunda v7.10 alpha4.

If you can help please let me know.

Thank you!

Jim

jamesfwood · December 2, 2018, 6:32pm

I haven’t gotten any responses… Do you think this could be an out of memory issue?

Thanks!

thorben · December 4, 2018, 12:40pm

Hi Jim,

There are plenty of potential causes of this. Doesn’t sound so much like a memory issue to me, but rather something related to parallel access to the same entities. For example: Let’s assume the following a job fails while at the same time the process instance is deleted. Now, the transaction handling the job failure commits the exception stacktrace, while the transaction deleting the process instance removes the job. Due to race conditions, the deleting transaction may not be aware that it must also remove the stacktrace before it deletes the job.

In order to understand the circumstances in which this occurs, you can enable some debug logging for these loggers:

org.camunda.bpm.engine.cmd => logs the start and end of all commands that the engine performs
org.camunda.bpm.engine.persistence => logs the identity of all entities that the engine inserts/deletes/updates in the database

Cheers,
Thorben

jamesfwood · December 6, 2018, 11:04pm

Hi Thorben,

Yes it sounds like a race condition issue to me. I’m basically running Camunda in a cluster system with many instances of Camunda grabbing external jobs when they can. I noticed that the more Camunda engines that I have running then the more incidences get created. If it is a race condition occurring, can you tell me a solution to prevent this from happening?

Thank you!

Jim

jamesfwood · December 10, 2018, 8:26pm

Hi,

I added the logging you suggested. Here is my log now:

PROJ : 2018-12-09 20:35:46,382 WARN [ pool-2-thread-2] camunda.bpm.engine.jobexecutor( 142) - ENGINE-14006 Exception while executing job 362a938e-fbf1-11e8-bcf6-0242da80d84b:
org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of ‘DELETE MessageEntity[362a938e-fbf1-11e8-bcf6-0242da80d84b]’ failed. Entity was updated by another transaction concurrently.
at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.concurrentUpdateDbEntityException(EnginePersistenceLogger.java:134)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.handleOptimisticLockingException(DbEntityManager.java:498)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.checkFlushResults(DbEntityManager.java:450)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:366)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperationManager(DbEntityManager.java:324)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:296)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:207)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:136)
at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:115)
at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:69)
at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:50)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:43)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:91)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:60)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
PROJ : 2018-12-09 20:35:46,382 DEBUG [ pool-2-thread-2] org.camunda.bpm.engine.cmd( 114) - ENGINE-13005 Starting command -------------------- ExecuteJobsCmd ----------------------
PROJ : 2018-12-09 20:35:46,382 DEBUG [ pool-2-thread-2] org.camunda.bpm.engine.cmd( 114) - ENGINE-13009 opening new command context
PROJ : 2018-12-09 20:35:46,416 DEBUG [ Thread-8] org.camunda.bpm.engine.cmd( 114) - ENGINE-13011 closing existing command context
PROJ : 2018-12-09 20:35:46,416 DEBUG [ Thread-8] camunda.bpm.engine.persistence( 114) - ENGINE-03006 Cache state after flush: [
/foreign key
PROJ : 2018-12-09 20:35:46,483 DEBUG [ pool-2-thread-6] camunda.bpm.engine.persistence( 114) - ENGINE-03010 SQL bulk operation: ‘DELETE’; Statement: ‘deleteByteArrayNoRevisionCheck’; Parameter: ‘00fb80f6-fbf2-11e8-bcf6-0242da80d84b’
PROJ : 2018-12-09 20:35:46,483 DEBUG [ pool-2-thread-6] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘UPDATE’; Entity: ‘ExecutionEntity[id=35c05fd6-fbf1-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 20:35:46,491 ERROR [ pool-2-thread-6] org.camunda.bpm.engine.context( 156) - ENGINE-16004 Exception while closing command context: ENGINE-03083 Exception while executing Batch Database Operations with message ’

Error flushing statements. Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails (`tesrocks`.`ACT_RU_JOB`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`))

Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails (`tesrocks`.`ACT_RU_JOB`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`))

java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
java.sql.SQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
'. Flush summary:
[
INSERT HistoricProcessInstanceEventEntity[01bd22f8-fbf2-11e8-bcf6-0242da80d84b]
INSERT HistoricActivityInstanceEventEntity[Task_1irwi18:01bc1187-fbf2-11e8-bcf6-0242da80d84b]
INSERT ExecutionEntity[01bd22f8-fbf2-11e8-bcf6-0242da80d84b]
INSERT VariableInstanceEntity[01bd22f9-fbf2-11e8-bcf6-0242da80d84b]
INSERT VariableInstanceEntity[01bd22fa-fbf2-11e8-bcf6-0242da80d84b]
INSERT VariableInstanceEntity[01bd22fb-fbf2-11e8-bcf6-0242da80d84b]
INSERT MessageEntity[01bd4a0c-fbf2-11e8-bcf6-0242da80d84b]
DELETE MessageEntity[362a6d78-fbf1-11e8-bcf6-0242da80d84b]
DELETE_BULK deleteByteArrayNoRevisionCheck 00fb80f6-fbf2-11e8-bcf6-0242da80d84b
UPDATE ExecutionEntity[35c05fd6-fbf1-11e8-bcf6-0242da80d84b]
]
org.camunda.bpm.engine.ProcessEngineException: ENGINE-03083 Exception while executing Batch Database Operations with message ’

Error flushing statements. Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails (`tesrocks`.`ACT_RU_JOB`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`))

@
/aa47deb3-fc06-11e8-bcf6-0242da80d84b
PERSISTENT ExecutionEntity[d685509c-fc04-11e8-bcf6-0242da80d84b]
PERSISTENT ProcessDefinitionEntity[process_day:1:32d117de-fa58-11e8-9011-0050568d04fe]
]
PROJ : 2018-12-09 23:03:39,164 DEBUG [ pool-2-thread-8] org.camunda.bpm.engine.cmd( 114) - ENGINE-13006 Finishing command -------------------- FailedJobListener$1 ----------------------
PROJ : 2018-12-09 23:03:39,164 DEBUG [ pool-2-thread-8] org.camunda.bpm.engine.cmd( 114) - ENGINE-13011 closing existing command context
PROJ : 2018-12-09 23:03:39,164 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03008 Flush Summary: [
DELETE MessageEntity[a9eee858-fc06-11e8-9114-0242d103390e]
UPDATE VariableInstanceEntity[d666a5b9-fc04-11e8-bcf6-0242da80d84b]
UPDATE VariableInstanceEntity[d666a5ba-fc04-11e8-bcf6-0242da80d84b]
UPDATE ExecutionEntity[d696410f-fc04-11e8-bcf6-0242da80d84b]
UPDATE ExecutionEntity[d6667ea6-fc04-11e8-bcf6-0242da80d84b]
UPDATE HistoricVariableInstanceEntity[d666a5b9-fc04-11e8-bcf6-0242da80d84b]
UPDATE HistoricVariableInstanceEntity[d666a5ba-fc04-11e8-bcf6-0242da80d84b]
]
PROJ : 2018-12-09 23:03:39,164 DEBUG [ pool-2-thread-8] camunda.bpm.engine.persistence( 114) - ENGINE-03006 Cache state after flush: [
PERSISTENT ByteArrayEntity[aa47deb3-fc06-11e8-bcf6-0242da80d84b]
PERSISTENT JobDefinitionEntity[32d117e0-fa58-11e8-9011-0050568d04fe]
PERSISTENT MessageEntity[d6db8771-fc04-11e8-bcf6-0242da80d84b]
PERSISTENT ProcessDefinitionEntity[process_day:1:32d117de-fa58-11e8-9011-0050568d04fe]
]
PROJ : 2018-12-09 23:03:39,164 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘DELETE’; Entity: ‘MessageEntity[id=a9eee858-fc06-11e8-9114-0242d103390e]’
PROJ : 2018-12-09 23:03:39,164 DEBUG [ pool-2-thread-8] camunda.bpm.engine.persistence( 114) - ENGINE-03008 Flush Summary: [
INSERT ByteArrayEntity[aa47deb3-fc06-11e8-bcf6-0242da80d84b]
UPDATE MessageEntity[d6db8771-fc04-11e8-bcf6-0242da80d84b]
]
PROJ : 2018-12-09 23:03:39,165 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘UPDATE’; Entity: ‘VariableInstanceEntity[id=d666a5b9-fc04-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,165 DEBUG [ pool-2-thread-8] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘ByteArrayEntity[id=aa47deb3-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,165 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘UPDATE’; Entity: ‘VariableInstanceEntity[id=d666a5ba-fc04-11e8-bcf6-0242da80d84b]’
@
/aa47deb3-fc06-11e8-bcf6-0242da80d84b
PERSISTENT ExecutionEntity[d685509c-fc04-11e8-bcf6-0242da80d84b]
PERSISTENT ProcessDefinitionEntity[run_retrieval:6:ec5fdc33-fbf3-11e8-9011-0050568d04fe]
PERSISTENT ProcessDefinitionEntity[process_day:1:32d117de-fa58-11e8-9011-0050568d04fe]
]
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03008 Flush Summary: [
INSERT HistoricProcessInstanceEventEntity[aa897ac0-fc06-11e8-bcf6-0242da80d84b]
INSERT HistoricActivityInstanceEventEntity[Task_1irwi18:aa88b76f-fc06-11e8-bcf6-0242da80d84b]
INSERT ExecutionEntity[aa897ac0-fc06-11e8-bcf6-0242da80d84b]
INSERT VariableInstanceEntity[aa897ac1-fc06-11e8-bcf6-0242da80d84b]
INSERT VariableInstanceEntity[aa897ac2-fc06-11e8-bcf6-0242da80d84b]
INSERT VariableInstanceEntity[aa897ac3-fc06-11e8-bcf6-0242da80d84b]
INSERT MessageEntity[aa89a1d4-fc06-11e8-bcf6-0242da80d84b]
DELETE MessageEntity[d6db8771-fc04-11e8-bcf6-0242da80d84b]
DELETE_BULK deleteByteArrayNoRevisionCheck aa47deb3-fc06-11e8-bcf6-0242da80d84b
UPDATE ExecutionEntity[d6a5f8da-fc04-11e8-bcf6-0242da80d84b]
]
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘HistoricProcessInstanceEventEntity[id=aa897ac0-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘HistoricActivityInstanceEventEntity[id=Task_1irwi18:aa88b76f-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘ExecutionEntity[id=aa897ac0-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘VariableInstanceEntity[id=aa897ac1-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘VariableInstanceEntity[id=aa897ac2-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘VariableInstanceEntity[id=aa897ac3-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,613 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘INSERT’; Entity: ‘MessageEntity[id=aa89a1d4-fc06-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,614 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03009 SQL operation: ‘DELETE’; Entity: ‘MessageEntity[id=d6db8771-fc04-11e8-bcf6-0242da80d84b]’
PROJ : 2018-12-09 23:03:39,614 DEBUG [ pool-2-thread-2] camunda.bpm.engine.persistence( 114) - ENGINE-03010 SQL bulk operation: ‘DELETE’; Statement: ‘deleteByteArrayNoRevisionCheck’; Parameter: ‘aa47deb3-fc06-11e8-bcf6-0242da80d84b’
/aa47deb3-fc06-11e8-bcf6-0242da80d84b
at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:122)
at org.apache.ibatis.executor.CachingExecutor.flushStatements(CachingExecutor.java:114)
at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:253)
… 16 more
Caused by: java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.mariadb.jdbc.MySQLPreparedStatement.executeBatch(MySQLPreparedStatement.java:293)
at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114)
at com.sun.proxy.$Proxy4.executeBatch(Unknown Source)
at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:122)
… 20 more
Caused by: java.sql.SQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.mariadb.jdbc.internal.SQLExceptionMapper.get(SQLExceptionMapper.java:132)
at org.mariadb.jdbc.internal.SQLExceptionMapper.throwException(SQLExceptionMapper.java:106)
at org.mariadb.jdbc.MySQLStatement.executeQueryEpilog(MySQLStatement.java:268)
at org.mariadb.jdbc.MySQLStatement.execute(MySQLStatement.java:296)
at org.mariadb.jdbc.MySQLPreparedStatement.execute(MySQLPreparedStatement.java:144)
at org.mariadb.jdbc.MySQLPreparedStatement.executeBatch(MySQLPreparedStatement.java:277)
… 26 more
Caused by: org.mariadb.jdbc.internal.common.QueryException: Cannot delete or update a parent row: a foreign key constraint fails (tesrocks.ACT_RU_JOB, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.mariadb.jdbc.internal.mysql.MySQLProtocol.getResult(MySQLProtocol.java:982)
at org.mariadb.jdbc.internal.mysql.MySQLProtocol.executeQuery(MySQLProtocol.java:1042)
at org.mariadb.jdbc.MySQLStatement.execute(MySQLStatement.java:289)
… 28 more

Any ideas?

Thank you!!

Jim

Yana · December 20, 2018, 1:48pm

Hi Jim,

These logs are not enough for investigation.
It doesn’t contain what happened with this job previously, how many times it was acquired, which commands were executed, etc.

jamesfwood · February 12, 2019, 9:53pm

I think it is related to History cleanup.

I’m using this call to set history time to live:

repositoryService.updateProcessDefinitionHistoryTimeToLive(deployedProcDef.getId(), Integer.parseInt(historyDaysToLive));

And I run this command every 6 hours:

historyService.cleanUpHistoryAsync(true);

Do you have a suggestion on how to avoid this while using history cleanup? Also, history cleanup is the only thing I run in batch mode.

Thanks!

Jim

Yana · February 20, 2019, 2:10pm

Hi Jim,

From what you previously wrote, I don’t see how it could be related to the History cleanup.

Is it possible to collect the traces mentioned above starting of the first occurrence of the problematic job?
That way we can do root causing of the problem, tracking down what happened to this job.

Best regards,
Yana

jamesfwood · May 15, 2019, 7:52pm

Ok I am finally able to reproduce this FK exception with this simple model.

Please run this and you will see the FK exception come up. It is all pure BPMN. The script task takes 10 minutes but it times out in 5 minutes (Camunda setting) and retries again and again. So basically, I think this might have something to do with retries referencing the same database records… I think maybe it kept some records alive in the database, and meanwhile Camunda threw it’s hands up and tried to cleanup the data, but still had some rows that were lingering…

workflow.bpmn (4.3 KB)

Please let me know when you have tested it and what should happen next.

Thank you!

Jim

Here is the stack trace:

org.camunda.bpm.engine.ProcessEngineException: ENGINE-03083 Exception while executing Batch Database Operations with message ’

Error flushing statements. Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (`cws_jw_dev`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`))

Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (`cws_jw_dev`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`))

java.sql.BatchUpdateException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (cws_jw_dev.act_ru_job, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
java.sql.SQLIntegrityConstraintViolationException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (cws_jw_dev.act_ru_job, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
java.sql.SQLException: Cannot delete or update a parent row: a foreign key constraint fails (cws_jw_dev.act_ru_job, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
Query is: delete from ACT_GE_BYTEARRAY where ID_ = ‘1ed76823-7746-11e9-8096-8c8590bd0ca6’
'. Flush summary:
[
INSERT HistoricVariableInstanceEntity[378451ce-7748-11e9-8096-8c8590bd0ca6]
INSERT HistoricJobLogEventEntity[9d2953a3-7749-11e9-8096-8c8590bd0ca6]
INSERT HistoricVariableUpdateEventEntity[378451cf-7748-11e9-8096-8c8590bd0ca6]
INSERT HistoricActivityInstanceEventEntity[EndEvent_1363m0d:9d290582-7749-11e9-8096-8c8590bd0ca6]
INSERT HistoricActivityInstanceEventEntity[ScriptTask_1uqqd6c:9d264661-7749-11e9-8096-8c8590bd0ca6]
INSERT HistoricActivityInstanceEventEntity[StartEvent_1:3782cb2c-7748-11e9-8096-8c8590bd0ca6]
INSERT HistoricActivityInstanceEventEntity[Task_11m86eq:3782cb2d-7748-11e9-8096-8c8590bd0ca6]
INSERT HistoricActivityInstanceEventEntity[Task_1yxxhsc:378451d0-7748-11e9-8096-8c8590bd0ca6]
DELETE IncidentEntity[ea456d96-7748-11e9-8096-8c8590bd0ca6]
DELETE MessageEntity[b71615c4-7744-11e9-9274-8c8590bd0ca6]
DELETE_BULK deleteByteArrayNoRevisionCheck 1ed76823-7746-11e9-8096-8c8590bd0ca6
DELETE ExecutionEntity[b714dd43-7744-11e9-9274-8c8590bd0ca6]
UPDATE HistoricIncidentEventEntity[ea456d96-7748-11e9-8096-8c8590bd0ca6]
UPDATE HistoricProcessInstanceEventEntity[b714dd43-7744-11e9-9274-8c8590bd0ca6]
DELETE_BULK deleteAuthorizationsForResourceId {resourceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, resourceType=8}
UPDATE_BULK updateHistoricActivityInstancesByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricTaskInstancesByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricVariableInstancesByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricDetailsByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricIncidentsByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateExternalTaskLogByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateJobLogByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateUserOperationLogByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateIdentityLinkLogByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateCommentsByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateAttachmentsByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateByteArraysByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricProcessInstanceEventsByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricDecisionInstancesByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricDecisionInputInstancesByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
UPDATE_BULK updateHistoricDecisionOutputInstancesByRootProcessInstanceId {rootProcessInstanceId=b714dd43-7744-11e9-9274-8c8590bd0ca6, removalTime=Fri May 17 12:42:46 PDT 2019}
]
at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.flushDbOperationsException(EnginePersistenceLogger.java:692)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:361)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperationManager(DbEntityManager.java:324)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:296)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:207)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:136)
at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:115)
at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:69)
at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:50)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:43)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:91)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:60)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ibatis.exceptions.PersistenceException:

Error flushing statements. Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (`cws_jw_dev`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`))

Cause: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (`cws_jw_dev`.`act_ru_job`, CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`))

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:255)
at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.flushOperations(DbSqlSession.java:90)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:356)
... 14 more

Caused by: org.apache.ibatis.executor.BatchExecutorException: org.camunda.bpm.engine.impl.persistence.entity.VariableInstanceEntity.deleteByteArrayNoRevisionCheck (batch index #7) failed. 6 prior sub executor(s) completed successfully, but will be rolled back. Cause: java.sql.BatchUpdateException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (cws_jw_dev.act_ru_job, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:146)
at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:129)
at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:122)
at org.apache.ibatis.executor.CachingExecutor.flushStatements(CachingExecutor.java:114)
at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:253)
… 16 more
Caused by: java.sql.BatchUpdateException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (cws_jw_dev.act_ru_job, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.mariadb.jdbc.MariaDbStatement.executeBatchExceptionEpilogue(MariaDbStatement.java:296)
at org.mariadb.jdbc.MariaDbPreparedStatementClient.executeBatch(MariaDbPreparedStatementClient.java:290)
at sun.reflect.GeneratedMethodAccessor201.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114)
at com.sun.proxy.$Proxy3.executeBatch(Unknown Source)
at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:122)
… 20 more
Caused by: java.sql.SQLIntegrityConstraintViolationException: (conn:185) Cannot delete or update a parent row: a foreign key constraint fails (cws_jw_dev.act_ru_job, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:152)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:118)
at org.mariadb.jdbc.MariaDbStatement.executeBatchExceptionEpilogue(MariaDbStatement.java:293)
… 27 more
Caused by: java.sql.SQLException: Cannot delete or update a parent row: a foreign key constraint fails (cws_jw_dev.act_ru_job, CONSTRAINT ACT_FK_JOB_EXCEPTION FOREIGN KEY (EXCEPTION_STACK_ID_) REFERENCES ACT_GE_BYTEARRAY (ID_))
Query is: delete from ACT_GE_BYTEARRAY where ID_ = ‘1ed76823-7746-11e9-8096-8c8590bd0ca6’
at org.mariadb.jdbc.internal.util.LogQueryTool.exceptionWithQuery(LogQueryTool.java:113)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol$1.handleResultException(AbstractQueryProtocol.java:307)
at org.mariadb.jdbc.internal.protocol.AsyncMultiRead.call(AsyncMultiRead.java:139)
at org.mariadb.jdbc.internal.protocol.AsyncMultiRead.call(AsyncMultiRead.java:67)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
… 3 more

jamesfwood · May 15, 2019, 8:03pm

I think this might be relevant too:

2019-05-15 04:26:42,262 WARN [ pool-3-thread-8] camunda.bpm.engine.jobexecutor( 142) - ENGINE-14006 Exception while executing job a727b0da-76c2-11e9-8096-8c8590bd0ca6:
org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of ‘DELETE MessageEntity[a727b0da-76c2-11e9-8096-8c8590bd0ca6]’ failed. Entity was updated by another transaction concurrently.
at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.concurrentUpdateDbEntityException(EnginePersistenceLogger.java:134)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.handleOptimisticLockingException(DbEntityManager.java:498)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.checkFlushResults(DbEntityManager.java:450)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:366)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperationManager(DbEntityManager.java:324)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:296)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:207)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:136)
at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:115)
at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:69)
at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:50)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:43)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:91)
at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:60)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

jamesfwood · May 15, 2019, 9:53pm

You have to wait like 25-35 minutes to see the FK exception.

jamesfwood · May 16, 2019, 7:56pm

I created a JIRA ticket for this bug.

Here:

https://app.camunda.com/jira/browse/CAM-10325

Thank you,

Jim

Sonali · July 5, 2021, 12:44pm

@jamesfwood - Hi …I am running into similar kind of problem . What resolution did you get ?
Getting FK constraint error for ACT_RU_VARIABLE when camunda trying to delete entry from ACT_RU_EXECUTION

Appreciate any pointers . Thanks !

system · January 30, 2024, 11:27am