Hi,
If we have jdbcBatchProcessing
set to false
like this:
<property name="processEngineName" value="default" />
<property name="dataSource" ref="bpmDataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseSchemaUpdate" value="true" />
<property name="jobExecutorActivate" value="true" />
<property name="history" value="full" />
<property name="authorizationEnabled" value="true" />
<property name="jobExecutorDeploymentAware" value="true" />
<property name="historyCleanupBatchWindowStartTime" value="00:00" />
<property name="historyCleanupBatchWindowEndTime" value="06:00" />
<property name="jdbcBatchProcessing" value="false" />
…
would we still expect batch statements like this?
ENGINE-14006 Exception while executing job b2540495-da50-11e9-9655-0242b5bbc4c5:
org.camunda.bpm.engine.ProcessEngineException: ENGINE-03004 Exception while executing Database Operation 'DELETE MessageEntity[b2540495-da50-11e9-9655-0242b5bbc4c5]' with message '
### Error updating database. Cause: java.sql.SQLTransactionRollbackException: (conn=29504152) Deadlock found when trying to get lock; try restarting transaction
### The error may involve org.camunda.bpm.engine.impl.persistence.entity.JobEntity.deleteMessage-Inline
### The error occurred while setting parameters
### SQL: delete from ACT_RU_JOB where ID_ = ? and REV_ = ?
### Cause: java.sql.SQLTransactionRollbackException: (conn=29504152) Deadlock found when trying to get lock; try restarting transaction'. Flush summary:
[
INSERT HistoricProcessInstanceEventEntity[7a622891-da51-11e9-9655-0242b5bbc4c5]
INSERT HistoricActivityInstanceEventEntity[Task_1irwi18:7a618c4b-da51-11e9-9655-0242b5bbc4c5]
INSERT ExecutionEntity[7a622891-da51-11e9-9655-0242b5bbc4c5]
INSERT VariableInstanceEntity[7a622892-da51-11e9-9655-0242b5bbc4c5]
INSERT VariableInstanceEntity[7a622893-da51-11e9-9655-0242b5bbc4c5]
INSERT VariableInstanceEntity[7a622894-da51-11e9-9655-0242b5bbc4c5]
INSERT VariableInstanceEntity[7a622895-da51-11e9-9655-0242b5bbc4c5]
INSERT MessageEntity[7a624fa7-da51-11e9-9655-0242b5bbc4c5]
DELETE MessageEntity[b2540495-da50-11e9-9655-0242b5bbc4c5]
DELETE_BULK deleteByteArrayNoRevisionCheck 7a530d2a-da51-11e9-9655-0242b5bbc4c5
UPDATE ExecutionEntity[b250cfb0-da50-11e9-9655-0242b5bbc4c5]
]
at org.camunda.bpm.engine.impl.db.EnginePersistenceLogger.flushDbOperationException(EnginePersistenceLogger.java:130)
at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flushDbOperations(DbEntityManager.java:345)
Or am I misinterpreting what a batch is, and what this flag controls?
Also, we are on Camunda 7.10.0. Does this ticket:
https://app.camunda.com/jira/plugins/servlet/mobile#issue/CAM-10126
Address anything related to the above deadlock?
Thanks,
Galen
tasso94
September 20, 2019, 6:01am
2
Hi Galen,
the error log you have shared is not related to jdbc batch processing. However, a deadlock was detected and therefore the transaction was rolled back.
Cheers,
Tassilo
Thanks Tassilo,
Any idea if upgrading to 7.12 (and getting CAM-10126) would help with the deadlock?
Thanks,
Galen
tasso94
September 20, 2019, 6:06am
4
Hi Galen,
CAM-10126 is unrelated to your deadlock situation.
Have you set the transaction isolation level of your database to READ_COMMITTED
?
Cheers,
Tassilo
Hi Tassilo,
Yes, we are using MariaDb 10.2,
and
SELECT @@GLOBAL.tx_isolation, @@tx_isolation, @@session.tx_isolation;
@@GLOBAL.tx_isolation @@tx_isolation @@session.tx_isolation
READ-COMMITTED READ-COMMITTED READ-COMMITTED
VERSION()
10.2.25-MariaDB
tasso94
September 20, 2019, 6:13am
6
Hi Galen,
are all indices according to the SQL create scripts present?
Cheers,
Tassilo
Galen_Hollins:
ACT_RU_JOB
Yes, I think so. We haven’t changed them from the camunda default create table scripts.
Here’s an example table:
CREATE TABLE `ACT_RU_JOB` (
`ID_` varchar(64) COLLATE utf8_bin NOT NULL,
`REV_` int(11) DEFAULT NULL,
`TYPE_` varchar(255) COLLATE utf8_bin NOT NULL,
`LOCK_EXP_TIME_` timestamp(3) NULL DEFAULT NULL,
`LOCK_OWNER_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
`EXCLUSIVE_` tinyint(1) DEFAULT NULL,
`EXECUTION_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`PROCESS_INSTANCE_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`PROCESS_DEF_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`PROCESS_DEF_KEY_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
`RETRIES_` int(11) DEFAULT NULL,
`EXCEPTION_STACK_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`EXCEPTION_MSG_` varchar(4000) COLLATE utf8_bin DEFAULT NULL,
`DUEDATE_` timestamp(3) NULL DEFAULT NULL,
`REPEAT_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
`HANDLER_TYPE_` varchar(255) COLLATE utf8_bin DEFAULT NULL,
`HANDLER_CFG_` varchar(4000) COLLATE utf8_bin DEFAULT NULL,
`DEPLOYMENT_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`SUSPENSION_STATE_` int(11) NOT NULL DEFAULT 1,
`JOB_DEF_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`PRIORITY_` bigint(20) NOT NULL DEFAULT 0,
`SEQUENCE_COUNTER_` bigint(20) DEFAULT NULL,
`TENANT_ID_` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`CREATE_TIME_` datetime(3) DEFAULT NULL,
PRIMARY KEY (`ID_`),
KEY `ACT_IDX_JOB_EXECUTION_ID` (`EXECUTION_ID_`),
KEY `ACT_IDX_JOB_HANDLER` (`HANDLER_TYPE_`(100),`HANDLER_CFG_`(155)),
KEY `ACT_IDX_JOB_PROCINST` (`PROCESS_INSTANCE_ID_`),
KEY `ACT_IDX_JOB_TENANT_ID` (`TENANT_ID_`),
KEY `ACT_IDX_JOB_JOB_DEF_ID` (`JOB_DEF_ID_`),
KEY `ACT_FK_JOB_EXCEPTION` (`EXCEPTION_STACK_ID_`),
KEY `ACT_IDX_JOB_HANDLER_TYPE` (`HANDLER_TYPE_`),
CONSTRAINT `ACT_FK_JOB_EXCEPTION` FOREIGN KEY (`EXCEPTION_STACK_ID_`) REFERENCES `ACT_GE_BYTEARRAY` (`ID_`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
tasso94
September 20, 2019, 6:20am
8
Hi Galen,
to find out if it helps to update to a newer minor version, I would recommend you to search through our issue tracker to spot deadlock issues related to the deletion of jobs for 7.10.0 that have been fixed.
Cheers,
Tassilo
Hi Tassilo,
Thanks. I’ve found a few that may be related. For example:
CAM-10360
CAM-9964
I’m not sure whether they may help, but I guess it’s worth a try to upgrade
Thanks,
Galen