Timers doesnt work after server restart

Hi,

The timers doesn’t work after camunda server restart. They work perfectly fine after deleting and redeploying the model. This approach wont be feasible in production since that would mean to delete all running instances and redeploy model again.

kindly guide me where could the problem be.
Note: I have enabled the variable suggested here: Timers not firing after server restart - #9 by StephenOTT. But still timers dint work after restart

Attached is my Model
ApprovalTeamResponse.bpmn (26.7 KB)

Hi,

How did you deploy your model, was it via the modeller or via the REST API?

If the engine is configured to be deployment aware, then on restart, those models deployed via the REST API or modeller will not be recognised as deployed in that engine instance.

This document section talks to deployment aware.

Camunda behaviours are fine in production, however you need to understand the architecture required of your particular architecture and the associated behaviours…

regards

Rob

1 Like

How did you deploy your model, was it via the modeller or via the REST API?
–> Models are deployed via Modeler and calls to external tasks are done via Rest API

If the engine is configured to be deployment aware, then on restart, those models deployed via the REST API or modeller will not be recognised as deployed in that engine instance.
–> I tried both with and without deployment aware flag, in both cases if server restarts, new external and user tasks works fine but already stuck tasks do not continue. Timers do not work unless redeployed. After restart the instances that were active before restart stay there(as seen on Admin Portal) but API calls do not seem to pick them. Everything including timers work fine once running instances are killed and we redeploy our models again

About Architecture:
Camunda server is deployed on only one node, its not heterogeneous or in cluster

Hi,

ARe there any incidents visible in cockpit? If a process has an incident after a timer, the engine may roolback to the timer so it appears the time is not working…

regards

Rob

Yes, following are still visible in cockpit but not processed by API.
image

Hi,
Your cockpit view shows active tokens in blue. Incidents wold show as red badges, so there doesn’t seem to be any incidents (eg failed jobs which have exceeded retries).

Am I right in assuming your model is classless? In other words you deploy the model from cockpit, and all service tasks are external so there is no code required to be deployed?

regards

Rob

Hi,

BTW I just had a close look at your model. Your timer definitions look like they use expressions such as “${requestorNotificationDelaySeconds}” - what should these expressions resolve to? eg Java Beans? Other?

regards

Rob

Am I right in assuming your model is classless?
→ Yes

These variables e.g. arDecisionTimeoutMinutes are passed while starting this model via API.

Few things i would like to share from cockpit that might give us some clue

  1. Due date passed and no retries done

    I have never seen these being retried(since they are not failed jobs). Do you think these retires can be a reason?
  1. Variables still have valid values after restart:

Hi @areebahmushtaq,

Could you please specify from where you have changed the value of “jobExecutorDeploymentAware” property

File path: camunda/server/apache-tomcat/webapps/camunda-invoice/WEB-INF/classes/META-INF/processes.xml
Lines by default:
< property name=“isDeleteUponUndeploy”> false < /property>
< property name=“isScanForProcessDefinitions”>false< /property>
Line added:
< property name=“jobExecutorDeploymentAware”>true< /property>
I have added space before ‘property’ keyword so that it becomes visible here

Hi @areebahmushtaq,

Is your process definition part of camunda-invoice process application?

I don’t think the answer would be yes as you already said that your model is deployed through the modeler.
In that case you should do the change in bpm-platform.xml file instead.

https://docs.camunda.org/manual/latest/reference/deployment-descriptors/descriptors/bpm-platform-xml/

Hi,

I agree with Hassang. I suspect your engine is still runnung in deployment aware mode which would explain the timer behaviour you see. You need to change the engine config. A shared engine, embeded engine, springboot and Camunda Run have slightly different configuration places so verify which engine architecture you are using to determine where the config is actually picked up from.

regards

Rob

You were right, it was about the file in which i was changing my variable, I changed ‘jobExecutorDeploymentAware’ to false in both files:

  1. camunda-bpm-tomcat/server/apache-tomcat/webapps/camunda-invoice/WEB-INF/classes/META-INF/processes.xml
  2. camunda-bpm-tomcat/server/apache-tomcat/conf/bpm-platform.xml

Note: My installation was full-distribution–> tomcat --> pre-packaged.
After restart my pending tasks and jobs were done and timers worked fine.

Thank you so much for generous support