Hey @thorben I am circling back to this, and have some follow-ups:
You mentioned here:
That you can disable deploymentAware or register manually. If this is the case, when a shared engine is being used, is it typical to have deploymentAware disabled as a default?
What is the reasoning for deploymentAware to be enabled on the default camunda configuration?
How are camunda server restarts handled based on the default configuration? Based on the docs, and your comments above, the default settings would assume that if you have a async process, and you restart your server, then you must manually register the deployment? If yes, referring back to #2 above, why would this be the default behaviour of the engine?
If your processes depend on process application resources, you should not disable deploymentAware. If your process deployments are self-contained (e.g. only script tasks), it should be fine to do so.
This follows from the first answer. Developing process applications is the most frequent use case. Activating deploymentAware in the distributions by default avoids seeing class loading exceptions when using process applications. Especially as a getting started experience this would be awful.
If you have process applications, the engine recognizes the process applications automatically and makes registrations. If you don’t have process applications (or undeployed a process application but still want to execute jobs), then you must do manual registration in a deploymentAware setting.
@thorben what are your thoughts on making this env variable changeable? Default/if env is not provided then it defaults to current. Otherwise you can provide a env for this? (at least in the docker container)
@thorben, would it be valid to say that anyone using the engine in a Shared Engine environment, and is using deployments through the REST API, should have deploymentAware set to False?
People who struggle with REST API deployments and job execution will probably not look into a section on clustering for the solution to their problem. They will rather look into job execution documentation. Thus, it may be a good idea to document the jobExecutorDeploymentAware flag in the job execution section. There, we could simply document the behavior of this flag independent of cluster setups. The clustering section could then link there and refer to this description.
Im still a little puzzled by some of the terminology in this context and thus the expected behaviour. Hence let me give a few definitions so I can clarify some points. Context is Tomcat based, shared engine, not down to the column based multi-tenancy yet.
A node is a Tomcat Instance
A cluster is a set of nodes.
A logical engine is defined at the bpm-platform.xml level, thus a node can host many engine instances.
An engine instance uses a common database services.
A logical engine uses a defined DB schema within the DB service.
Hence if I deploy a BPMN resource via the REST API, I understand it will be stored in the DB repository associated with an engine. If the engine is configured to be deploymentaware, then an asynch continuation will not run as the deployment is not registered with the engine.
Hence for deployment aware to work, does that mean that as each node starts up, each engine instance builds an in memory map of the deployments associated with the node based on say examining the process application’s WAR file. What I am struggling with is, is it at the node level, and thus by implication the engine instance level that deployments are registered, or is it at the logical engine level, or is it at the Tomcat node level?
(I can see the deployment IDs passed across in the job executor select jobs SQL, I just dont follow where they are coming from…
Deployment registrations are managed per logical engine in your terms, or per ProcessEngine Java object. Registrations are not persisted, to keep any topology data about engine or cluster setup out of the database. The job executor registrations boil down to a set in ProcessEngineConfigurationImpl, see
In addition, the registrations of deployments with process applications is managed in ProcessApplicationManager, which is held by ProcessEngineConfigurationImpl, so this is also managed per logical engine.
I suspect the reason is because a server cannot know whether additional required resources have been deployed to the node or not. With a process application in a war file, the container can inform the server that the resources are there.
Perhaps an attribute could be added to the deployment entity, indicating if the deployment need be deployment aware or not. For those which are not deployment aware, ie no additional external resources, on engine startup it could build a list of non deployment aware deployments and register them into the server node. The down side of this is if a new deployment is made, how are all servers notified of the new deployment (introduce polling)? In addition, more thought is required as to the intrinsic race condition/overlap of process application driven deployment versus API driven deployment…
Having a deployment aware but deployment method agnostic cluster would be a nice feature to have…
I had similar problem.
After BPMN schema deploy over REST API, async jobs started perfectly.
After Camunda platform Docker restart, no jobs were executed.
In default camunda-bpm-platform Docker configuration (bpm-platform.xml) jobExecutorDeploymentAware value is true.
Just remove jobExecutorDeploymentAware setting from configuration file was my solution.
We’ve experienced this issue - can I ask - what are the downsides of setting jobExecutorDeploymentAware? It sounds from earlier discussion by @StephenOTT and @Webcyberrob that the REST deployment option requires some work to be a first class citizen? REST is certainly the more desirable deployment option for us - fits better with current deployment tools and pipelines. We are also planning to deploy Camunda via Docker containers.
My best understanding after reading the thread is that if deployment aware is switched off (which it seems is a requirement when deploying by REST), Camunda engines will need to be restarted to become aware of the new deployments?
Also, how does one go about
I’m struggling to find a description of this in the documentation.
I had the same error, and yes, if using REST API deployments, you must set this flag to false. There’s the explanation I received: The registration is always a local and in-memory state inside an engine. For deployments made via the REST API, the engine will not recreate that in-memory registration after restarting. So that the Job Executor won’t pick up any jobs to execute.
Sorry to resurrect an old thread, but wanted to communicate that we got burned by this situation today, as well. It took awhile to find this solution and I think it should be highlighted more clearly in the documentation. The Job Executor section describes why the flag should be True in a heterogeneous environment, but it does not at all mention the possible adverse affects of having it set to True in a container implementation using REST API deployments.
Would it make sense to have this value defaulted to False in the published container distribution?
It took me a few hours of sleep as well. My case was very similar:
When does It accurs:
Deployment with REST API
Most cases when your Application restart.
How to solve:
1- Verify your bpm-platform.xml file with this tag: <property name="jobExecutorDeploymentAware">false</property>
2 - Redeploy your APP (Docker, whatever)
