Delete broken deployment from cockpit

adat · January 10, 2018, 4:21pm

I have been testing multiple version of deployment and one of them had a infinite loop related error. I have deleted the application from tomcat, but unable to delete the broken deployment process . I have tried the following:

Stop the running process suing cockpit and delete with cascade option selected - no luck
Reset the db - no luck
Keep db down and delete the deployment on cockpit - no luck.
Delete the app from tomcat/ tomact work folders and try to remove the app from cockpit - no luck.
Removed all the camunda apps from tomcat and copied a fresh one - still the camunda process application shows up in cockpit and I am unable to delete.

Where does camunda store this in tomcat? How can it be deleted?

Thank
adat

Yana · January 11, 2018, 7:49am

Hello Adat,

Which camunda version does you use?
Do you use any errors in the log after you tried to delete it from cockpit?
If you still see the deployment in cockpit, please try to delete it again with cascade option selected. After that upload the catalina log file.

Best regards,
Yana

Yana · January 11, 2018, 7:52am

Hi again,

Another thing which you can try is to suspend the problematic process definition first and then try to delete the deployment.

Best regards,
Yana

adat · January 11, 2018, 1:00pm

I use 7.8 I have tried all those options including suspend and delete. It seems like it is storing it outside somewhere in tomcat. The only option I can think of now to get rid of this instance is to re install tomcat with a new camunda depployment

Adat

adat · January 11, 2018, 3:26pm

Here is how you can produce this error.

Create a process starting with a timer that can repeat forever say every 1 min interval R/PT1M.
Deploy the process.
Make some changes the process and redeploy with new version.
Try to delete the processes that are repeating (suspend the process using cockpit and cascase delete).
The process refuses to get deleted

Yana · January 12, 2018, 12:09pm

Hi Adat,

I tried it and the deletion is working fine for me.
Could you please explain what do you see when you tried to delete it? An error or nothing happens.
Please attach the log trace after the attempt to delete it. Without the log we are just guessing what it could go wrong.

Best regards,
Yana

adat · January 12, 2018, 3:21pm

Yana -

Here is the log.

Finished :
Could not delete deployment: ENGINE-03004 Exception while executing Database Operation ‘DELETE ExecutionEntity[0a92fd65-f619-11e7-999d-6c3be5133377]’ with message ’ ### Error updating database. Cause: org.h2.jdbc.JdbcSQLException: Referential integrity constraint violation: “ACT_FK_VAR_EXE: PUBLIC.ACT_RU_VARIABLE FOREIGN KEY(EXECUTION_ID_) REFERENCES PUBLIC.ACT_RU_EXECUTION(ID_) (‘0a92fd65-f619-11e7-999d-6c3be5133377’)”; SQL statement: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ? [23503-190] ### The error may involve org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution-Inline ### The error occurred while setting parameters ### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ? ### Cause: org.h2.jdbc.JdbcSQLException: Referential integrity constraint violation: “ACT_FK_VAR_EXE: PUBLIC.ACT_RU_VARIABLE FOREIGN KEY(EXECUTION_ID_) REFERENCES PUBLIC.ACT_RU_EXECUTION(ID_) (‘0a92fd65-f619-11e7-999d-6c3be5133377’)”; SQL statement: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ? [23503-190]’. Flush summary: [ INSERT HistoricJobLogEventEntity[6be4bb6a-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6be6b73b-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6be7055c-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6be7537d-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6be7a19e-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6be7efbf-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6be9c480-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6be9eb91-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bea39b2-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bea60c3-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bea87d4-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bead5f5-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6beafd06-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6beb4b27-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6beb7238-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bec0e79-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bec5c9a-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6becaabb-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6becf8dc-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bed1fed-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bed6e0e-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bed951f-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bede340-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bee0a51-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bee3162-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bee5873-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bee7f84-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6beea695-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6beef4b6-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricJobLogEventEntity[6bef1bc7-f7ab-11e7-bdc2-6c3be5133377] INSERT HistoricDetailVariableInstanceUpdateEntity[b3b8afbe-f619-11e7-999d-6c3be5133377] DELETE HistoricDetailVariableInstanceUpdateEntity[
This deployment will be deleted.
There are 5 running process instances which belong to this deployment. In order to delete this deployment it is necessary to enable the Cascade flag.
Cascade

Skip Custom Listeners

Skip IO Mappings
Are you sure to delete the given deployment?
Close

Yana · January 15, 2018, 8:25am

Hi Adat,

It seems that the trace is not complete.
Could you please upload the catalina log file, which can be found under CAMUNDA_HOME\server\apache-tomcat-8.0.47\logs. In case you see a lot of files there, please check for this error: JdbcSQLException: Referential integrity constraint violation.

Best regards,
Yana

adat · January 15, 2018, 1:01pm

Yana

I did see that it was the db referential integrity error. After I could not delete this instance successfully like I mentioned initially I dropped all the db data and recreated it from scratch same with the tomcat cache. So my question was if the db Ian dropped it should have cleaned up the instance but it is holding this deployed file copy somewhere else in tomcat and when I try to delete this on cockpit it refers to the db to clean it. The db does not have any reference. It could be something to do with the transaction that is being held somewhere on tomact or that is what I guess what is hppening. I had seen the upload option. I will upload the trace file but it is the same as I have copied over just truncated to fit in the post

Thank
Adat

Yana · January 15, 2018, 2:22pm

Hi Adat,

If you cleanеd up (“revert to clean state”) the database and tomcat, you should not see the processes from the past.
Camunda stores the process only in the database and on the file system if it is deployed as an application.
I found out a “tomcat undeploy doesn’t work” topic but it is quite old and it doesn’t seem to be related to the case - https://stackoverflow.com/questions/20743916/undeploy-web-application-from-tomcat-doesnt-remove-the-deployed-directory

Best regards,
Yana

mppfor_manu · January 16, 2018, 6:06pm

Have you tried deleting it through the REST interface? That has worked for me in the past with the Cockpit GUI wouldn’t remove a deployment.

adat · January 16, 2018, 7:54pm

@mppfor_manu

That was good tip, but I got a resource error when I tries the url. When I run the GET is does list it as the process in the system, but delete fails.

http://localhost:8080/engine-rest/process-definition/key/HumanTaskProcess/delete?cascade=true

HTTP Status 404 - Could not find resource for relative : /process-definition/key/HumanTaskProcess/delete of full path: http://localhost:8080/engine-rest/process-definition/key/HumanTaskProcess/delete?cascade=true

mppfor_manu · January 16, 2018, 8:12pm

Couple of thoughts here. You said this was a broken “deployment”, not a specific process. If you are trying to remove a deployment, any processes running from that deployment must be stopped and/or deleted.

One thing that you might want to try an HTTP DELETE on this using something like curl. Example:

curl -X DELETE http://localhost:8080/engine-rest/engine/default/deployment/{id}

You should get an HTTP 204 back from this.

The last resort would be a direct database deletion, but this is dangerous if you don’t know precisely what you are doing. I’ve deleted jobs this way. To do it, you’ll need to determine all of the references the deployment might have and then perform the delete in the proper order. Doing this on a deployment would be difficult.

adat · January 16, 2018, 9:07pm

Issue is Resolved. Thank you for all your replies.

The issue was that is have H2 in the server mode and configured via jndi into tomcat. Tomcat had a folder called “camunda-h2-dbs” inside “camunda-bpm-tomcat-7.7.0\server\apache-tomcat-8.0.24\bin” folder. I deleted that file and restarted the DB and tomcat. There was no impact on the db schemas I had… They were all intact. The rouge instance of the process deployment is now gone!!

Correction!!!

The DB schemas and custom tables were intact , but the user/group data that was configured using Admin app was all deleted. As it is a dev server the impact was not huge. but have to be careful in other env if a lot of users and groups are configured

system · January 30, 2024, 11:05am