History cleanup - Issues

Camunda 7 Topics Discussion & Questions
1

Hello,

I am using Camunda version 7.10 at the moment (will be upgrading to 7.13 soon).
I am running it with the tomcat as a windows service.
I am using the engine’s rest API with my external C# services.
I am using postgreSQL 10.3

I have been trying to setup automatic history cleanup and no matter what I tried it does not seem to work. I read the other posts about it and tried to follow, I also read the documentation.

It seems that something is still missing.

This is what I added in the bpm-platform.xml file

  <property name="historyCleanupStrategy">endTimeBased</property>
  <property name="historyCleanupBatchWindowStartTime">00:01</property>
  <property name="historyCleanupBatchWindowEndTime">04:00</property>

On my different BPMN’s I got
<bpmn:process id="nameeee" isExecutable="true" camunda:historyTimeToLive="10">

When I am trying to send http requests to the camunda API about the cleanup it returns me what seem to be valid results.

Get configuration:

{

    "batchWindowStartTime": "2020-07-22T00:01:00.000+0300",

    "batchWindowEndTime": "2020-07-22T04:00:00.000+0300"

}

Get Jobs:

{

    "id": "abedefb2-5158-11e9-8321-00155d185251",

    "jobDefinitionId": null,

    "processInstanceId": null,

    "processDefinitionId": null,

    "processDefinitionKey": null,

    "executionId": null,

    "exceptionMessage": null,

    "retries": 3,

    "dueDate": "2020-07-22T00:01:00.000+0300",

    "suspended": false,

    "priority": 0,

    "tenantId": null,

    "createTime": "2019-03-28T14:54:49.498+0200"

}

I also tried manual removal which returned me this:

{

    "id": "b90db52c-4ef8-11e9-8142-509a4c11f669",

    "jobDefinitionId": null,

    "processInstanceId": null,

    "processDefinitionId": null,

    "processDefinitionKey": null,

    "executionId": null,

    "exceptionMessage": null,

    "retries": 3,

    "dueDate": "2020-07-21T16:32:40.245+0300",

    "suspended": false,

    "priority": 0,

    "tenantId": null,

    "createTime": "2019-03-25T14:22:57.505+0200"

}

A side note:
I read that when using manual delete I need to be admin. Do I need to add basic auth to the request to achieve that?

I also noticed another thing. When I query -

http://xxx:8089/engine-rest/engine/default/history/process-instance

[{

        "id": "2648157f-906b-11ea-9329-509a4c11f669",

        "businessKey": null,

        "processDefinitionId": "3b28dd82-8236-11ea-ba9a-509a4c11f669",

        "processDefinitionKey": "InvoiceBasedOnIntegratedOrder",

        "processDefinitionName": "InvoiceBasedOnIntegratedOrder",

        "processDefinitionVersion": 28,

        "startTime": "2020-05-07T17:00:46.638+0300",

        "endTime": "2020-05-07T17:00:51.363+0300",

        "removalTime": null,

        "durationInMillis": 4725,

        "startUserId": null,

        "startActivityId": "StartEvent_1",

        "deleteReason": null,

        "rootProcessInstanceId": "2648157f-906b-11ea-9329-509a4c11f669",

        "superProcessInstanceId": null,

        "superCaseInstanceId": null,

        "caseInstanceId": null,

        "tenantId": null,

        "state": "COMPLETED"

    }

or when I check the table public.act_hi_procinst

SELECT id_, proc_inst_id_, business_key_, proc_def_key_, proc_def_id_, start_time_, end_time_, removal_time_, duration_, start_user_id_, start_act_id_, end_act_id_, super_process_instance_id_, root_proc_inst_id_, super_case_instance_id_, case_inst_id_, delete_reason_, tenant_id_, state_
FROM public.act_hi_procinst ORDER BY end_time_

It seems that all of the instances have “removalTime”: null. I wonder if this might be part of the problem and if so how come is it null?

P.S.
I actually added endTimeBased strategy today. Before that I had the default option on.

2

If you’re using the endTimeBased strategy, the removal will not be set. Working with removal times happens to be slightly more convenient in 7.11 because there is a REST API call you can make to set the removal time after the fact. But for now, I think endTimeBased is your best bet.

It looks like your batch window is set correctly based on the due date of the cleanup job. When you run it manually, it looks like it is getting scheduled as soon as possible. That makes it seem like the TTL on the definitions isn’t set quite right or your expectations of what should be deleted isn’t matching what’s actually happening.

Double check that the historyTimeToLive is set on the process definition that corresponds to the instances you expect to delete. GET /process-definition (here) should give you a list of deployed process definitions with their TTL values. If you need to update the TTL on a definition that’s already deployed, you can. If you have an instance that has finished, and the end date + TTL has passed, that is not getting cleaned up, can you post the details of that process instance and its corresponding process definition here?

To answer your question about authorization, unless you’ve gone in and enabled authorization I wouldn’t expect you to have to do anything special (as evidenced by the scheduling of the cleanup job, I think you’re okay there).

1 Like
3

Thank you for the response!
Hopefully I will get back to work on it this week and also check the things you mentioned and update here.

4