Sequence Counter - History Events - How does counter incrementing work?

[StephenOTT]

Looking at history events as part of JSON History Provider (usecase: ElasticSearch indexing) - #2 by StephenOTT.

And i see that Sequence Counter was added for sorting:
https://app.camunda.com/jira/browse/CAM-3387
https://app.camunda.com/jira/browse/CAM-3564, etc

@thorben @camunda, when i look at the events i see the sequence order incrementing by 2. Is this expected? or am i missing some specific logs?

[thorben]

Hi Stephen,

That should be fine. The only guarantee you have with sequence counters is that they are increasing for events that have occurred in a happens-before relationship.

Cheers,
Thorben

[StephenOTT]

So to confirm, counters can increment by 2? Or is it more like there was events that are not captured in the history classes that incremented the counter?

Also: it’s a counter always within the history class and the process instance ID?

[thorben]

@StephenOTT

So to confirm, counters can increment by 2?

Yes, that is what you should always assume when working with counters.

@StephenOTT

Or is it more like there was events that are not captured in the history classes that incremented the counter?

The implementation reason for counters is increasing here is probably that other history events have received the intermittent numbers. These events have either been written to other tables or not at all (due to history level).

@StephenOTT

it’s a counter always within the history class and the process instance ID?

Not necessarily. For example two historic activity instances of the same process instance can have the same counter value. What you can rely on is the following:

• When you have two history entries of the same process instance
• And they have occurred in strict sequence with respect to sequence flow, i.e. it is not possible they could have occurred in any other based on the BPMN.
• Then they can be ordered by the counter according to their order of occurrence

Examples:

1. You can meaninfully compare two historic activity instances by counter when the two activities are connected by plain sequence flows
2. You cannot meaninfully compare two historic activity instances by counter when the two activities are on parallel branches emerging from the same parallel gateway.
3. You can meaningfully compare a historic variable instance to a historic activity instance in the same scenario as example 1. However, this is currently not exposed via an API.

I hope that makes sense.

[StephenOTT]

@thorben

The implementation reason for counters is increasing here is probably that other history events have received the intermittent numbers. These events have either been written to other tables or not at all (due to history level).

So in my example image, I captured(supposed to have) all history events generated by camunda, and history level is set to full.

[StephenOTT]

@thorben

You can meaninfully compare two historic activity instances by counter when the two activities are connected by plain sequence flows
You cannot meaninfully compare two historic activity instances by counter when the two activities are on parallel branches emerging from the same parallel gateway.
You can meaningfully compare a historic variable instance to a historic activity instance in the same scenario as example 1. However, this is currently not exposed via an API.

Is sequence counters used for internal sorting by the rest API? If yes, how do you detect if it was something like parallel gateway vs plain sequence flows?

[thorben]

@StephenOTT

Is sequence counters used for internal sorting by the rest API? If yes, how do you detect if it was something like parallel gateway vs plain sequence flows?

You can ask the REST API to do that by setting the sorting criterion to occurrence when making a history query.

@StephenOTT

If yes, how do you detect if it was something like parallel gateway vs plain sequence flows?

Only by considering the BPMN diagram. From the query results alone, you cannot tell that two entries are not comparable.

By the way, you can find some documentation on this here: History and Audit Event Log | docs.camunda.org

[StephenOTT]

@thorben

By the way, you can find some documentation on this here: History and Audit Event Log | docs.camunda.org

Thank you! did not think there was any docs, and nothing was found from my quick search!