Long Polling to ‘Fetch and Lock’ in Camunda cluster

Camunda 7 Topics Discussion & Questions
1

Hello guys. Great to hear about long polling in Camunda BPM 7.9.0-alpha2!!!

How does the Long Polling work in 2 Camunda instances with shared DB?
Camunda by itself should poll DB for External tasks, right? What is the internal polling interval (e.g. each 500ms)?

Lets say I’m waiting on a fetchAndLock on CamundaA Rest API call. Start new Process using CamundaB Rest API call then. What is the delay I receive the External task from CamundaA fetchAndLock?

2

Hi @Andrey_Falkov,

at the moment there is no periodical interval to query for external task instances. The query is triggered whenever another long polling request is performed to /external-task/fetchAndLock.

When we built this feature, we assumed that an external task worker never comes alone. In an environment where multiple independent workers fetch and lock for external task instances, the query would be executed continuously.

I am very interested in your opinion of this approach? Does this feature cover your requirements?

Cheers,
Tassilo

1 Like
3

Thank you

Waiting fetchAndLock can return result only when another fetchAndLock request comes, right? Are there any other events that may result to fetchAndLock return? E.g. I’m waiting on a Topic and corresponding Process has just been started using Rest API.

4

Hey @Andrey_Falkov,

Waiting fetchAndLock can return result only when another fetchAndLock request comes, right?

The fetch and lock request is only “waiting”, if no external task instances are available in the moment of the request.

Are there any other events that may result to fetchAndLock return?

If one single request is pending at all the query is performed as soon as the timeout is due. Aside from this situation no other events will trigger the query.

I hope I managed to answer your questions?

Cheers,
Tassilo

5

Ok. Does it mean there may be 2 queries - one at the beginning and another after a timeout?

6

Hi @Andrey_Falkov,

exactly! This is how it works.

Cheers,
Tassilo

7

Not sure it is a good idea to query second time (after a timeout). As there will probably be another fetchAndLock just after the timeout…

while (true) {
var task = await camunda.fetchAndLock({asyncResponseTimeout=3000});
if (task != null)
processTask(task);
}

8

Hi Tassilo.

This behavior is actually not really what I expected. I thought that we at least give a local hint if a new external task is created (like with the Job Executor). Then you could have an arbitrary waiting interval, and if a new external task is created at least on this Camunda node the query returns immediately. This way it would feel a bit like streaming in the perspective of the client.

Understandable what I mean?

I actually thought this is already what we have built :slight_smile: That would allow to work with external tasks without adding too much latency to the calls.

Cheers
Bernd

9

Hey guys, the implementtion long polling is not finished yet. The current state is the first iteration in which the waiting request is not yet notified when a new task becomes available while waiting for the timeout. The plan is to do that before the 7.9 final release. In the last alpha release, the team shipped the first increment which did not yet have the full set of optimizations. But you know, release early, release often …

2 Likes
10

Just to document, because this forum post was just referenced in a conversation: This feature was fully implemented in 7.9 and the behavior changes accordingly (long polling returns when a new task becomes available).

1 Like
11

Is there any detail on how it is implemented now? When a new external task is created, it notifies the cluster through DB and get it returned in the long polling request?

If it notifies through DB, is there a polling job? If not, how another instance handling the external task query knows one is created?

12