Building custom tasklist (REST API)


#1

We are having our own UI to display BPMN tasks and using camunda task list REST API.

REST API takes ~>2+ to 6+ secs to retrieve tasks(with filters).

How can we build custom task list which can improve performance and response time should be <1 sec?

We need optimized tasklist REST api to achieve performance.


#2

you need to provide some more information.

How are you using filters? you are using HAL to return multiple variables?

You are only running 1 filter at a time or require multiple filters to be executed as sync?

have you taken a look at the architecture of: https://github.com/holunda-io/camunda-bpm-taskpool ?

How many tasks are you trying to return?


#3

@StephenOTT we are filtering tasks with multiple process variables and we are paginating tasks(50 tasks per page). i.e., maxResults=50

We are using this post-task-query to get tasks. We pass filter values in process variables part in the payload of that rest api.

Sample payload:

{
  "processDefinitionKey": "testworkflow",
  "tenantId": "tenant1",
  "processVariables": [
    {
      "variableName": "customerId",
      "operator": "eq",
      "variableValue": "support@tenant1.com"
    },
    {
      "variableName": "programName",
      "operator": "LIKE",
      "variableValue": "%COPY_TASKLIST TEST 1%"
    }
  ],
 "sorting": [
  {
    "sortBy": "startTime",
    "sortOrder": "desc"
  }]
}

In backend camunda creates a sql query(SELECT statement) with combination of DISTINCT + ORDERBY which causes slowness in execution.


#4

Hi @aravindhrs,

what about using some NOSQL Database like Elasticsearch (or a new single table in your database) to store and query the information of all tasks? Here you use a structure that fulfill your needs.

You can feed the database with the help of task listeners.

The Camunda task table is used only as a backup.

Hope this helps, Ingo


#5

It’s a good option. But already we have listeners in tasks. I believe camunda will not follow the execution order of listeners if we have configured multiple listeners.

I think it would be a good addition in future releases, if we add implementations for ordered execution of listeners and also modeler should provide option for configuring listeners in which order.

Lets say, ListenerA.class should be execute in executionOrder=1,ListenerB.class should be execute in executionOrder=2


#6

I believe the listeners will execute in the order they are defined in the xml.

The query you provided is going to have query load because you have non indexed columns being returned / being queried.

A possible solution can be to build a Execution Listener that attaches to each Activity Start, and attach a Transaction Listener. When the Transaction completes, you send the completed data into ES.

Here is a example:

public class MyExecutionListener implements ExecutionListener {

    @Override
    public void notify(DelegateExecution execution) throws Exception {
        String activityInstanceId = execution.getActivityInstanceId();

        Context.getCommandContext().getTransactionContext()
                .addTransactionListener(TransactionState.COMMITTED, commandContext -> {
                    
                    HistoricActivityInstanceEventEntity activityEntity = commandContext.getDbEntityManager()
                            .getCachedEntity(HistoricActivityInstanceEventEntity.class, activityInstanceId);

                    System.out.println(activityEntity.getExecutionId());
                });
    }
}

The above code is just a sample that you can optimize in different ways depending on your use case and needs. The take away is that you can create a transaction listener that will get the HistoricActivityInstanceEventEntity, (you could also query for the User Task Entity version of this), and then index that entity in ES. The Entities attributes should have all the info you need know if its a Create or Update for ES.

(the above code is non-public api, thus API ~could break in future)


#7

@Ingo_Richtsmeier i found the camunda elastic search plugin from github:

This plugin seems very old, last commit was May 11, 2015. Since 4 years no code commit to this repo.

Will this plugin work for latest camunda latest version 7.11 and future releases?


#8

As Stephen wrote above, Camunda-BPM-Tasklist is a CQRS-based system for creating projections on a large amount of tasks. Currently a Mongo and an In-Memory projections exist which work pretty fast - based on it, you could easily build an elastic-search projection. The basic mechanism of getting all task changes from the engine could be reused…

It is primarily aimed to be used in distributed scenarios (single task view over multiple engines) but also would support a single-node usage as well.

Taskpool is currently being developed and used and any further usage / use cases are highly welcome. So, if you are interested, just contact one of the authors:


#9
  1. I can say camunda-bpm-taskpool was a wonderful work. I need a few clarifications for further using this library which are,

  2. We would like to use opensource libraries, if it has the Apache License or any other open source license would be nice.

  3. I can see frequent commits, and would like to know whether its compatible with all the camunda releases and future releases?

  4. If i set historyLevel=full, will this extension will capture all the events and persist to external datastore(mongo or inmemory)?

  5. I was running camunda server as standalone server(homogeneous cluster) with shared database. So how can i configure this extension to capture all the events to persist into mongodb?


#10

Here are the answers:

  1. Taskpool uses APACHE 2 license: https://github.com/holunda-io/camunda-bpm-taskpool/blob/develop/LICENSE

  2. / 4. Taskpool consists of several parts, responsible for different aspects. The collection of tasks is solved in a component called Task-Collector. This is connected to the history stream and requires history=full to have all relevant changes (you can actually reduce it, if for example change of assignment is not of your interest) in your application. Currently we implemented two read-projections to support task queries: In-Memory or Mongo. As I told you before, you can easily implement another projection and use a different storage technology (e.g. Elastic) for this.

  3. Runinng it as a homogenous cluster is not a problem. You will activate the taskpool-collector on every node and then choose between several options how to proceed with them. Are you using Camunda SpringBoot?

  • You use Axon Server (https://axoniq.io/product-overview/axon-server) as a distribution
    infrastructure and just install your taskpool-core and taskpool-view nodes (at least two of each for high availability) and connect them. Both of them would use a shared datastore (Mongo). You may decide to run core and view on the same nodes (this is ok too)
  • You rely on other messaging (e.g. you already have Kafka) - then you deploy taskpool-core on your camunda node and deploy two view node connecting them via Kafka.

Please have a look on the documentation of taskpool (i’m refactoring it right now to have more details about possible architecture and use case scenarios).

If you have any questions, join our slack channel: https://join.slack.com/t/holunda/signup

Kind regards,

Simon