FetchAndLock using asyncResponseTimeout returns empty body


#1

Using postman I can call the REST API for fetchAndLock. If I don’t use asyncResponseTimeout, I get an empty array in the body. If I use asyncResponseTimeout, then the body returned is empty, after the timeout.
This causes the External Task client library to throw an exception:

2018-11-02 16:21:06.205 DEBUG 4972 --- [criptionManager] org.apache.http.wire                     : http-outgoing-0 >> "POST /workflow/rest/external-task/fetchAndLock HTTP/1.1[\r][\n]"

2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “User-Agent: Camunda External Task Client[\r][\n]”
2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “Content-Type: application/json[\r][\n]”
2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “Content-Length: 253[\r][\n]”
2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “Host: localhost:8080[\r][\n]”
2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “Connection: Keep-Alive[\r][\n]”
2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “Accept-Encoding: gzip,deflate[\r][\n]”
2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “Authorization: Bearer xxx[\r][\n]”
2018-11-02 16:21:06.205 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “[\r][\n]”
2018-11-02 16:21:06.206 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 >> “{“workerId”:“b-627faed0-1a4d-4a9f-9ecd-6ef6fc39b2c9”,“maxTasks”:1,“asyncResponseTimeout”:30000,“topics”:[{“topicName”:“myTopic”,“lockDuration”:5000,“variables”:[“x”,“y”,“z”],“businessKey”:null}]}”
2018-11-02 16:21:36.237 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 << “HTTP/1.1 200 [\r][\n]”
2018-11-02 16:21:36.238 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 << “conversationId: 50587581-550f-4fc9-92c1-853c4258d7f0[\r][\n]”
2018-11-02 16:21:36.238 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 << “Set-Cookie: JSESSIONID=3EDFA399EB9034E6CF7DD485A5D11395; Path=/bpg-csp-workflow; HttpOnly[\r][\n]”
2018-11-02 16:21:36.238 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 << “Content-Type: application/json[\r][\n]”
2018-11-02 16:21:36.238 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 << “Content-Length: 0[\r][\n]”
2018-11-02 16:21:36.239 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 << “Date: Fri, 02 Nov 2018 16:21:36 GMT[\r][\n]”
2018-11-02 16:21:36.239 DEBUG 4972 — [criptionManager] org.apache.http.wire : http-outgoing-0 << “[\r][\n]”
2018-11-02 16:21:36.287 ERROR 4972 — [criptionManager] org.camunda.bpm.client : TASK/CLIENT-03001 Exception while fetch and lock task.

org.camunda.bpm.client.impl.EngineClientException: TASK/CLIENT-02005 Exception while mapping json object to response dto class ‘class [Lorg.camunda.bpm.client.task.impl.ExternalTaskImpl;’
at org.camunda.bpm.client.impl.EngineClientLogger.exceptionWhileMappingJsonObject(EngineClientLogger.java:47) ~[camunda-external-task-client-1.0.0.jar:1.0.0]
at org.camunda.bpm.client.impl.RequestExecutor.deserializeResponse(RequestExecutor.java:140) ~[camunda-external-task-client-1.0.0.jar:1.0.0]
at org.camunda.bpm.client.impl.RequestExecutor$1.handleEntity(RequestExecutor.java:115) ~[camunda-external-task-client-1.0.0.jar:1.0.0]
at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:73) ~[httpclient-4.5.5.jar:4.5.5]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:223) ~[httpclient-4.5.5.jar:4.5.5]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165) ~[httpclient-4.5.5.jar:4.5.5]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140) ~[httpclient-4.5.5.jar:4.5.5]
at org.camunda.bpm.client.impl.RequestExecutor.executeRequest(RequestExecutor.java:82) ~[camunda-external-task-client-1.0.0.jar:1.0.0]
at org.camunda.bpm.client.impl.RequestExecutor.postRequest(RequestExecutor.java:68) ~[camunda-external-task-client-1.0.0.jar:1.0.0]
at org.camunda.bpm.client.impl.EngineClient.fetchAndLock(EngineClient.java:68) ~[camunda-external-task-client-1.0.0.jar:1.0.0]
at org.camunda.bpm.client.topic.impl.TopicSubscriptionManager.fetchAndLock(TopicSubscriptionManager.java:126) [camunda-external-task-client-1.0.0.jar:1.0.0]
at org.camunda.bpm.client.topic.impl.TopicSubscriptionManager.acquire(TopicSubscriptionManager.java:95) [camunda-external-task-client-1.0.0.jar:1.0.0]
at org.camunda.bpm.client.topic.impl.TopicSubscriptionManager.run(TopicSubscriptionManager.java:81) [camunda-external-task-client-1.0.0.jar:1.0.0]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
at [Source: (org.apache.http.impl.io.EmptyInputStream); line: 1, column: 0]
at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4145) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4000) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3070) ~[jackson-databind-2.9.6.jar:2.9.6]
at org.camunda.bpm.client.impl.RequestExecutor.deserializeResponse(RequestExecutor.java:136) ~[camunda-external-task-client-1.0.0.jar:1.0.0]
… 12 common frames omitted
Using curl, I can recreate the issue.

curl --include http://localhost:8080/workflow/rest/engine/default/external-task/fetchAndLock -H ‘Accept: application/json’ -H ‘Authorization: Bearer xxx’ -H ‘Content-Type: application/json’ -H ‘cache-control: no-cache’ -d ‘{
“workerId”:“Nick”,
“maxTasks”:1,
“usePriority”:true,
“asyncResponseTimeout”: 10000,
“topics”:
[{“topicName”: “myTopic”,
“lockDuration”: 10000,
“variables”: [“x”]
}]
}’
HTTP/1.1 200
conversationId: 1ded4c04-d147-442d-a8b7-b92699921596
Set-Cookie: JSESSIONID=C634E0A76D81FE9A729CA76EE4EBA34A; Path=/workflow; HttpOnly
Content-Type: application/json
Content-Length: 0
Date: Fri, 02 Nov 2018 16:38:42 GMT

curl --include http://localhost:8080/workflow/rest/engine/default/external-task/fetchAndLock -H ‘Accept: application/json’ -H ‘Authorization: Bearer xxx’ -H ‘Content-Type: application/json’ -H ‘cache-control: no-cache’ -d ‘{
“workerId”:“Nick”,
“maxTasks”:1,
“usePriority”:true,
“topics”:
[{“topicName”: “myTopic”,
“lockDuration”: 10000,
“variables”: [“x”]
}]
}’
HTTP/1.1 200
conversationId: fb11c393-fe1a-46a8-9415-58e182a73550
Set-Cookie: JSESSIONID=3AD57C22ADDF400BB196828059BD1726; Path=/workflow; HttpOnly
Content-Type: application/json
Content-Length: 2
Date: Fri, 02 Nov 2018 16:39:01 GMT

[]


#2

I guess nobody thinks this is a defect?


#3

Hi Nick,

I tried to reproduce the problem but I didn’t succeed.
If there are external tasks available, are you able to get the expected response body?

Cheers,
Tassilo


#4

This problem occurs only if there are no external tasks available. The body returned from the REST call is different if the asyncResponseTimeout is used. This is a problem for the External Task Client library, as it always throws an exception when the body is empty (rather than the correct JSON for an empty list).

Our experience is that this has two effects:

  1. Nasty exceptions being logged by the external task client library.
  2. We seem to lose the ability to receive an external task via the external task client, when one is pending, but only after an undetermined length of time.

Now to answer your question, I believe that the server does have the correct body when there are external tasks available.

Even though the problem manifests via the External Task client library, I think the correct fix is to make the REST service return a consistent JSON for the empty list in the body.

BTW, I am using 7.9 and the spring boot starters for rest and webapp (and also the dependency order workaround) so that our server has the correct REST engine library.

Thanks,

Nick


#5

An update on this problem. When I tested the 7.10.0 docker image, it worked as expected. I passed the asyncResponseTimeout in the body of the fetchAndLock call, and after the appropriate time, I got an empty list back in the body
Unfortunately we don’t use the docker image, but we build ourselves a spring boot application, so I upgraded our build to the following:
springBootVersion = ‘2.1.1.RELEASE’
camundaSpringBootStarterVersion = ‘3.2.1’
camundaVersion = ‘7.10.0’
camundaSpinVersion = ‘1.6.5’
camelVersion = ‘2.23.1’
groovyVersion = ‘2.5.5’
spockVersion = ‘1.2-groovy-2.5’
and
implementation(“org.camunda.bpm.springboot:camunda-bpm-spring-boot-starter-rest:${camundaSpringBootStarterVersion}”){
_ exclude group: ‘org.camunda.bpm.springboot’, module: ‘camunda-bpm-spring-boot-starter-test’_
}
implementation “org.camunda.bpm.springboot:camunda-bpm-spring-boot-starter-webapp:${camundaSpringBootStarterVersion}”

In my built version, the defect remains. When I call fetchAndLock with asyncResponseTimeout provided, in my spring boot application I get an empty body back!

I will try attaching to the docker image and seeing if it has different jars than the app I’m building.

Nick.


#6

The md5 checksums in the docker image don’t match the md5 of the camunda-engine-rest-jaxrs2-7.10.0.jar.
In the central Maven repository the md5 (http://central.maven.org/maven2/org/camunda/bpm/camunda-engine-rest-jaxrs2/7.10.0/camunda-engine-rest-jaxrs2-7.10.0.jar.md5) is:
$ md5sum camunda-engine-rest-jaxrs2-7.10.0.jar
f01112ca1e9a34471b24eafeca0eb16a *camunda-engine-rest-jaxrs2-7.10.0.jar

In the docker image its:
bash-4.4$ md5sum camunda-engine-rest-jaxrs2-7.10.0.jar
8c08bce1ef513aa0ec23e00b3d9b1f00 camunda-engine-rest-jaxrs2-7.10.0.jar

Any idea where the docker image gets its copy of the jar from?

Nick.


#7

Hi @NickTheArchitect,

about which docker image are you talking? Could you post a link?

Cheers,
Tassilo


#8

https://hub.docker.com/r/camunda/camunda-bpm-platform
The github repo is at https://github.com/camunda/docker-camunda-bpm-platform

HTH, Nick


#9

The one that works, from the docker image, is from the file https://app.camunda.com/nexus/content/repositories/camunda-bpm/org/camunda/bpm/tomcat/camunda-bpm-tomcat/7.10.0/camunda-bpm-tomcat-7.10.0.tar.gz
When extracted the md5sum is:
8c08bce1ef513aa0ec23e00b3d9b1f00 *./tmp/server/apache-tomcat-9.0.12/webapps/engine-rest/WEB-INF/lib/camunda-engine-rest-jaxrs2-7.10.0.jar
This is clearly a better version of the jar that that available from Maven central (http://central.maven.org/maven2/org/camunda/bpm/camunda-engine-rest-jaxrs2/7.10.0/camunda-engine-rest-jaxrs2-7.10.0.jar), which has a md5 checksum of:
f01112ca1e9a34471b24eafeca0eb16a
@tasso94 - Is it possible to get the better camunda-engine-rest-jaxrs2 deployed into Maven central? If only as 7.10.1, or something?

Thanks, Nick


#10

Hi @NickTheArchitect,

I will look into the artefacts and come back to you tomorrow at the latest.

Cheers,
Tassilo


#11

Hi @NickTheArchitect,

I still cannot reproduce your problem.

This are the steps I have performed:

  1. I use spring-boot-starter-rest:3.2.1
  2. No External Tasks are available to ‘Fetch and Lock’
  3. I perform a POST request against http://localhost:8080/rest/external-task/fetchAndLock with the following payload:
    {"maxTasks": 1, "workerId": "aWorkerId", "asyncResponseTimeout": 20000, "topics": [{"topicName": "aTopicName", "lockDuration": 3000}]}
    

Observed behavior
After the asyncResponseTimeout is due, I get the following response: []

Could you please provide an example project with a unit test case that reproduces your problem?

Cheers,
Tassilo