Best pattern for non blocking HTTP requests

Camunda 7 Topics Discussion & Questions
1

Hi,

I would like to implement a non-blocking HTTP client that I can use in an embedded process engine. We have already an embedded process engine in a Scala Play Framework application. So I would like to use the Play WS HTTP client implementation which is based on Netty and which is able to delegate async execution to a Scala execution context. The client returns a Scala Future.

So I have thought of a solution that I can call from a ServiceTask. The client implementation should get its parameters (url, method, body, headers, timeouts) from process variables and it should also writes it output (responseStatus, respnseBody), after the future has completed, into process variables. On the Camunda side, it should not block the thread in which the process is executed.

Currently I have an implementation based on AbstractBpmnActivityBehavior.

class HttpRequestDelegate @Inject() (wsClient: WSClient)(
  implicit
  ec: ExecutionContext
) extends AbstractBpmnActivityBehavior {

  override def execute(execution: ActivityExecution): Unit = {
    (for {
      url <- getUrl(execution)
      method <- getMethod(execution)
      headers <- getHeaders(execution)
      body <- getBody(execution)
      requestTimeout <- getRequestTimeout(execution)
    } yield {
      wsClient
        .url(url)
        .withHttpHeaders(headers: _*)
        .withFollowRedirects(true)
        .withRequestTimeout(requestTimeout)
        .withBody(body)
        .withMethod(method)
        .execute().map { response =>
          execution.setVariable("responseStatus", response.status)
          execution.setVariable("responseBody", response.body)
          leave(execution)
        }.recover {
          case e: Throwable =>
            throw e
        }
    }).toTry.recover {
      case e: Throwable =>
        throw e
    }
  }
 ...
}

It will execute the request, but i’m not able to get the response variables in my test. The process instance is also marked as non ended even if I wait a few seconds.

I’m not sure that is the best approach to handle non-blocking requests. Maybe there is a better approach?

Best regards,
Christian

2

I’m not sure I understand everything correctly since there are things in your setup (Scala, Play WS Client) that I’m not familiar with. But I know the concept of the future, and that might be enough.

IIUC, the delegate is executed very quickly and is completed before the HTTP request is completed and the future does its work (set the variables). I.e. setting the variables in this activity might actually happen after the next activities have run. I’m not sure, but this might explain why the variables are “not set”.

As to why the process instance does not get to the end… Do you have a job executor running in the background? Are you sure all the necessary jobs are triggered?

3

I think the problem is, that the following code will be executed in another thread and so it looses the correlation to the executed process.

execution.setVariable("responseStatus", response.status)
execution.setVariable("responseBody", response.body)
leave(execution)

I’ve also tried to use a receive task after the request service task and correlate a message after the future completes successfully. But this also doesn’t work for the same reason.

So I think the external task pattern would be the only solution to handle a non-blocking HTTP request.

4

I don’t think this is true. These are object references, and they are valid regardless of the thread. There is no “correlation” here.

5