Function as a Service API

We are working on a Function as a service API / Middleware / wrapper around Camunda BPMN processes that return variables.

would be interested to get some feedback on the design and features.

Overall goal is to provide a way to control the Rest API response based on the BPMN returned variables.

Building on the scenarios discussed with

This came to mind today:

Where the Confirm Link Process. With the ability to simply control the Body, status code, and headers response, and the ability to do GET with query params that are translated into the /start camunda endpoint, we end up with a REALLY flexible solution for all sort of different callback scenarios that are routed with camunda.

This scenario only becomes interesting (at least to me) because of the ability to control the JSON response. In all our scenarios Camunda is never exposed to the outside (its always proxied by another security/transformation layer. So the ability to have a flexible return opens up a lot of options for technical users to deploy workflows without the need to bring it lots of heavy layers or lots of staff/knowledge.

As a follow-up to this r&d testing

here is our results so far:

Everything is setup as microservice communications:

  • Everything runs on docker.
  • Microservice communication is routed through the proxy
  • Client is postman or whatever browser/web clinet.
  • Proxy is nginx
  • Running Camunda 7.7
  • Services (GitHub - DigitalState/Services: The DigitalState Services Microservice) manages the communication to the Camunda microservice and provides pre-processing (of the client request) and post-processing (camunda’s default response).

Web Sequence overview:


In this example we repurpose the example from:

and we turn the GIS query into a function and only return the needed data. (its just a simple use-case with a publicly accessible web service.


  • The client request that is processed by Services is stored in a SPIN json variable named “request”.
  • The Response that we want to configure the client response with is stored in a variable named “response”, and that variable is picked up by services, while using withVariablesInReturn=true, and Services processes this variable to provide a client response.

Request JSON object:

{
    "method": "POST",
    "query": {
        "lat": "45.448948",
        "long": "-73.544454"
    },
    "headers": {
        "accept-encoding": [
            "gzip, deflate"
        ],
        "user-agent": [
            "PostmanRuntime/6.4.1"
        ],
        "accept": [
            "application/json"
        ],
        "content-type": [
            "application/json"
        ],
        "authorization": [
            "Bearer ...."
        ],
        "postman-token": [
            "...."
        ],
        "cache-control": [
            "no-cache"
        ],
        "content-length": [
            "16"
        ],
        "x-forwarded-port": [
            "80"
        ],
        "x-forwarded-ssl": [
            "off"
        ],
        "x-forwarded-proto": [
            "http"
        ],
        "x-forwarded-for": [
            "...."
        ],
        "x-real-ip": [
            "...."
        ],
        "connection": [
            "close"
        ],
        "host": [
            "...."
        ],
        "x-php-ob-level": [
            0
        ]
    },
    "body": {
        "test": 123
    }
}

You can see that the Request Params and Body are placed into their respective properties in the request + the headers.

The Response Object:

{
    "status": {
        "code": 200
    },
    "headers": {
        "Content-Type": "application/json"
    },
    "body": {
        "name": "Verdun",
        "number": 12
    }
}

Postman View


  • The Script task is configured as a Javascript External resource.
  • The .js file is uploaded with the bpmn file in the deployment.
  • We are using Jsoup for the HTTP request to carto.com
  • I did not add any sort of error handling, but you can easily see where the try and catches would go to return error responses.
    The Js file contains the following rough sample code:
function generateHeaders()
{
  var headers = {
    "Content-Type":"application/json",
  }
  return headers
}

function getBorough(lat, long)
{
  with (new JavaImporter(org.jsoup))
  {
    var doc = Jsoup.connect('https://stephenrussett.carto.com:443/api/v2/sql')
                    .method(Java.type('org.jsoup.Connection.Method').GET)
                    .header('Accept', 'application/json')
                    .header('Content-Type', 'application/json')
                    .data('q', 'SELECT name, number FROM montreal_boroughs WHERE ST_Intersects(montreal_boroughs.the_geom,CDB_LatLng(' + lat + ',' + long + '))')
                    .timeout(30000)
                    .ignoreContentType(true)
                    .execute()

    var resBody = JSON.parse(doc.body())
    return resBody
  }
}

function spinify(body)
{
  var parsed = JSON.parse(body)
  var stringified = JSON.stringify(parsed)
  var spin = S(stringified)
  return spin
}

function getLatLong()
{
  var params = execution.getVariable('request').prop('query')
  if (params.hasProp('lat') && params.hasProp('long')){
    var lat = params.prop('lat').value()
    var long = params.prop('long').value()
    return {
      "lat": lat,
      "long": long
    }
  }
}

function setResponse(statusCode, headers, body)
{
  var response = {
    "status": {
      "code": statusCode
    },
    "headers": headers,
    "body": body
  }

  var responseSpin = S(JSON.stringify(response))
  execution.setVariable('response', responseSpin)
}

var latLong = getLatLong()
var borough = getBorough(latLong.lat, latLong.long)
// 45.508948,-73.554454
var body = borough['rows'][0]

setResponse(200, generateHeaders(), body)

The response from the Carto.com HTTP request is:

{
    "rows": [
        {
            "name": "Ville-Marie",
            "number": 20
        }
    ],
    "time": 0.019,
    "fields": {
        "name": {
            "type": "string"
        },
        "number": {
            "type": "number"
        }
    },
    "total_rows": 1
}

and you can see in the line var body = borough['rows'][0] in the JS code above where we get the specific object and subsequently add it into the body property for the response to the client.


Few notes and thoughts:

  1. Services microservice runs php7 using symfony 3 / api-platform.com.
  2. Services microservice is running in “dev mode” (/app_dev.php/)
  3. Response time for total round trip (in dev mode) was anywhere from 200ms to 400ms.
  4. Overall speed considerations are on-going. Looking at various optimizations
  5. Going to be testing speed impacts from various BPMN designs using single transactions
  6. Still to add further error handling in the script as well as on the Services Microservice
  7. Also going to look at proxying all the requests from camunda to external servers, through the proxy: so when camunda talks to carto.com it is being proxied rather than a direct connection.

Here is another usage example that uses BPMN Errors to generate Error Responses:

process_with_error_task

Where in the Determine Borough script task we have code like this:

...
var BpmnError = Java.type('org.camunda.bpm.engine.delegate.BpmnError')

if (borough['rows'].length >= 1){
  var body = borough['rows'][0]
  setResponse(200, generateHeaders(), body) // Generates success response
} else {
  throw new BpmnError('bad_data'); // activates the BPMN error with the error code "bad_data"
}

and the Set Response script task has the following script:

function generateHeaders()
{
  var headers = {
    "Content-Type":"application/json",
  }
  return headers
}

function setResponse(statusCode, headers, body)
{
  var response = {
    "status": {
      "code": statusCode
    },
    "headers": headers,
    "body": body
  }

  var responseSpin = S(JSON.stringify(response))
  execution.setVariable('response', responseSpin)
}

var body = {
  "error": "Borough Not Found",
  "message": "The Lat and Long were not in any boroughs in Montréal"
}

setResponse(200, generateHeaders(), body)

You can see duplicated code between the two scripts, so there are some sharing of common functions/a common library that can be loaded for easier response data.


Also you could just remove the second script task and place the error generation response as a listener in the BPMN error, sequence flow, or the end event:

function_test_bpmn-error


Postman Responses:


BPMN Error event configuration:

1 Like

@StephenOTT - thanks very much for sharing these real life scenarios. Interesting problems to solve.

1 Like