Process variables used for Process configuration stored in JSON/YML?


#1

Has anyone does any use cases or patterns for storing what i call “configuration variables” in a single process Variable and/or in external files such as JSON/YAML.

Example of configuration variables:

  1. N days before a reminder/Timer is executed
  2. Who certain tasks are assigned to
  3. Due date logic that is common across many tasks
  4. Basically anything that you would store in a process variable rather than hard coding it into the process and having to change the value in multiple locations throughout the process, or have to dig into scripts and task config to change values that would be easily changed in the configuration variable/YML. A sort of ENV equivalent but on a per process level.

Was thinking something like

or could be loaded through delegate as part of the start event.

then when you want to redeploy a process with the updates, in many cases it would be a change to the YML/JSON rather than the BPMN file.

Any thoughts on this? Best practices, words of warning ? :slight_smile:


Use same workflow for multiple business partners with overrrides for process variables
#2

anyone from @camunda have any thoughts? Working through the scenario at the moment and would love some feedback or just general thoughts.


#3

Hi @StephenOTT,

Sounds like a valid approach. Since loading the configuration is a purely technical activity, it may also be a candidate for an execution listener that is not visually represented in the model. Similar to the discussion you had with @Philipp_Ossler about determining a call activity’s called element dynamically, I think it’s fine to have dynamic values when they are not required to understand the process model. So for technical configuration it is fine (e.g. a mail server host name), for more business-relevant configuration, it might not be (e.g. a timer that is due in 5 days).

If you notice that we do not respond to best practice questions, that is typically because the Camunda folks active in this forum are mostly developers that only have limited experience in building solutions with Camunda BPM. For such questions, I always like to see other users sharing their experiences and thoughts.

Cheers,
Thorben


#4

Never a problem :slight_smile: Always happy to get Dev insight on “feasibility” and can worry about if it makes sense in practice later on :wink:


it may also be a candidate for an execution listener that is not visually represented in the model.

agreed


#5

Hi @StephenOTT,

Did you manage to load variables from a JSON file?
I’m also interested in something similar.

Thanks!


#6

@kontrag you should be able to load the JSON file with something like the following (based on the work I did here ):

Run this script in the Start Event execution listener

Script: Type: Javascript

var processDefinitionId = execution.getProcessDefinitionId();

var deploymentId = execution.getProcessEngineServices().getRepositoryService().getProcessDefinition(processDefinitionId).getDeploymentId();

var resource = execution.getProcessEngineServices().getRepositoryService().getResourceAsStream(deploymentId, 'MyConfigurationFileName.json');

var IOUtils = Java.type('org.apache.commons.io.IOUtils');
var String = Java.type('java.lang.String');

var myConfigFile = S(new String(IOUtils.toByteArray(resource), 'UTF-8'));

execution.setVariable("_configuration", myConfigFile);

The above would result in the _configuration process variable being created as type JSON. You would then use the SPIN accessors to get the specific configuration.

The above is not fully tested with the SPIN parser. So you may have to do:

You may have to do:

var myConfigFile = S(JSON.stringify(new String(IOUtils.toByteArray(resource), 'UTF-8')));

To break down what the above does:

var myConfigFileAsString = new String(IOUtils.toByteArray(resource), 'UTF-8');
var myStringifiedConfigFile = JSON.stringify(myConfigFileAsString);
var mySpinParsedConfigFile = S(myStringifiedConfigFile);

#7

Thanks @StephenOTT!

Where do you put your ‘MyConfigurationFileName.json’ file normally so the .getResourceAsStream can fetch it? Under src/main/resources?


#8

The file is deployed next to your bpmn file. In my use case I was deploying with the rest API


#9

@kontrag

Here is a full function that you can drop in:

/**
 * Load configuration file as a SPIN JSON variable in-memory and optionally as a process variable.
 *
 * @param string fileName The name of the configuration file in the deployment.
 * @param string key The top level JSON key in the configuration file that will be saved, and other keys/objects are omitted.
 * @param boolean persist Whether to save the configuration as a process variable.
 * @return SPIN JSON Object
 */
function loadConfig(fileName, key, persist)
{
  'use strict';

  if (typeof(persist) == 'undefined') {
    persist = false;
  }

  if (typeof(key) == 'undefined') {
    key = null;
  }

  var processDefinitionId = execution.getProcessDefinitionId();
  var deploymentId = execution.getProcessEngineServices().getRepositoryService().getProcessDefinition(processDefinitionId).getDeploymentId();
  var resource = execution.getProcessEngineServices().getRepositoryService().getResourceAsStream(deploymentId, fileName);

  var Scanner = Java.type('java.util.Scanner');

  var scannerResource = new Scanner(resource, 'UTF-8');

  var configText = scannerResource.useDelimiter('\\Z').next();
  scannerResource.close();

  var configAsJson = S(configText);

  if (key === null) {
    var config = configAsJson;
  } else {
    if (!configAsJson.hasProp(key)) {
      throw 'Key "' + key + '" does not exist.';
    }
    var config = configAsJson.prop(key);
  }

  if (persist) {
    execution.setVariable('_config', config);
  }

  return config;
}

loadConfig('config.json', 'myProcess', true);
// loadConfig('config.json');
// loadConfig('config.json', null, true);
// loadConfig('config.json', null, false);
// loadConfig('config.json', 'myprocess');
// loadConfig('config.json', 'myprocess', true);
// loadConfig('config.json', 'myprocess', false);

and a example of what the config.json file could look like:

https://github.com/StephenOTT/ProcessProjectTemplate/blob/master/resources/config.json

Edit: cleaned up function


How to read external configuration file in service task?
#10

Have you tried reading configuration in from a location outside the deployed BPMN artifact? I.e., to get the file from an arbitrary location on the server’s filesystem?

Sorry if anybody objects to me posting on this old thread.


#11

@tim take a look at the nashorn load() function. its mentioned through the forum (mainly by myself), and you can read up on it.

it lets you load from a url such as load('http://myurl.com/myfile.js and from the classpath such as: load('classpath:myJsFile.js')

https://wiki.openjdk.java.net/display/Nashorn/Nashorn+extensions


#12

If you run camunda as a Spring application, you can use configuration beans which can be referenced in expressions.

MyConfig.java:

@Component
@ConfigurationProperties(prefix = "my")
public class MyConfig {

    private String baseUrl;
    // ..getters and setters
}

application.yml:

my:
  baseUrl: http://localhost:8090

reference configuration bean property in bpmn:

<camunda:connector>
  <camunda:inputOutput>
    <camunda:inputParameter name="url">${myConfig.baseUrl}/myResource</camunda:inputParameter>
    <camunda:inputParameter name="method">POST</camunda:inputParameter>
    <camunda:inputParameter name="headers">
      <camunda:map>
        <camunda:entry key="Accept">application/json</camunda:entry>
        <camunda:entry key="Content-Type">application/json</camunda:entry>
      </camunda:map>
    </camunda:inputParameter>
    <camunda:inputParameter name="payload">{ "foo": [ "${myFormVar}" ] }</camunda:inputParameter>
    <camunda:outputParameter name="statusCode">statusCode</camunda:outputParameter>
    </camunda:inputOutput>
    <camunda:connectorId>http-connector</camunda:connectorId>
</camunda:connector>