Pattern Review: DMN Looping for Array Input


#1

Looking for some community feedback:

I have a scenario I came across where you have a list you want to iterate through and find all of the matches.

The pattern that quickly hacked together was to use DMN to find all of the matches in a Collect List usage of the DMN.

Mocked Example:

You have a list of “Systems” and you want to get the list of the Managers of those Systems.

Notice that you could have the same “Managers” of different systems. In this scenario lets assume that only unique hits are supported, but we could also have scenarios where we have more than 1 manager per system and thus use Collect List Hit policy.

The complexity came as (far as i can tell), you cannot inject a “array” into a DMN. Example: Send a array of ["System 1", "System 3"], and have the DMN tell you the outputs/hits. The expected/desired output would have been ["user 1", "user 3"].

A scenario where this pattern seemed to be desired is when you want to get a list that use in a Multi-Instance sub-process. Example Scenario: A user fills out a form for System Access to various computers system in the organization. That form has a listing of “Systems” that are checked. This creates a array of “systems”. And lets assume that each system has a different “Manager”/user who needs to review the request.

something like:

So in this example a Form would have been filled out and the DMN determines which users need to review the request and the desired action is that each Manager of the systems would get their own Task to complete with the relevant information.

I believe I could have structured the DMN differently to support the desired list, but it did not seem manageable. Assume that we have hundreds of rules in the DMN with System to User mappings. The cleaner the file the better.

So in this BPMN I did the following:

I generate a fictional array in the Script task with Groovy:

def collection = ["System 1", "System 3"]
execution.setVariable("collection", collection)

def jsonValue = S('{ "values":[] }')

execution.setVariable("combinedResult", jsonValue)

The jsonValue is used a placeholder to append DMN outputs as we will see next.

I created a DMN and set it to Multi-Instance Sequential (basically iterate over the collection of “Systems” that would have been checked off in the Start Form).

The DMN Map Decision Result is set for SingleResult

I then used a listener to grab the DMN Output at the end of each execution of the DMN instances.

var combinedResult = execution.getVariable("combinedResult");
var dmn_output = execution.getVariable("dmn_output")["user"];

combinedResult.prop("values").append(dmn_output)

execution.setVariable("combinedResult", combinedResult)

I used SPIN to append the DMN instances output to the “combinedResult” variable. So basically after each instance of the DMN execution, the combinedResult variable would be appended with the DMN’s output value.

I then set the Multi-Instance sub-process to use the array elements as a collection:

I created simple placeholder User Tasks that were assigned to the same user (for testing purposes) but appended the Element Variable value into the Task Name.

so we get:

So overall:

This pattern allowed me to use DMN in basically a “Loop Over a Array of Values and combined DMN instance outputs into a single collection”.

What are peoples thoughts? A better more simple way? I had found reference to this: How can pass multiple values to input variable in request body, but it seemed more complicated to manage.
Other than the Execution Listener, there is not “sneaky” usage going on.

Files for anyone that wants to test:

array_test.dmn (1.4 KB)
array-input-dmn.bpmn (7.7 KB)


DMN JSON Object Parsing + Single Object for All Inputs
How to maintain all varible in loop?
Multi instance task (REST API)
Results from multiple DMN tables
Splitting batch with sequential multiple instance
Wrong output of DMN in result variable within multi instance sub process task
Business Rule Task
How to Accomplish Multi-instance I/O Mapping
What about super global process variable?
Appending to parent process variable
Task Forms in Tasklist
Best Practice For Populating Multi-Instance List Variable
List How can work in Camunda with static list
Multi-Instance loop over JSON array
How can I fetch json values from all the completed instances in a parallel Multi-Instance User Task Form?
How can I fetch json values from all the completed instances in a parallel Multi-Instance User Task Form?
Task Forms in Tasklist
How to add different inputs to MultiInstance call activity
Server Error: SPIN/JACKSON-JSON-01005 Unable to create node for object of type 'Undefined’
How to pass array to DMN input variable
Reset form fields for form used in multi-instance process
How can I assign a Multi-Instance User Task to multiple users?
Server Error: SPIN/JACKSON-JSON-01005 Unable to create node for object of type 'Undefined’
Need advise on modeling an usecase
#2

Here is a updated version that uses JSON objects and javascript throughout:

array_test.dmn (1.4 KB)
array-input-dmn.bpmn (7.8 KB)

Few major changes were needed to accommodate JSON objects:

  1. adding .value() to the input expression of the DMN. So now the expression for the input is: collElement.value().

  2. The Collection field of the Business Rule Task, in the Multi-Instance section was set to: ${collection.prop("collection").elements()}. An interesting aspect was that when the elements are passed to the DMN they are still passing as JSON rather than a String. And this is occurring even though the JSON is not valid json. @camunda this is a possible bug? Basically each element that was passed in the DMN was Typed as JSON but when you inspected the data in the cockpit it would show the value of the JSON object as something like System 1. No array, no JSON brackets, etc. So it would appear as a String.

  3. Array Generation was changed to:

var collection = S('{ "collection" : ["System 1", "System 3"] }');
execution.setVariable("collection", collection)

var jsonValue = S('{ "values":[] }')

execution.setVariable("combinedResult", jsonValue)

DMN table groovy
Best way to store variables in multiple subprocesses
#3

@Niall do you have any insight? Have you done something like this before during consulting?


Use a DMN table in a service task as a function?
#4

Hi Stephen,

I used your pattern recently. It works well. And It become a little bit easier if you do it with Java listeners, because you don’t need the type mapping.

I havn’t seen an easier solution to decide on an array of values.

Cheers, Ingo


New Release of the FEEL extension 1.5.0
DMN Rule - Single Rule "Any of" Value Count Limit?
Scope of local variable of Multi instance task
#5

I have also been exploring working with a Collection of Collection, where the inner Collection is evaluated by multiple Input columns. Has some interesting possible use cases such as evaluating a ‘Shopping Cart of Items’, where each item has attributes.


How to return a variable form an embedded form on a multi-instance subprocess
#6

I have implemented this pattern. It is indeed a good one and works well.


#7

Do you have any ideia on how to make a contains ANY ITEM between arrays? I have an array of inputs and i need to output only the FIRST rule to have ANY item in my input array.

Ex: my input it [a,b,c] e i have 3 rules in dmn:

  • [g,f,i] output FIRST
  • [f,u,b] output SECOND
  • [o,p,a] output THIRD

In this case, it would output SECOND, coz it have the letter B and it is in input array. Im going crazy with this hehe


#9

Hi @StephenOTT,

In your example, you define the Element Variable “collNname” of the multi-instance Subprocess.
I have a similar example (instead of a subprocess I have a normal User Task) and my Element Variable (I call it “orderline”) has the following value:
{"orderlineID":1,"orderlinepages":300,"orderlineprice":50}

I try to access this variable in the task form:

<form class="form-horizontal" role="form">
    <div class="control-group">
        <label class="control-label" for="orderlinespagesField">Pages</label>
        <div class="controls">
            <input id="orderlinespagesField" class="form-control" ng-model="orderline.orderlinespages" readonly="readonly"/>
        </div>
    </div>


<script cam-script type="text/form-script">
var variableManager = camForm.variableManager;

camForm.on('form-loaded', function() {
  // fetch the variable 'orderline'
  variableManager.fetchVariable('orderline');
});

camForm.on('variables-fetched', function() {
    <!--// this callback is executed *after* the variables have been fetched from the server-->
    $scope.orderline = variableManager.variableValue('orderline');
});
</script> 
</form>

Howerver when the task loads the form I get a message “Loading” and nothing is shown.
It seems like the variable “orderline” is not fetched.
Any ideas why?

Thanks!


#10

@kontrag not sure. I would recommend that you step through your code and iteratively add each line and test to see where it specifically breaks, and once you figure out where it breaks, then attempt to print out raw values without the camForm functions, to see what sort of data you are getting.


#11

Hey all.

Just wanted to share some internal code i have used in the past to work around concurrency update needs for creating the aggregate variable such as that used in the DMN looping

It basically provides a in-mem concurrentHashMap and a static helper to perform common actions.

See the docs for example. Note that it would still be up to you to control how to handle failures, rollbacks, etc and clearing out the map. See the unit test and in-line code for further details.

The code examples are in Java Delegates, but of course you can modify them to be used with groovy, nashorn, etc.

enjoy


#12

Hi,

I have a same scenario with BPMN. I am facing an issue with ElementVariable in the multiinstance sendTask. I am passing this as an input variable to the task. But getting the below error :

Caused by: org.camunda.bpm.engine.impl.javax.el.PropertyNotFoundException: Cannot resolve identifier ‘nameElement’

What could be the possible cause?

Regards,
Hetal


#13

Post osme pictures of your configuration from the modeler


#14

Hey Stephen,

Thanks for the quick response! Below are the screenshots. Attaching the bpmn as well.


Input/output mapping. It’s a send task.
iomapping

Thanks.agile_connect_network.bpmn (44.9 KB)


#15

So few things to explain:

  1. YOu have ${elementName} as a input variable and as the variable name for “Element Variable”. Can you explain your usage here? “Element Variable” is the name of the variable that camunda will create inside each instance.

  2. You have Loop Cardinality and Collection. Usually these are used as one or the other. Can you explain your usage?

  3. Your collection: ${name}, this is a array of “names” ?

Where is the specific error line number? There would be a stack trace in the logs with more info about specifically where the error occurred.


#16

So,

Name is defined as a script task

def names=[“Camunda System 1”, “Camunda System 3”]
execution.setVariable(“name”, name)

  1. What I understood is, ${elementName} is a variable which camunda creates in each instance based on the variable names like in the example posted above for collName . Which I want to use as an input for that instance.
    Is it like it is not able to find the variable at the start of the Instance (because the variable is created after the instance starts)?

  2. Yes, my collection object can have more values. But looping has to be on the Loop Cardinality. Isnt that possible?

  3. Yes.

The complete stack trace :
2018-12-27 21:04:13.734 ERROR 10892 — [ntainer#0-0-C-1] org.camunda.bpm.engine.context : ENGINE-16006 BPMN Stack Trace:
Task_createL3VPNInstance (activity-start, ConcurrentExecution[dd10377e-09ec-11e9-a8a2-507b9dfa65b0], pa=catalogue-processes)
Task_createL3VPNInstance, name=createL3VPNInstance
^
|
Task_createL3VPNInstance#multiInstanceBody
^
|
Task_0722qey, name=dummy_1
^
|
Task_1nxjlhw, name=groovy-task

2018-12-27 21:04:13.743 ERROR 10892 — [ntainer#0-0-C-1] org.camunda.bpm.engine.context : ENGINE-16004 Exception while closing command context: Unknown property used in expression: ${nameElement}. Cause: Cannot resolve identifier ‘nameElement’

org.camunda.bpm.engine.ProcessEngineException: Unknown property used in expression: ${nameElement}. Cause: Cannot resolve identifier ‘nameElement’
at org.camunda.bpm.engine.impl.el.JuelExpression.getValue(JuelExpression.java:62) ~[camunda-engine-7.10.0.jar:7.10.0]
at org.camunda.bpm.engine.impl.el.JuelExpression.getValue(JuelExpression.java:50) ~[camunda-engine-7.10.0.jar:7.10.0]
at org.camunda.bpm.engine.impl.el.ElValueProvider.getValue(ElValueProvider.java:39) ~[camunda-engine-7.10.0.jar:7.10.0]

Thanks.


#17

Hi @StephenOTT.

I 've been reading your example here and I would like to ask you the following:

In my case where I don’t have a Business Rule Task, what must I define as the output of each instance in order to be grabbed by the End Execution Listener ?

My issue here is that my result variable is unique (my json variable) and every output of my Multi-Instance subprocess is written there.
So, this json value is overwritten after the completion of each instance :face_with_raised_eyebrow:

Thanks,
Steve


#18

@StephenOTT, I think that I found the way but I have to test it in a new deployment to be sure.
As an output of each instance, I’ve defined my json variable (“selectedProduct” in my case) and instead of the “combinedResult” variable of your example, I’ve defined the array of assignees for my Multi-Instance subprocess) which is created in the End Execution Listener of the 1st task (like as your Script task).

I will inform you in any case,
Steve


#19

Hi @StephenOTT and to anyone here.

I ran again my process with my changes (mentioned above) but when I try to complete the first instance of User Task in Subprocess (“Select Products from the List”), the server displays the following error :
SPIN/JACKSON-JSON-01005 Unable to create node for object of type 'Undefined’

This is the relative server log file for anyone interested :
catalina.2019-01-19.log (50.0 KB)

What I did here, was to use an End Execution Listener in the Multi-Instance Task of my Subprocess (“Select Products from the List”) in order to grab the values of my created json variable (selectedProduct) on the completion of each instance. I must mention here, that this json variable is created (via camForm function) on the completion of each instance. I don’t know if this fact plays a role for my server error. :thinking:

A screenshot with the script from my Multi-Instance Task in Subprocess :

And from my process :

What I’ve been trying to achieve here, is to append the values to this json variable (selectedProduct) on the completion of each instance.

Can anyone help me please ?

Thanks in advance,
Steve


#20

For anyones future reference: here is another way to do the looping but with parallel execution: https://github.com/StephenOTT/camunda-concurrency-helpers-process-engine-plugin

You can use this helper to have the concurrent map update from each of your executions and then update your process variables at the end with the final result.


#21

Hi @StephenOTT.

I’ve been using Maven. Could you please tell me where (into the pom.xml file) must I write your suggested snippet (for installing the plugin as a dependency) and as a consequence the relative dependency ?

My pom.xml file :
pom.xml (1.7 KB)

As far as the version is concerned, which number must I write and where can I find it ?

Sorry for my many questions here but I’m not so experienced in some aspects. :face_with_raised_eyebrow:

Thank you very much,
Steve