Script Based unit testing with Coverage Reporting (Spock Framework)


#1

Hey all

Wanted to share a work in progress:

This is a implementation of Spock Framework as discussed on:

But we are now taking it further to add BPMN Coverage testing similar to:

But the coverage library i have found to generally be complicated to extend and was very confusing on how it worked. The goal here was to open coverage testing to “everyone” / even the less technical bpmn devs.

So we end up with

some differences between the current BPMN coverage lib:

  1. It works with the default engine; no custom bean configs required
  2. it uses all of the built in camunda engine services to generate the required data; there is no custom data models being stored.
  3. it uses all of the in-house spock framework features and the spock-reports extension, providing really easy extendability.

You can follow some of the dev discussions here:

Or feel free to ask questions or get involved.


Unit Testing: How to get Call Activity's called process instance from runtime?
#2

Update with more visual support:

BPMN xml, activity instances, sequence flow instances, User Tasks, and Async Configs for all elements.


#3

Okay updated with Multi-Feature support

So this is one Spec with multiple Features


#4

Updated with Mult-Feature and Multi-Iterations Per Feature using Spock Data Tables.

Notice how there are multiple Coverage Maps in the second Feature. This is a Coverage Map for Each Iteration :+1:

@nikku If you have any suggestions on best ways to optimize loading of many BPMN viewers on a single page that would be appreciated.


click on image to see full length

Notice how Iteration Coverage have Iteration Counts that align with the Iteration Counts in the Data Table.


#5

Here is a update image of Multi-Iterations with different coverage outcomes per iteration:


click image to see full length, and scroll to bottom of image


#6

New update for further refinements and better theme


click on image to see full length


#7

another usage of bootswatch


#8

Some more UI cleanups for easier use:

  1. Uses BPMN Navigated Viewer allowing scrolling and zoom on each of the BPMN diagrams (Thanks @nikku)

  2. Supports Spocks @Unroll feature allowing Data tables to be run as independent feature runs.

  3. More clear Pass Fail

  4. Bootstrap cleanup

  5. Narrative cleanup

  6. Fully Responsive support so it can be easily shared.

  7. All dependencies are loaded through CDN (BPMN-JS, bootswatch, logos, etc) The goal is that it is 1 file you can easily share with whomever and it will open on their browser.

  8. Support for duplicate Data table usage (every BPMN diagram is a unique ID)

Enjoy


click on image to view full length

@thorben thanks for the visuals feedback :wink:


#9

Okay some good updates today on this:

Coverage has not been added as a dynamic action that can be added that any point in a test:

So at anypoint you can do:

reportInfo(
   generateCoverageData(
                        processEngine(), 
                        callActivityCoverage1ProcessInstance, 
                        "CallActivityCoverage1.bpmn Completion"
                        )
)

The three inputs are the processEngine instance to run the query against (has interesting implications for Multi-Engine testing), the specific ProcessInstance to get the coverage for, and the human name for the coverage diagram.

You Can run this at any time and multiple times / as you see fit for your purposes.

You get output such as:

Single Feature usage:

Multi-feature usage:

@felix-mueller