Workflow Engines and ERP: current state of the discussion

As said I am quite new to BPM, therefore I would like to know the current state of the question if a workflow engine should be placed on top of an ERP, or better inside.
If it would be placed inside, then there would potentially be two workflow systems, one inside the ERP and one general WF-system like the Camunda Process Engine that handles processes that the ERP can not handle alone.

I found a very interesting article about that, though it is a bit old:
ERP AND WORKFLOW SYSTEMS

Also, this article is cited a lot:
Workflow Management Systems and ERP Systems: Differences, Commonalities, and Applications
https://pdfs.semanticscholar.org/9319/3c4251c9884d41756c536ff08b4a9f83ef65.pdf?_ga=1.102994496.1377418632.1477493996

My very own opinion is that the separation between ERP and Workflow Engines is an artificial one, and only due to the fact that ERP systems just came up earlier at a time where nobody thought about process centric systems.

Integrating both as separate systems will for sure result in many big projects.
But would it not be a better view to not see both systems as separate, but right away create a holistic approach, that is both data-centric and process-centric?
Currently Microservices are discussed much, but I doubt they can create a stable complete system, as they even add complexity.

Any comments are most welcome. :slight_smile:

After reading the second article above in more detail and thinking about it, my strong impression is that the authors compare apples with peaches:
ERP systems consist of ready-to-use business applications, while WfMS (workflow management systems) are tools to build business applications.
So they compare a house with a crane.
It is absolutely not possible to replace the business software of any company, which consists nowadays of some kind of ERP system, with a WfMS. So it just makes no sense to confront both, implying they could be alternatives.
What needs to be compared for sure is the data-centric approach and the process-centric approach, but again both are not alternatives, but handle the same business objects and their data and real world processes.
So the main question is how to best integrate data and processes, not to think of both as replacements for each other.

1 Like

@bernd.korthaus we deal with these types of questions a lot with governments. I like to explain the relationship as the ‚ÄúData source‚ÄĚ and the orchestrator of process.

In my view: the ERP is the data source. It is the place where you store information and all sorts of activities are actioned. The WfMS is the orchestrator that is interacting with the ERP and very likely many other systems. The ERP is likely in place for many many years, and the WfMS should be something that can be switched out for other tools as they become available, or even have many Workflow tools at the same time.

Really comes down to ‚Äúmessages‚ÄĚ and web services. The ERP generates states and messages for the various data, and the workflow system interacts and manipulates those states and the ERPs data based on messages received or data pulled from the ERP through web services.

so essentially: The ERP generates messages/signals at various times, and the WfMS listens for various activities and executes processes based on those messages/signals or are activated manually by users. The WfMS interacts with the ERP through web services. (If not web services, then some sort of SDK/API - but lets assume WS for simplification purposes).

@bernd.korthaus

Here is a example, while not a ERP, this is a hyper-generic case object that stands outside of Camunda and has no association with CMMN. It is i believe the same concepts you talk about.

The key point is the ‚ÄúAssociated Process Instances‚ÄĚ box. Any work that is done by a WfMS is associated in the WfMS (and sometimes/rarely on the case/system side). So when w WfMS (camunda or otherwise) interacts with the system (ERP or otherwise), the system is the source of record/data, and the WfMs is the source of orchestration - the steps that are followed. But anytime there is information that should be stored in the long term, it is placed within the system/ERP.

@StephenOTT
I like the generic case object, though I am more familiar with ‚Äúsimple‚ÄĚ data objects like projects, invoices, business_partners and persons. Projects for example could be associated with the process ‚Äúproject handling‚ÄĚ, that has itself a lot of sub-processes like ‚Äúproposal handling‚ÄĚ or ‚Äúinvoicing‚ÄĚ, which again deal with other data objects. In those cases there is often a 1:1-relationship between process (= invoicing) and data object (= Invoice).
As an example for your generic case object I could imagine an insured event (‚ÄúVersicherungsfall‚ÄĚ), like a car accident that will generate such a case with a lot of processes and related data. But again the case itself can be seen as a data object that stores the data and holds all relations.
I understand then a WfMS as a working horse that orchestrates, as you said, or as an intelligent master control station (‚ÄúLeitstand‚ÄĚ). And it makes much sense to keep this component flexible with BPMN diagrams, and not hard-code it.
And I guess that all associated process instances and their related tasks will again be stored beneath the case object, which makes everything well documented and traceable.

You also get a fairly good separation of complexity, as the WfMS like camunda is an independent engine that is well-understood and can keep the process views a bit parallel to the data objects, so that everything is not a complete over-complex information mess.

Same idea of a case. Invoices are just a form of record that gets stored. How that invoice gets generated, reminders, workflow, business rules etc, that execute around the invoice can be orchestrated by one or many workflows working together or in complete separation.

The I like to really break down the objects into what is the container and what is the input/output. The container may be the request to generate a invoice. The input is the various data about what should go into the generated invoice, and the output is the actual invoice record/data object. I have found that this approach provides flexibility around how the business wants to operate and not have to conform to how the system has designed it data objects.

A project is very much like a case. A project has all sorts of processes that could be activated around that project in sequential, parallel, and ad hoc ways. The project itself is really just a collection of associations and raw data. How the data got done may be part of the WfMS, another system or other code in the ERP. So I would tend not to create a Project handling process. I would create many smaller processes that can be activated around the Project object. (this is the similar use case for using CMMN).

In the camunda case, and the simple case example I gave you, the tasks and process instances solely live in Camunda. Anything that is generated by Camunda against the case is logged as attributes of the data object as a SourceSystem and the IDs relating back to the process instance.

Something to note: You may of noticed there is the concept of KPIs at the case level. While there can/is KPIs in the process, we found it better to allow a third-party source to capture KPI markers because KPIs could be generated by camunda, but also by many other systems and actors that are all part of the same case instance.

Yes, but the case is part of a large system. So that large system has Users, business units, permissions, records, reports, related objects like invoices, and notifications. The case just represents a form of container to store a timeline and data and actions that occurred.

You could see a project handling process as a high-level wrapper process that has itself a lot of sub-processes with finer granularity. Because there is a start event that has to create a new project, and an end event when the project finally has the status ‚Äúclosed/locked‚ÄĚ and may not be changed or booked any more. Between both, the project handling process mostly reacts to events, like ‚ÄúCustomer asks for a changed proposal‚ÄĚ, which will initiate the sub-process ‚ÄúProposal handling‚ÄĚ.

@bernd.korthaus Yes! we see that thinking quite often in the government service request space. You make a request for something and eventually it is resolved.

BUT :wink:

The ‚Äústatus‚ÄĚ that you mentioned really comes down to a sort of State Machine, and for many cases I tend to argue that the state machine should be on the ‚ÄėERP‚Äô/system side rather then being controlled in the process. if you have a high level wrapper process, then that process instance must exist throughout the entire lifecycle of the project. The creation of a project can be a small process in it self: ‚ÄúCreate New Project Process‚ÄĚ, and closing a project out may also be a small process. And between the Open/Create and the Close, there can be one or more processes that are run (including sub-processes). This pattern supports non-linear business processes that have knowledge workers manage the overall activity of the project. Same sort of reasons to use CMMN as your overall container for a project.

What benefit does the high level wrapper process give you? Have you looked at what CMMN provides?

We chose not to use CMMN because 1. it was fairly new, but 2. because it required a fair amount of definition around the steps and flow (2.5. many people still find it hard to understand the CMMN layer on top of BPMN). This work well is many cases, but our goal was high levels of flexibility to map varied business processes to these ‚Äėgeneric containers‚Äô.

in our case example, the state machine / your high-level wrapper of what defines a State, middle and Close/end of a case/project is defined at the system/ERP level. The business processes add control to who does what, how data is manipulated, and how automation is enabled.

I would suggest that this is not a Sub-Process. This is a independent process that will interact with the ‚ÄúCase‚ÄĚ/Project, and manipulate data within the project. based on the outcome of that Change request, other processes may be started.

TLDR: I suggest that a common pattern is that a process, does not represent the lifecycle of a project. The lifecycle/states of a project is managed by the ERP. The WfMS manages the user tasks and automation tasks that are within the lifecycle, and change the state of a project through the various business processes. And Yes is many simple scenarios, a single business process which represents both the lifecycle of the project and all of the activities occurring during that lifecycle is also a common pattern. :slight_smile:,

@bernd.korthaus through this together quickly to help to visualize the concept:

  • Users may interact with a project directly, through a business process, or another system.
  • The other systems, processes may interact with the user directly.
  • The processes may interact with other systems, users, and the project.
  • we make the assumption that is any of the processes, it is possible that it is a straight data submit to the project, or there could be N number of steps, manual and automated that occur before the data ends up in the Project.
  • The project in this scenario is like the ‚Äúpersistent‚ÄĚ record that contains the state. Processes are used to do things like Create a new project, or start the Project closeout process. (Again see CMMN as depending on your structure needs, it may be of lots of help). And there are all sorts of other independent processes that run that are not part of the project lifecycle, but still interact with that project. Hence that wrapper process may be un-needed complexity. and using this helps to ensure that we do not have processes that are running for excessive amounts of time, where the process is active solely to store the state of the project, just seems to be a costly burden for complex scenarios :smiley:

@StephenOTT Thanks for the insights!

A simple explanation is that:
1/ ERPs are systems of record - they organize and integrate data.
2/ Process engines are systems of engagement - they orchestrate activities and pro-actively inform and engage process players

In time, ERP systems have been enriched to include workflow fragments relevant to ERP modules. However, since the ERP architecture relies on a shared database (so a strong coupling concept), adding workflow fragments does not lead to a sustainably open architecture in terms of distributed transactions, microservices etc.

A sustainable approach is to fix operational processes first in your workflow engine, and then implement a minimal ERP, since the ERP is just a customer of operational processes.

1 Like

Like that word in this context. Seems to fit well.

1 Like

How would a CRM fit in all this? As a CRM is kind of a second datastorage? Could a workflow engine be seen as the link in between all these systems?

I’ll play heretic here.

Everything we do in programming comes down to this basic process:

  1. A ‚Äústate‚ÄĚ exists which contains objects with certain attributes
  2. A change (event, movement of time, alteration of attribute, etc.) is introduced
  3. The state is evaluated against a set of rules
  4. The results drive an action

I don’t have a computer science degree, so those of you with more formalized backgrounds and more experience (which is most of you), can point and laugh at this. However, I look at this as an abstract starting point when thinking about questions like yours. That is because the question is really one of how you encode logic in the most economical way (i.e. what delivers the greatest value for the resources used to produce it).

In your case, just because it‚Äôs ERP doesn‚Äôt mean it can‚Äôt be integrated with BPM systems like Camunda. I would simply ask, which technologies offer the best way to encode the logic I need to execute. Say ERP systems are only ‚Äúthis‚ÄĚ or ‚Äúthat‚ÄĚ really limits you. It may be that your ERP system offers tooling that easily emulates what a BPM system would provide for the purposes you require. It might not be a generalized solution for all use cases, but I always try to leverage technology across as many solutions as possible.

For example, you can build fairly impressive reporting tools in Excel. To be sure, there are tools out there which out of the box offer more focused functionality. However, the cost of acquiring and/or learning these tools might not be attractive because investing in Excel tools, building up a library of routines and experience, etc. can potentially be leveraged across a larger set of use cases than a dedicated report tool.

BPM is just logic, expressed in an economical form. Camunda and others have provided tools that make the creation and execution of logic easier, with an emphasis on business processes. You can do everything Camunda is C code if you want, there’s nothing stopping you. Or, for that matter, you can leverage your ERP system’s tools to automate processes.

One thing about Camunda is that it may be considered a ‚Äúdevelopment‚ÄĚ environment (ironically, that‚Äôs also one of its greatest weaknesses for the general user). So there‚Äôs nothing stopping you from using it to replace your ERP. There are few limits to it use, it‚Äôs all a matter of the best way to encode, execute, and manage the logic particular to your situation.

There’s also interesting information about types of erp architecture - monolithic and postmodern.