Shell Script Invoked by Camunda


#1

Hi Community,

My goal is to run a shell script via Camunda service task.
What I have managed to do is to use service task to connect to server via Jsch and run a prepared script on server, which basically works as supposed but I have trouble with optimistic locking as the task is running for a long time (could be hours) I have tried to adjust bpm-platform.xml “lockTimeInMillis” and “failedJobRetryTimeCycle” to 0 but unfortunately I am not getting results I would like to see.
I know that there is a different approach to use external task which at the end might be the right choice.
But I want to ask you experienced community how would you solve the task I am trying to get thru?

Thanks.

Michal


#2

was a similar discussion recently:

Whats the aversion to using external tasks?


#3

@Michal_S, if you just need to get it going without the external task pattern, a workaround you can throw together is running your shell command in the background, and then wait for a message back to the engine. So as part of your shell you provide some sort of unique ID, which you can then message back to camunda through CURL to the /message endpoint.
I am not advocating this as the “solution”. Just giving you a option to get something working until you have a task worker in place.

something like:
shell


#4

I have done that, I mean my process have implemented message Intermediate Catch Event which listen to curl call at the end of shell script,
But the biggest hurdle is optimistic locking, basically Camunda triggers the service task twice and after a time again.
Perhaps it is my poor adjustment of bpm-platform.xml and process.xml where I did put <property name="jobExecutorActivate">true</property>

Thanks for the quick answer Stephen you gave above.


#5

Can you explain this further? Provide a sample of a BPMN image?


#6

This basically works as supposed, Intermediate Catch Event receives a message once Service Task’s shell script reach the end of script, unfortunately Camunda in my implementation triggers multiple instances of Service Task’s shell script. example


#7

Can you explain this further? You are running a Java Delegate in “Service Task” ? and in that java code you are executing multiple instances of your shell script?

Edit:
Made some assumptions, but what about something like this (this also assumes you are running a shell command outside of the executors limits - not tested. Thinking says probably not…):


You will have to generate message names (or correlation keys/variables) that are uniuqe for each instance of each run of script 1 and script 2)


#8

I am using mysql database , the below is output of ACT_HI_JOB_LOG for the service task (which has name UserTask_076y3m5)

ID_, TIMESTAMP_, JOB_ID_, JOB_DUEDATE_, JOB_RETRIES_, JOB_PRIORITY_, JOB_EXCEPTION_MSG_, JOB_EXCEPTION_STACK_ID_, JOB_STATE_, JOB_DEF_ID_, JOB_DEF_TYPE_, JOB_DEF_CONFIGURATION_, ACT_ID_, EXECUTION_ID_, PROCESS_INSTANCE_ID_, PROCESS_DEF_ID_, PROCESS_DEF_KEY_, DEPLOYMENT_ID_, SEQUENCE_COUNTER_, TENANT_ID_

‘c32f4fbd-e425-11e7-9a0d-0242ac140003’, ‘2017-12-18 19:00:49’, ‘77e3f381-e425-11e7-9a0d-0242ac140003’, NULL, ‘3’, ‘0’, NULL, NULL, ‘2’, ‘44311700-e425-11e7-9a0d-0242ac140003’, ‘async-continuation’, ‘async-before’, ‘UserTask_076y3m5’, ‘77e37e50-e425-11e7-9a0d-0242ac140003’, ‘61eaf364-e425-11e7-9a0d-0242ac140003’, ‘XXX:1:4430efd8-e425-11e7-9a0d-0242ac140003’, ‘XXX’, ‘441662f5-e425-11e7-9a0d-0242ac140003’, ‘3’, NULL
‘77e41a92-e425-11e7-9a0d-0242ac140003’, ‘2017-12-18 18:58:43’, ‘77e3f381-e425-11e7-9a0d-0242ac140003’, NULL, ‘3’, ‘0’, NULL, NULL, ‘0’, ‘44311700-e425-11e7-9a0d-0242ac140003’, ‘async-continuation’, ‘async-before’, ‘UserTask_076y3m5’, ‘77e37e50-e425-11e7-9a0d-0242ac140003’, ‘61eaf364-e425-11e7-9a0d-0242ac140003’, ‘XXX:1:4430efd8-e425-11e7-9a0d-0242ac140003’, ‘XXX’, ‘441662f5-e425-11e7-9a0d-0242ac140003’, ‘1’, NULL


#9

@Michal_S see the above edit.


#10

Correct, my service task is Java Class which implements JavaDelegate.
Thanks for the assumption, unfortunately I think it is not exactly what I am trying to achieve
My process has a businessKey which is enough unique for a correlation of REST message as I expect to have only one running instance of a shell script per running Camunda Bpmn Instance, more then a one means failure.

Could you please explain a little bit more what executors limits are?
“this also assumes you are running a shell command outside of the executors limits - not tested. Thinking says probably not…):”

Thank you very much for your help! It is very appreciated :slight_smile:


#11

See the details here where Thorben goes through the executor: Shell Script Invoked by Camunda


#12

@Michal_S here is a script for you to test.

I have not tested against the timeouts of the executor to see what is actually is occurring, but from a cockpit perspective is looks like its running outside of the executor:

So look at this:

shell_test

I run the following javascript:

with (new JavaImporter(org.apache.commons.exec)) {

  var myString = 'echo "hello Steve!" && sleep 15s && curl --request POST --url http://localhost:8080/engine-rest/message --header "Accept: application/json" --header "Content-Type: application/json" --data \'{"messageName":"myMessage"}\' && echo "hello Stephen!"'

  var shellCommand = new CommandLine("sh").addArgument("-c")
  shellCommand.addArgument(myString, false)

  var resultHandler = new DefaultExecuteResultHandler()
  var watchdog = new ExecuteWatchdog(5 * 60000)
  var executor = new DefaultExecutor()
  executor.setExitValue(1)
  executor.setWatchdog(watchdog)
  executor.execute(shellCommand, resultHandler)
}

Output would be something like this:

camunda_1  | hello Steve!

… 15 seconds later…

camunda_1  |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
camunda_1  |                                  Dload  Upload   Total   Spent    Left  Speed
100    27    0     0  100    27      0    393 --:--:-- --:--:-- --:--:--   397
camunda_1  | hello Stephen!

In this scenario above, curl output and the hello Stephen! echo is from the script that executed in the “Run Shell” task, but it occurred while the engine was at the “Get Background” task.

What appears to happen is the background job is created outside of the Camunda Executor.
and then we run the Localhost:8080 curl to message back to the engine.
I also installed curl on the camunda server to run the command in the sh script.

Would be interested to hear from @camunda / @thorben about some likely issues with doing this ;).

Note:
The script was based on: https://stackoverflow.com/a/41159427


Wait till a shell script is completed
#13

@StephenOTT
indeed, this looks interesting! the sad part is that I am not JavaScripter but definitely I have got homework to look at it :smile: and perhaps to learn some basics during holidays, could you please share your BPMN file?

Thanks

Michal


#14

My code is just java code but written as JS. You can just change the “var” into their proper types and remove the first line and last line and you will have java. See the stack overflow link in my previous post.


#15

I will not try to amplify Stephen’s excellent and comprehensive answers, I grovel before his knowledge. You might consider learning and using Groovy script as I’ve found it offers a wide range of functionality and “feels” similar to Java. That said, I have done the vast majority of my work in shell scripts in the past, so I don’t blame you for continuing to use them.


#16

Hello Stephen,

Could you please explain how you made the script run in background ? was it set as an asynchronous task?


#17

@Rajiv_Cj see the explanation from:

and note the last line that shows where my logic came from: https://stackoverflow.com/questions/22844268/execute-shell-command-with-org-apache-commons-exec-defaultexecutor/41159427#41159427

The snippet of Javascript is basically executing apache commons exec creating a command line argument. We then use the Execute Watchdog feature of the exec lib to run the process as a background process/async process. https://commons.apache.org/proper/commons-exec/apidocs/index.html

This has does not have anything to do with Camunda’s “async” feature.


#18

just to people whom it may help, using jsch

with (new JavaImporter(com.jcraft.jsch, java.util)) {

var jsch = new JSch();

var  config = new java.util.Properties(); 
config.put("StrictHostKeyChecking", "no");

var session = jsch.getSession('name','address',22);

session.setConfig(config);
session.setTimeout(20000);
session.setPassword('pass');
session.connect();

var channel = session.openChannel('exec');

channel.setCommand("cd /home/yyy/xxx && ./test.sh");

channel.connect();

channel.disconnect();
session.disconnect();


}

#19

@Michal_S can you code quote your code snippet?


#20

@Michal_S, your example would be blocking, correct? Meaning that the duration of your execution on the remote session would take up the job executor. Has your requirement changed from your original post?