Https baseUrl with camunda-external-task-client-js does not work

When I attempt to use a https baseUrl with Client object like this:

const client = new Client({
    baseUrl: process.env.CAMUNDA_BACKEND_URL,
    use: logger.bind(null, log),
    asyncResponseTimeout: 30000,
    workerId: os.hostname(),
    maxTasks: 3,  //max number of tasks to fetch in one poll
    maxParallelExecutions: 3,  //max number of outstanding (incomplete) async tasks
})

then a subsequent call via the Client object fails at this line:

https://github.com/camunda/camunda-external-task-client-js/blob/master/lib/__internal/EngineService.js#L50

with a message like this

RequestError: connect ECONNREFUSED 127.0.0.1:443`

The camunda library uses a library called ‘got’. According to this problem report:

the probable reason is that a third party library, https-proxy-agent, patches the http.request function in a manner that is not compatible with node 10 (or got). https-proxy-agent seems to be fairly ubiquitous in the node eco-system, being used by sentry and datadog and auth0 libraries, for example.

As far as I can tell, my only option at this point is to stop using the camunda JavaScript client library and start using axios directly.

Is this everyone else’s experience or is there a supported way to use camunda-external-task-client-js with https URLs?

I did a little bit more investigation. I stripped out the libraries that dragged in https-proxy-agent and the problem went away.

I’ve decided that the path I will take is to fork camunda-external-task-client-js and replace got with a got replacement shim implemented with axios. I can make this branch available upon request once it is complete.

Thanks for filling us in on this @Jon_Seymour
Did you create a feature request or bug report for this on the external task js repo?

No, I have not. Is the way that I do that to create an issue on the GitHub repo? I guess the problem is that strictly speaking it is not a bug in the camunda code itself - it is just that it falls victim to malpractice by another widely used package (https-proxy-agent) if they are deployed in the same node process.

I think the perfect solution would be for https-proxy-agent to be altered to avoid the malpractice. However, if the fix requires an interface change it may take a while for it to ripple up through all the packages that use https-proxy-agent as a dependency.

FWIW: Creating the shim is looking like more work than I bargained for. I may end up adapting the one of the workarounds for the got issue in my workflow client process. It will be a point solution for me, I think, rather than a generally useful solution for everyone else.

I think it’s good to document the issue, because even if it does mean a change in dependencies on our side it would be interesting to explore the possibilities.

You can post bug reports or features requests (this may be the latter) with the information you’ve shared here at our JIRA system. This would help in getting the problem in front of the relevant camunda team member.

For reference here is a monkey patch of https.request that I used to undo the consequences of the monkey patch performed by https-proxy-agent. It works by turning all 3-argument calls to https.request into equivalent 2-argument calls so that all the arguments make it to the original https.request implementation semantically intact. Note that I only copied the properties I needed myself. If you need to propagate other URL properties into the options object, you will need to add the appropriate assignment statement.

   const https = require("https")

   ...

   https.request = function(request) {

    //
    // This function overrides the https.request function (from Node 10+) so that all 3-argument http.request
    // calls are converted into equivalent 2-argument request calls prior to passing those calls onto the original
    // handler. This is required to undo the effects of a monkey patch performed by https-proxy-agent which
    // inteferes with a completely different library, got, that expects to be able to use the
    // 3-argument version of https.request
    //
    // The ultimately correct fix for this issue is for httos-proxy-agent to change and then for all
    // dependent node packages to upgrade their dependencies to depend on the fixed https-proxy-agent
    // package but for whatever reason this has not yet been done.
    //
    // For more information about this issue refer to:
    //
    // - https://forum.camunda.org/t/https-baseurl-with-camunda-external-task-client-js-does-not-work/26362
    // - https://jira.camunda.com/browse/CAM-13347
    //

    log.info('installing https-proxy-agent monkey patch compensation...')

    return function(...args) {
        if (args.length == 3) {
          const url = args[0]
          const options = args[1]
          const callback = args[2]

           // URL {
           //    href: 'https://camunda.acme.net/engine-rest/external-task/fetchAndLock',
           //    origin: 'https://camunda.acme.net',
           //    protocol: 'https:',
           //    username: '',
           //    password: '',
           //    host: 'camunda.acme.net',
           //    hostname: 'camunda.acme.net',
           //    port: '',
           //    pathname: '/engine-rest/external-task/fetchAndLock',
           //    search: '',
           //    searchParams: URLSearchParams {},
           //    hash: ''
           //  },

          options.protocol = url.protocol;
          options.hostname = url.hostname ? url.hostname : url.host;
          options.auth = url.username ? url.username+":"+url.password : "";
          options.port = url.port;
          options.path = url.search ? url.pathname+"?"+url.search : url.pathname;

          return request.call(https, options, callback);
        } else {
          return request.call(https, ...args);
        }
    }
  }(https.request);

Note: the code above has been corrected a previous version was in error.

I have raised this issue in JIRA - https://jira.camunda.com/browse/CAM-13347. I will try to create a test-case and a fix for https-proxy-agent itself.

1 Like