Support for Azure SQL

Process Engine
#1

Hello, I am looking at using Camunda engine embedded within my services. We are in Azure and using MS SQL database. I notice the latest 7.11 version of Camunda supports up to Microsoft SQL Server 2017. From what I can gather from Azure SQL the major version is 12 which is MS SQL Server 2014.

Does the Camunda team have any advice about running Camunda over Azure SQL?

Many thanks,
Roddy

#2

Hi,

Is it possible to provide an update on this question?

We are investigating possibilities for moving to azure and have to chose a DBMS for that environment. Is postgresql the only option or is azure sql also supported?
I know azure sql is not listed at https://docs.camunda.org/manual/7.12/introduction/supported-environments/ but since it is a major choice for my application architecture I’d like to feel more confident that it is really not supported from camunda perspective.

Could you please confirm whether or not azure sql is supported by camunda?

Kind regards,
Patrick

#3

Hi @Niall
I am also looking for an answer for the above query.
We are planning to migrate our BPM Processes to Camunda and soon we have plans to move all BPM application on Azure.
I have noticed that the database supported by Camunda does not have Azure DB in the list and was looking for an appropriate answer but still there is no luck.
Is there a way to integrate/replace H2 DB with Azure Database?

Can someone please suggest or share some light on this would be highly appreaciated.

#4

Hello @UtjalMajumder, @Patrick, @herrier

As you noted already, Camunda does not support Azure DB.

You can pick database of the supported and try to use it in Azure environment, however Camunda cannot guarantee normal execution of the process engine in that environment, as we haven’t executed any test suites against such setup.
One of the requirements that need to be met is the isolation level configuration.
Also, you can try to execute some of the engine test suites if you like (I can give you some guidance in case you need).

I am not certain how many users have try to do similar setup (and succeed). I would assume not having an community answer and confirmation to the thread means that it is not common at all.

Regarding the questions, how and when a new environment is added to the supported list. That decision is made based on interest among users and customers, testing and maintenance efforts for the team and resources to achieve it. As far as I am aware, support for Azure is not scheduled.

I hope the above gives you hints which direction to take, please let me know if you like to collaborate further on that.

Best,
Yana

2 Likes
#5

Thank you @Yana
It would be great if you can share the details on executing the engine test suites.

Cheers,
Utjal

#6

Hi @Yana,

Thanks for this trigger. We are actually using applications with emdedded camunda on azure with postgresql as provided on azure. This has been running without issues for 10 or 11 months. But as I understand from your response this is not a supported platform configuration. So maybe we’ve been kind of lucky so far.
I anticipated that azure sql would not be supported, but postgresql on azure would be. But after reading your contribution I understand that postgresql on azure is also not supported.

That is good to know now as we are getting ready to scale up significantly and move all our processes to camunda. However, if this setup is not supported we will have to stop this effort and have to look for another solution.

What would be your advice for applications running on PCF on azure, which is my company standard? What would be a supported database setup?

Maybe the question should be more general. What is the recommended setup to run in a cloud environment? Another thread in this forum says that this should not be a problem. But what is the recommended (and supported) persistence layer in this case? it seems to me that cloud providers come with their own database solutions and these are not supported.

I am really curious on your recommendations on this topic. I hope there is a supported way to run camunda on PCF on azure.

Moreover. I will be happy to execute the tests you mention but what will it bring me? I may be reassuring, but it will not make the platform configuration supported by Camunda.

Best regards,
Patrick

#7

@Yana, @Patrick
Absolutely true, Azure SQL is a family of SQL Server database engine products in the cloud and from the same company.
I was assuming since Camunda supports MS SQL it should support Azure SQL and if not then we need to search for other alternative best suited for the organizational roadmap.

#8

Hi all,

I’d like to clarify some things on this topic and Camunda’s support for databases in general.

Fundamentals:

  • Camunda BPM interacts with a database via JDBC, so a database’s behavior on that level is decisive if a Camunda version works with it or not
  • Behavior on JDBC level includes supported SQL syntax, implemented transaction isolation levels, locking behavior, consistency guarantees, etc.

Some definitions:

  • The database technology itself that implements a certain behavior on JDBC level (example: Postgres, Amazon Aurora Postgres, Oracle, MS SQL Server)
  • The environment that the database runs in (examples: operating system, Cloud provider, network, …)
  • A database is an actual appliance that runs somewhere (e.g. Postgres 10.4 on my Windows machine, MySQL hosted on Azure, Azure SQL), or simplified: database = database technology + environment

What does Camunda support and test?

What about database technology X in environment Y?

  • In order to make a statement regarding Camunda support, we need to understand if technology X is one of the technologies we already support, or if it is a distinct technology
  • Different databases may share the same or a similar name, but they can still be different technologies: For example, IBM DB2 z/OS behaves quite different to IBM DB2 on Linux, Unix, Windows. Amazon Aurora Postgres is different to a standard Postgres.

What does that mean for Azure SQL?

  • In the product team at Camunda we currently do not know how much Azure SQL differs from a standard SQL server. Therefore we cannot answer the question if they are distinct techologies and should be supported (and therefore tested) independently, or if we can claim that Azure SQL is covered by our support for SQL Server
  • We are aware of demand for Azure SQL and plan to look into this topic in the future, but cannot promise a date by when this will happen or by when we can claim support

I hope that helps.

Cheers,
Thorben

3 Likes
#9

Hi @thorben,

Thanks for your extensive reply. Could you also comment on support PostgreSQL on azure? How do you regard this?

Given that you specifically mention amazon aurora postgresql as being supported, it might very well be that postgresql on azure is not.

If that would be the case, could you then mention the supported databases on azure? Or is there no supported configuration to run camunda on azure?

Best regards,
Patrick

#10

Hi Patrick,

I think you’ll have to make some of these decisions yourself as I am not familiar with Postgres on Azure. Is it a standard Postgres being hosted on Azure? Then it would fall into the “Postgres” database technology and its fully supported. Is it conceptually very different or behavior-wise different to a standard Postgres? Then probably not. Maybe Azure documentation and support can help with answering these questions.

Cheers,
Thorben

#11

Hi @thorben,

What I found here and here is

Azure Database for PostgreSQL is a relational database service based on the open-source Postgres database engine.

So I would expect that it classifies as a standard postgresql database and would be supported as such, given the criteria you mentioned.

However, in the end I think it is your call whether or not it is considered to be supported. I would rather be confident before scaling up our camunda efforts, then unhappily surprised when we actually have to raise a support request at a later stage.

Could you confirm whether postgresql on azure is actually supported given the input above, or what else would be necessary to be able to provide that answer?

Best regards,
Patrick

#12

Hi @Patrick,

you can run the engine tests against a given database by yourself.

I’ve tried it with a simple postgresql in my Docker container, that contains the required tables already:

git clone https://github.com/camunda/camunda-bpm-platform.git
cd camunda-bpm-platform/engine
mvn clean test -Pdatabase,postgresql -Ddatabase.url=jdbc:postgresql:pgdb -Ddatabase.username=pguser -Ddatabase.password=pgpassword

The test run took about 01:44 h and ran 14637 tests.

If you provide the connection details from your Azure database, you can become confident, that the database technology will work with the Camunda engine (or not).

Hope this helps, Ingo

#13

Hi @Ingo_Richtsmeier,
Thanks for your advice. I will give it a try and let you know what the outcome is.
Best regards,
Patrick

#14

Hi @Yana,

Request you to please share the test suites that you have mentioned in your response, it would be really helpful.

Looking forward for your response.

Cheers,
Utjal

#15

Hi Utjal,

Here are some links that could be helpful:

Don’t forget to update accordingly your maven settings xml

2 Likes
#16

Thanks a lot @Yana

#17

Hi,

It is really cumbersome to run those tests, so a short heads up. From azure devops it seems I cannot quite control to only apply the engine module (also -f engine or -f ./engine/pom.xml does not work). So I run it from my local machine against a test database in our azure environment that has been opened for me to test from home. Running the tests in this setup takes well over 20 hours and has intermittent errors. When a test fails, just re-running the failed test is successful.

What I do notice is that the JdbcStatementTimeoutTest fails persistently in this setup.

The version provided on azure for postgresql is 11.6.

Any hint to run the tests in an azure devops pipeline are appreciated.

Cheers,
Patrick

#18

Hi @Patrick,

I assume it’s because of the high network latency between your laptop and the azure computing center.

I don’t know any details here, but would it be possible to deploy your own docker container to azure? Then you could try to setup a container with ubuntu, java, git and maven and run the statements from post #12 inside your docker container on azure. This should reduce the network latency.

Hope this helps, Ingo