Storing all the history tables data into external data store

I would like to store all the history tables data in external data stores like Cassandra/mongodb/elastic search.

Do we have have any github example or approach for reference how to stream all the history tables data into external store?

Main idea is to query historic process instances, historic task instances, and historic variable instances for identifying the state of the process instance and task instance by filtering with process/task variables?

I don’t think you would be able to store history table records in these DB’s
Check the below URL
https://docs.camunda.org/manual/7.11/introduction/supported-environments/

Supported Database Products

  • MySQL 5.6 / 5.7
  • MariaDB 10.0 / 10.2 / 10.3
  • Oracle 10g / 11g / 12c / 18c
  • IBM DB2 9.7 /10.1 / 10.5 / 11.1 (excluding IBM z/OS for all versions)
  • PostgreSQL 9.1 / 9.3 / 9.4 / 9.6 / 10.4 / 10.7 / 11.1 / 11.2
  • Amazon Aurora PostgreSQL compatible with PostgreSQL 9.6 / 10.4 / 10.7
  • Microsoft SQL Server 2008 R2/2012/2014/2016/2017 (see Configuration Note)
  • H2 1.4 (not recommended for Cluster Mode - see Deployment Note)

If you want to write in your some custom DB , then probably you have to hack some way ? History Event Stream and write in your preferred DB .

@anisk this doc page i know. My use case is related to achieving performance. For storing camunda states we use RDBMS and im trying to use NOSQL store for history data. Still history data will also be persisted in RDBMS as backup purpose.

My query is related to History event streams to store all history data in NoSQL store.

Do you have github example for this scenario?

Hi @aravindhrs,

there is a quite old community extension, that may serve as a template to create something for the current version of Camunda BPM: https://github.com/camunda/camunda-bpm-elasticsearch.

Hope this helps, Ingo

@aravindhrs Were you able to achieve this?

@Mass_Shake this extension is not actively maintained and I didn’t used it, because we are not interested in the history data.

@Mass_Shake were you able to implement this functionality?

@Ingo_Richtsmeier for history data, is it possible to use multiple data stores? For example RDBMS and ElasticSearch together.

Example:
Writing to RDBMS and set TTL but for real time reporting to use ElasticSearch.