What's coming up next in Obsrv

Following table captures the high level road-map planned for the next few releases of Obsrv.


Connectors Management APIs

Ability to add and manage a connector into Obsrv via APIs

Job Framework

Ability to create and drop in a stream (Flik) or batch job (Spark) into Obsrv

Job Management APIs

Ability to add and manage custom jobs in Obsrv via APIs

Obsrv Exporter

Export open telemetry compliant monitoring data out to be able to integrate with any external monitoring system

Support additional data formats

Add capability to support Parquet, Avro, ORC and XML data formats out of the box

Auto-Schema Detection

Add capability to auto-detect the schema (both input data and storage table) based on the data format and connector type

Schema Evolution

Add capability to ensure that schema evolution is handled automatically at the processing and storage layer

Masking & Encryption

Add capability to mask or encrypt the data while flowing in to address data privacy concerns

JSONata and SQL Transformations

Add capability to provide custom transformation scripts in JSONata or SQL which perform transformations in real-time


Add capability for lakehouse. Hudi is already tested and is in experimental mode. It is now being battle hardened before being added as part of open-source

Right to be forgotten

Add scripts to provide the ability to safely delete a dataset from all storages

Data Aliases

Add aliases to tables or datasources (similar to ElasticSearch) so that data replay and migrations are simplified and can be performed without any downtime

Query Access Control

Add capability to add access control of the data via OPA rules. An API to manage all access policies across any dataset

Sink Connectors

Add capability to reverse ETL the processed and enriched data

Auto Scaling

Add ability to auto-scale the infra based on processing speed, data lag/back-pressure and query response times

API Management

Add ability to configure consume tokens for the APIs so that integrations with end-user systems are seamless

New Connectors

Add new Database connectors - Oracle, SQL Server, MongoDB, Cassandra, ElasticSearch

Add new Stream connectors - Postgresql Debezium, MySQL Debezium, DB2 Debezium, Oracle Debezium, SQL Server Debezium, MongoDB Debezium, Cassandra Debezium

Add new File connectors - Azure Blob Storage, MinIO, Google Cloud Storage

Simplified Archival and Retention policies

Ability to apply the archival and retention policy on the datasets and table via a unified API without any knowledge about the underlying storage (whether the data is stored in Hudi or Druid)

Last updated