Obsrv 1.1.0 Beta Release

📅 Release Date: 31st August 2024

🌟 What's New?

🚀 Core Innovations

  • 🌟 Transaction Data Lake Platform

    • Revolutionize Your Data Landscape with Obsrv! Introducing our all-new Obsrv Transaction Data Lake Platform, now supercharged with Apache Hudi (v0.15.0). This powerhouse is designed to elevate your data management to stratospheric heights! 🌌

  • 🔗 Data Connector Framework

    • 🛠️ Easy Connector Setup: Quickly configure connectors and start gathering data from various sources with ease.

    • 🔧 Flexible Framework: Customize and build your own connectors by extending the core framework.

    • 🌟 Introduction to Batch and Stream Connectors: Effortlessly implement a batch connector using Python or Java or Scala and a stream connector with Scala or Java, leveraging the connector frameworks and provided examples.

    • 🌐 New Source Connectors:

      • Kafka 🟪 Stream real-time data from external Kafka clusters by configuring the Kafka connector with your topic and host details. Easily integrate live data into your system.

      • Object Store 🗂️ Access and manage data from various cloud storage providers such as AWS, Azure, and GCP using the Object Store connector. Simplify your cloud data retrieval.

      • JDBC 🔗 Connect to relational databases and efficiently pull data with the JDBC connector. Configure it to interface with your database, ensuring smooth and reliable data integration.

      • Knowlg 📚 Connect with the Sunbird Knowlg graph events by providing topic and host details. Leverage this integration for enhanced data-driven insights and decision-making.

  • ✨ Real-Time Data Transformation Capabilities Transform your data effortlessly with advanced features such as encryption, data masking, and JSONata transformations. Your data is well-protected and efficiently processed. Learn more about JSONata at jsonata.org.

  • ⚡ Cache Indexing Speed matters. Our new Redis Cache indexing, powered by a cutting-edge streaming Flink job, ensures your data retrieval for denormalization is faster than ever.

  • 🔍 Schema Validation Enhancement Stay compliant and future-proof with our upgraded validator, now supporting Draft-12 of the JSON Schema specification.

  • 🛠️ Table Schema Modifications We’ve refined our table schemas to seamlessly integrate new features, making your data management smoother and more efficient.

  • 🔐 Enhanced Security Measures As part of our core innovations, we have tackled critical security vulnerabilities, including SQL injection issues, to significantly strengthen the security of our platform. These improvements are designed to protect your data from potential threats and ensure a more reliable and resilient data management experience.

🌐 API Enhancements

  • 📊 Dataset Management APIs

    • 📁 CRUD Operations: Full control at your fingertips—Create, Read, Update, and Delete your datasets with ease.

    • 🔄 Status Transition: Effortlessly manage and transition dataset statuses with our new API.

    • 🔎 Health Monitoring: Keep your system in check with our comprehensive health monitoring APIs.

    • 📤 Export & Import: Export and import dataset configurations seamlessly across environments.

    • 🌀 Copy & Reset: Simplify dataset management with powerful copy and reset functionalities.

    • ⚙️ Schema Generation: Automatic schema generation just got simpler—let our API do the heavy lifting.

    • 📋 Dataset Listing: Easily retrieve a comprehensive list of all available datasets.

  • 🔌 Connector APIs

    • 🗂️ Read & List Connectors: Manage and explore your connectors effortlessly with our read and list APIs.

  • 📋 Schema and Event Validation API Ensure your data's integrity with powerful APIs for validating schemas and events against established standards.

  • 🧩 Template APIs

    • 🛠️ Template Management: Full CRUD operations for templates, giving you ultimate control.

    • 🔍 Template Query: Query data from various sources using our robust template-driven API.

  • 📈 Data APIs

    • 🔎 Data Query: Access and query your data effortlessly using SQL or native formats.

    • 📥 Data Ingestion: Bring your data into the system with our streamlined ingestion API.

    • 📤 Data Exhaust: Retrieve and manage your data from the object store without hassle.

  • 🚨 Alert and Notification APIs

    • 🔔 Custom Alerts: Configure custom alerts tailored to your needs. Set up notifications for specific events to stay informed and proactive.

    • 📢 Notification Channels: Easily configure and manage various notification channels, including Slack, email, and Discord, to ensure you receive alerts through your preferred methods.

🔧 Obsrv Management Console

  1. 🚀 Effortless Dataset Management Total Control at Your Fingertips! Manage your datasets with precision throughout their entire lifecycle—Draft, Ready for Publish, Publish, and Retire. The Obsrv Management Console offers unparalleled ease of use, allowing you to navigate complex data workflows with confidence.

  1. 🔗 Advanced Connector Configuration Connect Like Never Before! Seamlessly configure Source Connectors and Sink Connectors with a few clicks. The console’s intuitive design simplifies the process of linking diverse data sources and destinations, empowering you to manage and monitor connections effortlessly.

  1. 🔄 Seamless Export & Import Data Mobility Made Simple! Transfer datasets between environments with our streamlined Export and Import functionalities. Whether you're migrating data, sharing configurations, or backing up important information, Obsrv makes the process smooth and secure.

  1. ⚙️ Automatic Schema Generation & Indexing Efficiency at Its Best! Instantly generate schemas tailored to your datasets, and index data across multiple sources, including Redis, Druid, Lakehouse and more. Obsrv ensures that your data is always ready for quick retrieval and analysis.

  1. 🚨 Proactive Alerts & Health Monitoring Stay Ahead with Intelligent Monitoring! Configure smart alerts and keep a close watch on the health of your datasets and connectors, ensuring smooth and uninterrupted operations.

  1. 📊 Comprehensive Resource Monitoring Keep Your Operations in Check! Monitor your system's performance with advanced resource monitoring tools, fully integrated into the Obsrv Management Console.

  1. 🔍 Powerful Data Querying & Analysis Unleash the Power of Your Data! Utilize built-in querying tools to extract insights from your datasets. Run SQL queries, explore data in native formats, and generate reports with ease.

And that's just the beginning! The Obsrv Management Console is packed with even more features designed to enhance your data management experience. Explore the full range of functionalities and discover how Obsrv can transform your data operations. 🌟

🔧 Automation Upgrades

  • 🚀 Enhanced Automation Scripts

    • ☁️ Cloud & On-Premises Support: Expanded support across AWS, GCP, Azure, and on-premises environments—your automation just got more versatile.

    • 🔧 Simplified Installation: Deploy with confidence using our newly refined, easy-to-use installation scripts.

🚫 Deprecation Notice

  • ⚠️ Deprecation of v1 APIs: APIs from version 1.0.x will no longer be supported in this release. We recommend transitioning to the new APIs to ensure ongoing support and access to the latest features.

🔄 Migration Guide

  • 🔧 How to Migrate from v1.0.x to v1.1.0: To Do: Detailed migration instructions will be added soon. Stay tuned for the comprehensive migration guide.

PreviousRelease Notes

Last updated