Sunbird Obsrv
  • Introduction
    • The Value of Data
    • Data Value Chain
    • Challenges
    • The Solution: Obsrv
  • Core Concepts
    • Obsrv Overview
    • Key Capabilities
    • Datasets
    • Connectors
    • High Level Architecture
    • Tech Stack
    • Monitoring
  • Explore
    • Roadmap
    • Case Studies
      • Agri Climate Advisory
      • Learning Analytics at Population Scale
      • IOT Observations Infra
      • Data Driven Features in Learning Platform
      • Network Observability
      • Fraud Detection
    • Performance Benchmarks
  • Guides
    • Installation
      • AWS Installation Guide
      • Azure Installation Guide
      • GCP Installation Guide
      • OCI Installation Guide
      • Data Center Installation Guide
    • Dataset Management APIs
    • Dataset Management Console
    • Connector APIs
    • Data In & Out APIs
    • Alerts and Notification Channels APIs
    • Developer Guide
    • Example Datasets
    • Connectors Developer Guide
      • SDK Assumptions
      • Required Files
        • metadata.json
        • ui-config.json
        • metrics.yaml
        • alerts.yaml
      • Obsrv Base Setup
      • Dev Requirements
      • Interfaces
        • Stream Interfaces
        • Batch Interfaces
      • Classes
        • ConnectorContext Class
        • ConnectorStats Class
        • ConnectorState Class
        • ErrorData Class
        • MetricData Class
      • Verifying
      • Packaging Guide
      • Reference Implementations
    • Coming Soon!
  • Community
  • Previous Versions
    • SB-5.0 Version
      • Overview
      • USE
        • Release Notes
          • Obsrv 2.0-Beta
          • Obsrv 2.1.0
          • Obsrv 2.2.0
          • Obsrv 2.0.0-GA
          • Obsrv 5.3.0-GA
          • Release V 5.1.0
          • Release V 5.1.2
          • Release V 5.1.3
          • Release V 5.0.0
          • Release V 4.10.0
        • Installation Guide
        • Obsrv 2.0 Installation Guide
          • Getting Started with Obsrv Deployment Using Helm
        • System Requirements
      • LEARN
        • Functional Capabilities
        • Dependencies
        • Product Roadmap
        • Product & Developer Guide
          • Telemetry Service
          • Data Pipeline
          • Data Service
          • Data Product
            • On Demand Druid Exhaust Job
              • Component Diagram
              • ML CSV Reports
              • Folder Struture
          • Report Service
          • Report Configurator
          • Summarisers
      • ENGAGE
        • Discuss
        • Contribute to Obsrv
      • Raise an Issue
  • Release Notes
    • Obsrv 1.1.0 Beta Release
    • Obsrv 1.2.0-RC Release
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. Guides

Connector APIs

List of APIs to manage data input connectors

PreviousDataset Management ConsoleNextData In & Out APIs

Last updated 5 months ago

Was this helpful?

Connector Config APIs

Connectors are used to ingest data from external sources. It provides a set of APIs to register, list and read connectors configurations API

Read Connector

get
Path parameters
connector_idstringRequired

Unique identifier for the connector

Responses
200
OK
application/json
Responseobject
404
Not Found
application/json
get
GET /v2/connectors/read/{connector_id} HTTP/1.1
Host: localhost:3000
Accept: */*
{
  "id": "api.connectors.read",
  "ver": "v2",
  "ts": "2024-07-31T18:17:54+05:30",
  "params": {
    "status": "SUCCESS",
    "resmsgid": "7587f564-c2d7-49a8-9e56-dc56f6808ced"
  },
  "responseCode": "OK",
  "result": {
    "id": "postgres-connector-1.0.0",
    "connector_id": "postgres-connector",
    "name": "PostgreSQL",
    "type": "source",
    "category": "Database",
    "version": "1.0.0",
    "description": "The PostgreSQL Connector is used to move data from any Postgres Table to the Obsrv platform",
    "licence": "MIT",
    "owner": "Sunbird",
    "iconurl": "https://upload.wikimedia.org/wikipedia/commons/2/29/Postgresql_elephant.svg",
    "status": "Live",
    "ui_spec": {
      "schema": {
        "type": "object",
        "properties": {
          "connector_config": {
            "title": "Connector Config",
            "type": "object",
            "encrypt": true,
            "properties": {
              "databaseType": {
                "type": "string",
                "title": "Database Type",
                "enum": [
                  "PostgreSQL",
                  "MySQL"
                ],
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": ""
                  }
                ]
              }
            },
            "dependencies": {
              "databaseType": {
                "oneOf": [
                  {
                    "properties": {
                      "databaseType": {
                        "enum": [
                          "PostgreSQL",
                          "MySQL"
                        ]
                      },
                      "connection_info": {
                        "title": "Connection Information",
                        "type": "object",
                        "properties": {
                          "host": {
                            "type": "string",
                            "title": "Database Host",
                            "pattern": "/^(?:[a-zA-Z0-9-]+\\.)+[a-zA-Z]{2,}(?:/[^\\s]*)?|localhost(?:/[^\\s]*)?|((?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "port": {
                            "type": "number",
                            "title": "Database Port",
                            "minimum": 1,
                            "maximum": 65535,
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "name": {
                            "type": "string",
                            "title": "Database Name",
                            "pattern": "^[a-zA-Z0-9_]{1,64}$",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "username": {
                            "type": "string",
                            "title": "Database Username",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "password": {
                            "type": "string",
                            "title": "Database Password",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          }
                        }
                      },
                      "schemaInfo": {
                        "title": "Schema Information",
                        "type": "object",
                        "properties": {
                          "table": {
                            "title": "Table Name",
                            "type": "string",
                            "pattern": "^[a-zA-Z_][a-zA-Z0-9_]{0,62}$",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "timestampColumn": {
                            "title": "Timestamp Column",
                            "type": "string",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          }
                        }
                      }
                    }
                  }
                ]
              }
            }
          },
          "operations_config": {
            "title": "Operations Configuration",
            "type": "object",
            "properties": {
              "batch_size": {
                "type": "number",
                "title": "Batch Size",
                "default": 100,
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": ""
                  }
                ]
              },
              "max_batches": {
                "type": "number",
                "title": "Maximum Batches",
                "default": 10,
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": ""
                  }
                ]
              },
              "pollingInterval": {
                "type": "string",
                "title": "Polling Interval",
                "enum": [
                  "Once",
                  "Periodic"
                ],
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": "Select polling interval"
                  }
                ]
              }
            },
            "dependencies": {
              "pollingInterval": {
                "oneOf": [
                  {
                    "properties": {
                      "pollingInterval": {
                        "enum": [
                          "Periodic"
                        ]
                      },
                      "schedule": {
                        "type": "string",
                        "title": "Schedule",
                        "enum": [
                          "Hourly",
                          "Daily",
                          "Weekly",
                          "Monthly"
                        ],
                        "fieldDescription": [
                          {
                            "type": "string",
                            "description": ""
                          }
                        ]
                      }
                    },
                    "required": [
                      "schedule"
                    ]
                  }
                ]
              }
            }
          }
        }
      },
      "properties": {
        "connector_config": {
          "connection_info": {
            "password": {
              "ui:widget": "password"
            }
          }
        },
        "operations_config": {
          "batch_size": {
            "ui:readonly": true
          },
          "max_batches": {
            "ui:readonly": true
          }
        }
      }
    },
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM",
    "created_date": "2024-06-25T04:38:28.732Z",
    "updated_date": "2024-06-25T04:38:28.732Z",
    "live_date": "2024-06-25T04:38:28.732Z"
  }
}
  • Connector Config APIs
  • POSTRegister Connector
  • GETRead Connector
  • POSTList Connectors

Register Connector

post

Registers a connector by uploading a file.

Body
file3string ยท binaryOptional

Connector distribution file to upload.

Responses
200
Connector registered successfully.
application/json
500
Internal server error while registering the connector.
application/json
post
POST /v2/connector/register HTTP/1.1
Host: localhost:3000
Content-Type: multipart/form-data
Accept: */*
Content-Length: 18

{
  "file3": "binary"
}
{
  "id": "api.connector.register",
  "ver": "v2",
  "ts": "2024-11-19T09:28:47+00:00",
  "params": {
    "status": "SUCCESS",
    "resmsgid": "fc2ce24f-a333-479f-931c-024d61039801"
  },
  "responseCode": "OK",
  "result": {
    "message": "connector registered successfully."
  }
}

List Connectors

post
Body
objectOptionalExample: {"id":"api.connectors.list","ver":"v2","ts":"2024-04-10T16:10:50+05:30","params":{"msgid":"4a7f14c3-d61e-4d4f-be78-181834eeff6d"},"request":{"filters":{"status":["Draft"]}}}
Responses
200
OK
application/json
Responseobject
400
Bad Request
application/json
post
POST /v2/connectors/list HTTP/1.1
Host: localhost:3000
Content-Type: application/json
Accept: */*
Content-Length: 173

{
  "id": "api.connectors.list",
  "ver": "v2",
  "ts": "2024-04-10T16:10:50+05:30",
  "params": {
    "msgid": "4a7f14c3-d61e-4d4f-be78-181834eeff6d"
  },
  "request": {
    "filters": {
      "status": [
        "Draft"
      ]
    }
  }
}
{
  "id": "api.connectors.list",
  "ver": "v2",
  "ts": "2024-07-30T15:25:51+05:30",
  "params": {
    "status": "SUCCESS",
    "msgid": "4a7f14c3-d61e-4d4f-be78-181834eeff6d",
    "resmsgid": "f506e725-eed4-41df-86dc-2477d5c4d19a"
  },
  "responseCode": "OK",
  "result": {
    "data": [],
    "count": 0
  }
}