Sunbird Obsrv
  • Welcome to Obsrv
    • The Value of Data
    • Data Value Chain
    • Challenges
    • The Solution: Obsrv
  • Core Concepts
    • Obsrv Overview
    • High Level Architecture
    • Key Capabilities
    • Datasets
    • Connectors
    • Tech Stack
    • Monitoring
  • Explore
    • Roadmap
    • Case Studies
      • Agri Climate Advisory
      • Learning Analytics at Population Scale
      • IOT Observations Infra
    • Performance Benchmarks
  • Guides
    • Installation Guide
      • AWS
      • Azure
      • GCP
      • OCI
      • Data Center
    • API Specification
      • Dataset Management APIs
      • Connector APIs
      • Data In & Out APIs
      • Alerts and Notification Channels APIs
    • Dataset Management Console
    • Developer Guide
    • Example Datasets
    • Connectors Developer Guide
      • SDK Assumptions
      • Required Files
        • metadata.json
        • ui-config.json
        • metrics.yaml
        • alerts.yaml
      • Obsrv Base Setup
      • Dev Requirements
      • Interfaces
        • Stream Interfaces
        • Batch Interfaces
      • Classes
        • ConnectorContext Class
        • ConnectorStats Class
        • ConnectorState Class
        • ErrorData Class
        • MetricData Class
      • Verifying
      • Packaging Guide
      • Reference Implementations
    • Coming Soon!
  • Community
  • Previous Versions
    • SB-5.0 Version
      • Overview
      • USE
        • Release Notes
          • Obsrv 2.0-Beta
          • Obsrv 2.1.0
          • Obsrv 2.2.0
          • Obsrv 2.0.0-GA
          • Obsrv 5.3.0-GA
          • Release V 5.1.0
          • Release V 5.1.2
          • Release V 5.1.3
          • Release V 5.0.0
          • Release V 4.10.0
        • Installation Guide
        • Obsrv 2.0 Installation Guide
          • Getting Started with Obsrv Deployment Using Helm
        • System Requirements
      • LEARN
        • Functional Capabilities
        • Dependencies
        • Product Roadmap
        • Product & Developer Guide
          • Telemetry Service
          • Data Pipeline
          • Data Service
          • Data Product
            • On Demand Druid Exhaust Job
              • Component Diagram
              • ML CSV Reports
              • Folder Struture
          • Report Service
          • Report Configurator
          • Summarisers
      • ENGAGE
        • Discuss
        • Contribute to Obsrv
      • Raise an Issue
  • Release Notes
    • Obsrv 1.1.0 Beta Release
    • Obsrv 1.2.0-RC Release
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. Guides
  2. API Specification

Connector APIs

List of APIs to manage data input connectors

PreviousDataset Management APIsNextData In & Out APIs

Last updated 6 months ago

Was this helpful?

Connector Config APIs

Connectors are used to ingest data from external sources. It provides a set of APIs to register, list and read connectors configurations API

Read Connector

get
Path parameters
connector_idstringRequired

Unique identifier for the connector

Responses
200
OK
application/json
Responseobject
404
Not Found
application/json
get
GET /v2/connectors/read/{connector_id} HTTP/1.1
Host: localhost:3000
Accept: */*
{
  "id": "api.connectors.read",
  "ver": "v2",
  "ts": "2024-07-31T18:17:54+05:30",
  "params": {
    "status": "SUCCESS",
    "resmsgid": "7587f564-c2d7-49a8-9e56-dc56f6808ced"
  },
  "responseCode": "OK",
  "result": {
    "id": "postgres-connector-1.0.0",
    "connector_id": "postgres-connector",
    "name": "PostgreSQL",
    "type": "source",
    "category": "Database",
    "version": "1.0.0",
    "description": "The PostgreSQL Connector is used to move data from any Postgres Table to the Obsrv platform",
    "licence": "MIT",
    "owner": "Sunbird",
    "iconurl": "https://upload.wikimedia.org/wikipedia/commons/2/29/Postgresql_elephant.svg",
    "status": "Live",
    "ui_spec": {
      "schema": {
        "type": "object",
        "properties": {
          "connector_config": {
            "title": "Connector Config",
            "type": "object",
            "encrypt": true,
            "properties": {
              "databaseType": {
                "type": "string",
                "title": "Database Type",
                "enum": [
                  "PostgreSQL",
                  "MySQL"
                ],
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": ""
                  }
                ]
              }
            },
            "dependencies": {
              "databaseType": {
                "oneOf": [
                  {
                    "properties": {
                      "databaseType": {
                        "enum": [
                          "PostgreSQL",
                          "MySQL"
                        ]
                      },
                      "connection_info": {
                        "title": "Connection Information",
                        "type": "object",
                        "properties": {
                          "host": {
                            "type": "string",
                            "title": "Database Host",
                            "pattern": "/^(?:[a-zA-Z0-9-]+\\.)+[a-zA-Z]{2,}(?:/[^\\s]*)?|localhost(?:/[^\\s]*)?|((?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "port": {
                            "type": "number",
                            "title": "Database Port",
                            "minimum": 1,
                            "maximum": 65535,
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "name": {
                            "type": "string",
                            "title": "Database Name",
                            "pattern": "^[a-zA-Z0-9_]{1,64}$",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "username": {
                            "type": "string",
                            "title": "Database Username",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "password": {
                            "type": "string",
                            "title": "Database Password",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          }
                        }
                      },
                      "schemaInfo": {
                        "title": "Schema Information",
                        "type": "object",
                        "properties": {
                          "table": {
                            "title": "Table Name",
                            "type": "string",
                            "pattern": "^[a-zA-Z_][a-zA-Z0-9_]{0,62}$",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          },
                          "timestampColumn": {
                            "title": "Timestamp Column",
                            "type": "string",
                            "fieldDescription": [
                              {
                                "type": "string",
                                "description": ""
                              }
                            ]
                          }
                        }
                      }
                    }
                  }
                ]
              }
            }
          },
          "operations_config": {
            "title": "Operations Configuration",
            "type": "object",
            "properties": {
              "batch_size": {
                "type": "number",
                "title": "Batch Size",
                "default": 100,
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": ""
                  }
                ]
              },
              "max_batches": {
                "type": "number",
                "title": "Maximum Batches",
                "default": 10,
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": ""
                  }
                ]
              },
              "pollingInterval": {
                "type": "string",
                "title": "Polling Interval",
                "enum": [
                  "Once",
                  "Periodic"
                ],
                "fieldDescription": [
                  {
                    "type": "string",
                    "description": "Select polling interval"
                  }
                ]
              }
            },
            "dependencies": {
              "pollingInterval": {
                "oneOf": [
                  {
                    "properties": {
                      "pollingInterval": {
                        "enum": [
                          "Periodic"
                        ]
                      },
                      "schedule": {
                        "type": "string",
                        "title": "Schedule",
                        "enum": [
                          "Hourly",
                          "Daily",
                          "Weekly",
                          "Monthly"
                        ],
                        "fieldDescription": [
                          {
                            "type": "string",
                            "description": ""
                          }
                        ]
                      }
                    },
                    "required": [
                      "schedule"
                    ]
                  }
                ]
              }
            }
          }
        }
      },
      "properties": {
        "connector_config": {
          "connection_info": {
            "password": {
              "ui:widget": "password"
            }
          }
        },
        "operations_config": {
          "batch_size": {
            "ui:readonly": true
          },
          "max_batches": {
            "ui:readonly": true
          }
        }
      }
    },
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM",
    "created_date": "2024-06-25T04:38:28.732Z",
    "updated_date": "2024-06-25T04:38:28.732Z",
    "live_date": "2024-06-25T04:38:28.732Z"
  }
}
  • Connector Config APIs
  • POSTRegister Connector
  • GETRead Connector
  • POSTList Connectors

Register Connector

post

Registers a connector by uploading a file.

Body
file3string ยท binaryOptional

Connector distribution file to upload.

Responses
200
Connector registered successfully.
application/json
500
Internal server error while registering the connector.
application/json
post
POST /v2/connector/register HTTP/1.1
Host: localhost:3000
Content-Type: multipart/form-data
Accept: */*
Content-Length: 18

{
  "file3": "binary"
}
{
  "id": "api.connector.register",
  "ver": "v2",
  "ts": "2024-11-19T09:28:47+00:00",
  "params": {
    "status": "SUCCESS",
    "resmsgid": "fc2ce24f-a333-479f-931c-024d61039801"
  },
  "responseCode": "OK",
  "result": {
    "message": "connector registered successfully."
  }
}

List Connectors

post
Body
objectOptionalExample: {"id":"api.connectors.list","ver":"v2","ts":"2024-04-10T16:10:50+05:30","params":{"msgid":"4a7f14c3-d61e-4d4f-be78-181834eeff6d"},"request":{"filters":{"status":["Draft"]}}}
Responses
200
OK
application/json
Responseobject
400
Bad Request
application/json
post
POST /v2/connectors/list HTTP/1.1
Host: localhost:3000
Content-Type: application/json
Accept: */*
Content-Length: 173

{
  "id": "api.connectors.list",
  "ver": "v2",
  "ts": "2024-04-10T16:10:50+05:30",
  "params": {
    "msgid": "4a7f14c3-d61e-4d4f-be78-181834eeff6d"
  },
  "request": {
    "filters": {
      "status": [
        "Draft"
      ]
    }
  }
}
{
  "id": "api.connectors.list",
  "ver": "v2",
  "ts": "2024-07-30T15:25:51+05:30",
  "params": {
    "status": "SUCCESS",
    "msgid": "4a7f14c3-d61e-4d4f-be78-181834eeff6d",
    "resmsgid": "f506e725-eed4-41df-86dc-2477d5c4d19a"
  },
  "responseCode": "OK",
  "result": {
    "data": [],
    "count": 0
  }
}