Dev Requirements

Java / Scala

Prerequisites

Stream Connectors

• Java 11

• Scala 2.12.11

• Apache Flink 1.17.2

• Apache Kafka 2.8.1

Batch Connectors

• Java 11

• Scala 2.12.11

• Apache Spark 3.5.1

• Apache Kafka 2.8.1

Libraries

Make sure you have the necessary repositories for the development

git clone [email protected]:Sunbird-Obsrv/job-sdk-scala.git
git clone [email protected]:Sunbird-Obsrv/connector-sdk-scala.git

Setup

1. job-sdk-scala

cd job-sdk-scala
mvn clean install

1. connector-sdk-scala

cd connector-sdk-scala
mvn clean install

Adding Dependencies

Stream Connectors

Add the following to your project's pom.xml file under dependencies

pom.xml

<dependencies>
    ...
        <dependency>
            <groupId>org.sunbird.obsrv.connector</groupId>
            <artifactId>connector-sdk-flink</artifactId>
            <version>1.0.0</version>
        </dependency>
    ...
</dependencies>

Batch Connectors

Add the following to your project's pom.xml file under dependencies

pom.xml

<dependencies>
    ...
        <dependency>
            <groupId>org.sunbird.obsrv.connector</groupId>
            <artifactId>connector-sdk-spark</artifactId>
            <version>1.0.0</version>
        </dependency>
    ...
</dependencies>

Python