In this document, we show how to build custom connectors to any relational/non-relational database management system. These connectors interface Sclera with an arbitrary database system, relational or non-relational, providing access to the underlying data, and also enable Sclera to push down computations in relevant parts user queries and commands on the interfaced database system.

Sclera - Oracle Connector, Sclera - MySQL Connector, Sclera - PostgreSQL Connector, and Sclera - Apache HBase Connector are built using this SDK. For examples of how these connectors are used in Sclera, please refer to the documentation on connecting Sclera to database systems.

Building Database System Connectors

To build a custom datasource connector, you need to provide implementations of the following abstract classes in the SDK:

  • DBService (Scala, Java)
    • Provides the database system as a service to Sclera.
    • Contains an id that identifies this service.
    • Contains a method createLocation that is used to create a new location instance for this service.
  • Location (Scala, Java)
    • Represents the underlying database system.
    • Provides the configuration parameters, and properties (e.g. temporary or persistent, read-only or read-write, etc.).
    • Provides the driver for interfacing with the underlying system.
  • StatementDriver (Scala, Java)
    • Executes the statements provided by Sclera on the underlying database system, and passes back the results.
    • Provides the metadata driver for accessing the metadata for the data stored in the underlying database system.
  • StatementMetadataDriver (Scala, Java)
    • Provides the metadata for the data stored in the underlying database system.

Special case: Relational Databases

If the underlying system is a relational database system, which talks SQL and uses JDBC as the interface, Sclera does most of the work. For such systems, you need to provide implementations of the following:

The Sclera - MySQL Connector, included with the Sclera platform, is open source and implements the relational database interface mentioned above.

Packaging and Deploying the Connector

The included Sclera - MySQL Connector implementation uses sbt for building the connector (installation details). This is not a requirement – any other build tool can be used instead.

Dependencies

For Scala:

  • The Scala implementation has a dependency on the "sclera-core" library. This library is available from the Sclera repository. Note that the dependency should be annotated "provided" since the jar for "sclera-core" will be available in the CLASSPATH when this connector is run with Sclera.

For Java:

Deployment Steps

The connector can be deployed using the following steps:

Alternative 1 (Recommended)

  • First, publish the implementation as a local package. In sbt, this is done by running sbt publish-local.
  • Run the following to install the component and its dependencies:

    > $SCLERA_HOME/bin/install.sh package_name package_version package_org
    • $SCLERA_HOME is the directory where Sclera is installed
    • package_name is the name of the package being installed
    • package_version is the version of the package being installed
    • package_org is the org of the package being installed
  • Run the following to include the path to the installed component package jar and the dependencies in the CLASSPATH:

    > . $SCLERA_HOME/assets/install/classpath/package_name-classpath.sh
    • The bash script package_name-classpath.sh was automatically created in the previous step, while installing the package. The package_name part of the file name is the name of the package installed.

The connector should now be visible to Sclera, and can be used in the queries.

Alternative 2

  • First, compile and package the implementation into a jar file. In sbt, this is done by running sbt package. If you have external dependencies, you may need to assemble everything into a single jar – in sbt, this can be done using the sbt-assembly plugin.
  • Next, add the path to the generated jar file to the CLASSPATH environment variable.

The connector should now be visible to Sclera, and can be used in the queries.

Note: Please ensure that the identifier you assign to the connector is unique, that is - does not conflict with the identifier of any other available DBService instance.


Related Documentation