We know getting existing data into Orchestrate is one of the barriers to trying us out. As such, we are pleased to announce the first in a suite of tools to help users migrate existing data, the Orchestrate JDBC Importer. The Importer was initially forked from Algolia’s JDBC Java Connector, but has since seen major changes and improvements.

The JDBC Importer allows users to transfer data from existing JDBC compatible databases. Out of the box it supports MySQL and PostgreSQL. Other databases will be supported in the future, but given that Oracle DB’s and Microsoft SQL Server’s JDBC drivers are under restrictive licensing they will be supported under a bring your own driver model.

To get started with the JDBC Importer, download the latest version from the releases page on Github. You will need Java 7 or later and uuencode/uudecode installed on your system. To run the importer, execute orchestrate-jdbc-importer passing in a JSON configuration file like so:

{
   "selectQuery" : "A SQL query used to fetch row to be imported",
   "primaryField" : "The field to be used as the key in Orchestrate" ,
   "source" : "jdbc:mysql://your.db.host/your.db",
   "username" : "DB Username",
   "password" : "DB Password",
   "apiKey" : "YourOrchestrateAPIKey",
   "collection" : "YourCollection",
   "dump": true
}
orchestrate-jdbc-importer configuration.json

Once started, the JDBC Importer will iterate over the result set produced by the “selectQuery” and convert each row into JSON. It will then PUT the data into Orchestrate with the provided “collection” and the value of the row’s “primaryField”. The importer also has the ability to run periodically and either re-execute the “selectQuery” or use an “updateQuery” and only import the data that was been added or changed since the last run.

With the importer, users of RDBMS can easily migrate to and synchronize data with Orchestrate. Before, migrating data would require writing a one-off ETL tool. Now, developers can simply point us at their existing database, tell us how to traverse it, and we do the rest.

The importer is MIT Licensed. Contributions and issue reporters are welcomed on Github.

P.S. Be on the lookout for a similar solution for the NoSQL world.