Elasticsearch / Jörg Prante – JDBC importer

Background

On the ElasticSearch’s web site, one of the tools listed under the “Data Import & Export Validation page” is Jörg Prante’s – JDBC importer.

In this post, we will do a quick assessment.

 

Artifacts

Jörg Prante’s – JDBC importer is open source and it is available on GitHub.

The project’s URL is https://github.com/jprante/elasticsearch-jdbc.

The releases URL is https://github.com/jprante/elasticsearch-jdbc/releases.

As of today (2018-July-30), here is a screen shot.

The most recent release is 2.3.4.1.

The date is 2016-August-28th; a month short of 2 years ago.

Download

Downloaded the latest version.

 

Build

Built the maven project by invoking the gradlew.bat batch file.

 

Prepare

Environment Variable

JAVA_HOME

Prepared the command shell environment by setting the JAVA_HOME Environment variable.

Here we set JAVA_HOME to our installed Java JDK 1.8 folder.

 

set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181

 

Command

 

gradlew clean install 

Output

 

Build Deliverable

The build deliverable is placed in the build\distributions folder.

Based on our Source Code Release Number; the exact file name is elasticsearch-jdbc-2.3.4.1-2.3.4.1-dist.zip.

 

Copy to Lib Folder

Please extract the Lib folder from the *dist.zip into the Lib folder under the main folder.

 

Processing

Preparation

Prepare payload

We chose to package our script into two files.  The main initiator and the JSON configuration.

stackoverflow2010.json


{
    "jdbc" :
	{
        "url" : "jdbc:sqlserver://localhost:1433;databaseName=Stackoverflow2010;"
        , "user" : "stackoverflow"
        , "password" : "hIy8jA2lNl"
        , "sql" :  "select top 1 * from Stackoverflow2010.dbo.Users"
	}

    , "elasticsearch" :
		{
             "cluster" : "elasticsearch"
           , "host" : "localhost"
           , "port" : 9200
        }

    , "index" : "stackoveflow2010"
	, "type" : "user"
}

 

invokeStackoverflow2010.cmd


@echo off

setlocal

set DIR=%~dp0

set LIB=%DIR%..\lib\*

set BIN=%DIR%..\bin

set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181"

"%JAVA_HOME%\bin\java" -cp %LIB% -Dlog4j.configurationFile="%BIN%\log4j2.xml" "org.xbib.tools.Runner" "org.xbib.tools.JDBCImporter"	stackoverflow2010.json

endlocal

Invoke Command File

Invoked Command File.

Nothing shows up on the console.

But,  bad news on the elasticSearch log file.

Text


java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [5.6.0]
	at org.elasticsearch.transport.TcpTransport.ensureVersionCompatibility(TcpTransport.java:1462) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1409) ~[elasticsearch-6.3.2.jar:6.3.2]
	at 

Explanation

  1. Received message from unsupported version: [2.0.0] minimal compatible version is: [5.6.0]

 

Other Issues

Compile \ Build

Java JDK

Version

Unfortunately, when we tried using Java JDK 10x we ran into a ditch.

Prepare Environment

To prepare the environment:


set java_home=C:\Program Files\Java\jdk-10.0.2

Invoke gradlew

Command Issued is:


gradlew --help

Output

and the response received is ….

Image

build_java_jdk_v10_20180731_0812PAM

Textual

>set java_home=C:\Program Files\Java\jdk-10.0.2

>gradlew --help

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine java version from '10.0.2'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

>

Summary

Other issues aside, looks like we fell victim to Client’s API compatibility issue.

Our tool is a bit dated as it is stuck at v 2.0.0

Unfortunately, ElasticSearch requires at minimum Version 5.6.0

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s