On MS Windows, Profiling Java Apps Memory Utilization Using VMMap

Background

In our last post, we covered the usage of a few Oracle standard tools to profile Java Applications.

BTW the referenced post is available here.

 

SysInternals

VMMap

In this post, we will briefly cover using SysInternal’s VMMap GUI to track down memory utilization.

 

Artifact

VMMap is available from here.

The current version is v3.21 and it was released a couple of years ago; July 20, 2015 to be exact.

Download & Extract

Download and Extract it; no need for installation.

Usage

ElasticSearch

Select or Launch Process

 

Metrics

 

WebSphere

Select or Launch Process

 

Metric

Logstash – Configuration – Field Names – document_id

Background

In Logstash, got bogged down trying to use the document_id Configuration Option.

Reproduction

logstash

Configuration

Image

configuration_file_20180809_0820AM.PNG

ElasticSearch

Query

Image

query_stackoverflowUser_20180809_0825AM.PNG

Data

Image

stackoverflowUser_output_20180809_0830AM.PNG

 

Troubleshooting

ElasticSearch

Data Review

  1. Hits
    • Number of Hits
      • We added 10 records
        • Data Source Query says “select top 10 *
      • But,  hits/total
      • Reads  1
    • Lone Hit
      • Data
        • _id
          • “_id”: “%{[Id]}”
        • field
          • id
            • “id”: 10

Logstash

Output

Image

output_20180809_0851AM.PNG

Textual


{"id":-1,"lastaccessdate":"2008-08-26T07:16:53.810Z","reputation":1,"accountid":-1,"location":"on the server farm","downvotes":890820,"displayname":"Community","@timestamp":"2018-08-09T13:48:09.088Z","age":null,"emailhash":null,"views":649,"@version":"1","creationdate":"2008-07-31T07:00:00.000Z","aboutme":"Hi, I'm not really a person.

\n\nI'm a background process that helps keep this site clean!

\n\nI do things like

\n\n
<ul>\n
	<li>Randomly poke old unanswered questions every hour so they get some attention</li>
\n
	<li>Own community questions and answers so nobody gets unnecessary reputation from them</li>
\n
	<li>Own downvotes on spam/evil posts that get permanently deleted</li>
\n
	<li>Own suggested edits from anonymous users</li>
\n
	<li><a href="http://meta.stackexchange.com/a/92006">Remove abandoned questions</a></li>
\n</ul>
\n","websiteurl":"http://meta.stackexchange.com/","upvotes":225495}

SQL Server

Diagram

dbo.Users

Image

stackOverflow2010.Users.PNG

Explanation
  1. Column name is Id

Resolution

In Logstash, Field names are lower cased

Logstash

Configuration

Image

configuration_file_20180809_0903AM.PNG

ElasticSearch

Query

Query Output

Image

output_20180809_0908AM.PNG

Explanation
  1. hits = 10

References

  1. Logstash
    • Configuring
      • Logstash Reference [6.3] » Configuring Logstash » Accessing Event Data and Fields in the Configuration
        Link
    • Transforming Data
      • Logstash Reference [6.3] » Transforming Data » Extracting Fields and Wrangling Data
        Link
    • Configuring Logstash
      • Logstash Reference [6.3] » Configuring Logstash » Structure of a Config File
        Link
    • Output
      • Logstash Reference [6.3] » Output plugins » Elasticsearch output plugin
        Link
    • Filter
      • Logstash Reference [6.2] » Filter plugins » Mutate filter plugin
        Link

 

 

 

 

Logstash – Error – ” Unrecognized VM option ‘UseParNewGC’ “

Background

During my initial evaluation of Logstash ran into an easy to address error.

Reproduce

Invoke

The invocation is straight forward


set "_binfolder=C:\Downloads\Elastic\Logstash\v6.3.2\extract\bin"
set "_configuration=stackOverflow2010.User.conf"

call %_binfolder%\logstashImpl.bat -f %_configuration%

Output

Image

UseParNewGC_20180802_0334PM

Textual


Unrecognized VM option 'UseParNewGC'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Troubleshooting

Java

Java Version

Outline

We can request the version Number for the java runtime executable, java.exe.

Script


java -version

Output

Image

java_version_20180808_0332PM

Textual


java version "10.0.2" 2018-07-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)

Knowledge Base ( KB)

Googled on the the term and got a good hit

  1. [META] Java 10 Support #9345
    • Opened By :- andrewvc
    • Date Opened :- 2018-April-9th
    • Link
      Link
    • Resolution
      • Resolution #1
        • By :- Armin Braun ( original-brownbear )
        • Web Links
        • Workaround :- Currently there is no temporary fix sorry. Switching to Java 8 is the only stable solution for the time being.
      • Resolution #2
        • By :- Josh Daone ( JoshDaone )
        • Web Links
        • Workaround :- @original-brownbear thanks! Switched back to openjdk-8-jre and working properly.

Resolution

Target Specific Java Version

Objective

Rather than rely on Java that is referenced in our path, we will set the environment variable JAVA_HOME to a pre-v10 Version.

Review Installed Java

Launched Windows explorer and access the “C:\Program Files\Java” folder.

Review Java Folders

java_explorer_20180808_0424PM

Code


set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181"

Additional Reading

  1. Andy Luis
    • mpvjava
      • JDK 9 Migration : 5 point checklist for Garbage Collection
        Link
      • About
        Link
  2.  OpenJDK
    • JEP 214: Remove GC Combinations Deprecated in JDK 8
      Link
  3. ORACLE
    • JDK 9 Release Notes – Removed APIs, Features, and Options
      Link

 

Dedicated

Dedicated to Armin Braun ( original-brownbear )
Less Talking … More Coding …

References

  1. Elastic
    • elastic/logstash
      [META] Java 10 Support #9345
      Link

ElasticSearch – Type – Mapping

Background

It is very easy to create new types, tables in traditional tables, and corresponding columns in ElasticSearch.

ElasticSearch

If we do not create mappings before hand, ElasticSearch will create the type, columns and column types for us.

Schema-Less

The ElasticSearch exercises we have tackled thus far relies on the ability of ElasticSeach to create columns upon usage; the first time a record is added.

biblekjv

Here is the schema created for our biblekjv example.

Image

biblekjv_mapping_20180806_1024PM_01

Textual

  1. Columns
      • book
        • type is text
        • And, type is keyword
    • bookID
      • type is long
    • chapter
      • type is long
    • chapterID
      • type is long
    • passage
      • type is text
      • And, type is keyword

Explanation

  1. Numeric entries
    • Are codified as long
    • The largest available datatype for numbers
  2. Strings
    • Represented as both text and keyword
    • Datatype
      • Text
        • Tokenizable
      • Keyword
        • Exact Matches

Schema

Definition

Definition – ElasticSearch

Link

Mapping is the process of defining how a document, and the fields it contains, are stored and indexed. For instance, use mappings to define:

  • which string fields should be treated as full text fields.
  • which fields contain numbers, dates, or geolocations.
  • whether the values of all fields in the document should be indexed into the catch-all _all field.
  • the format of date values.
  • custom rules to control the mapping for dynamically added fields.

Sample – event

Goal

In this post we will create a schema to underine an event.

Sample

Sample – Periscope

Single Event Tables and Common Analysis Queries
Link

event_periscope_20180807_0238PM

Code

Create Type
Outline
  1. We will have two files, the JSON file and the command file
  2. The JSON file
    • Mapping
      • Top – mappings
      • Type :- _doc
      • Properties
        • Property
          • userID :- long
          • loginFirst :- date
          • loginLast :- date
          • firstSpendAt :- date
          • totalSpend :- double
          • dob :- date
          • gender :- keyword ( m/f/etc)  {Gender Inclusive}
          • platform :- keyword ( ios/android)
json

{
    "mappings":
    {
        "_doc":
        {
            "properties":
            {

                "userID":
                {
                    "type": "long"
                }

                , "loginFirst":
                {
                    "type": "date"
                }               

                , "loginLast":
                {
                    "type": "date"
                }                               

                , "firstSpendAt":
                {
                    "type": "date"
                }                                               

                , "totalSpend":
                {
                    "type": "double"
                }                   

                , "dob":
                {
                    "type": "date"
                }   

                , "gender":
                {
                    "type": "keyword"
                }   

                , "platform":
                {
                    "type": "keyword"
                }
            } // "properties"

        } // mapping name

    } //mappings

}  

Batch file

setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

set "_url=http://localhost:9200"

set "_esindex=/event"
set "_estype="

set "_espath=%_esindex%%_estype%"

set "_contentType=Content-Type: application/json

Rem Index - Delete

%_bin% -XDELETE -i %_url%%_espath% 

Rem Index / Mapping - Create

set "_jsonfile=jsonfile\event_mapping.json"

set "_jsonfileFQDN=%cd%\%_jsonfile%"

%_bin% -i  -XPUT %_url%%_espath%  -H "%_contentType%" --data-binary @%_jsonfileFQDN%

endlocal

Output
Output – Delete

type_create_delete_20180807_0253PM.png

Output – Create

typeMappingCreate_20180807_0253PM

Display Type
Outline
  1. We will have a lone file, the command file
  2. Command file
  3. Payload
Batch file

setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

set "_url=http://localhost:9200"

set "_esindex=/event"
set "_estype="
set "_esoperator=/_mapping"

Rem Index / Mapping - List

set "_espath=%_esindex%%_estype%%_esoperator%?pretty"

%_bin% -i  %_url%%_espath% -H "%_contentType%" 

Output
Output – Browse

typeMappingBrowse_20180807_0320PM

Load Data
Outline
    1. There are two files, the JSON file and the command file
    2. The JSON file
      • Two lines for each record
        • The first record will contain the _id
        • The second record sill contain the actual data
          • Property/Value pair
    3. Command file
      • Syntax
        • curl.exe -i -H “Content-Type: application/x-ndjsonhttp://%5Bserver}:{port}/{index}/{type}/_bulk?pretty –data-binary @[filename]
      • Sample
JSON

{"index":{"_id":1}}
{"userID":137314,"loginFirst":"2015-04-01","loginLast":"2015-04-05","firstSpendAt":"2015-04-03","totalSpend":50,"dob":"1994-01-01","gender":"f","platform":"ios"}
{"index":{"_id":2}}
{"userID":137312,"loginFirst":"2015-04-01","loginLast":"2015-04-02","firstSpendAt":"2015-04-02","totalSpend":2500,"dob":"1987-01-01","gender":"m","platform":"android"}
{"index":{"_id":3}}
{"userID":137310,"loginFirst":"2015-04-01","loginLast":"2015-04-01","dob":"2000-01-01","gender":"m","platform":"ios"}
{"index":{"_id":4}}
{"userID":137311,"loginFirst":"2015-04-01","loginLast":"2015-04-02","dob":"1995-01-01","gender":"f","platform":"ios"}
{"index":{"_id":5}}
{"userID":137313,"loginFirst":"2015-04-01","loginLast":"2015-04-04","dob":"1976-01-01","gender":"m","platform":"andriod"}

Command

setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

set "_url=http://localhost:9200"

set "_esindex=event"
set "_estype=_doc"
set "_esoperation=_bulk"

set "_espath=/%_esindex%/%_estype%/%_esoperation%"

set "_jsonfile=jsonldfile\event.json"
set "_jsonfileFQDN=%cd%\%_jsonfile%"

set "_contentType=Content-Type: application/x-ndjson"

%_bin% -i -H "%_contentType%"  %_url%%_espath%?pretty --data-binary @%_jsonfileFQDN%

endlocal
Browse Data
Outline
  1. There are two files the json file and command file
  2. Payload
    • Syntax :-
      • curl.exe -i -H “Content-Type: application/json” http://{server}:{port}/{type}/_search?pretty –data-binary @{filename}
    • Sample :-
JSON

{
   "sort" :
  {
      "_id": {"order": "asc"}

  }
}

Command

setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

set "_url=http://localhost:9200"

set "_esindex=event"
set "_estype=_doc"
set "_esoperation=_search"
set "_espath=/%_esindex%/%_estype%/%_esoperation%"

set "_jsonfile=jsonfile\event_data_browse.json"
set "_jsonfileFQDN=%cd%\%_jsonfile%"

set "_contentType=Content-Type: application/json"

%_bin% -i -H "%_contentType%"  %_url%%_espath%?pretty --data-binary @%_jsonfileFQDN%

endlocal
Output

dataBrowse_20180807_0357PM

Source Control

GitHub

DanielAdeniji/elasticSearchMappingEvent
Link

References

  1. Elastic
    • Docs
      • Mapping
        • Definition
        • Field Datatypes
          • Elasticsearch Reference [6.3] » Mapping » Field datatypes
            Link
          • Elasticsearch Reference [6.3] » Mapping » Field datatypes »Data datatypes
            Link
          • Elasticsearch Reference [6.3] » Mapping » Field datatypes » Numeric datatypes
            Link
          • Elasticsearch Reference [6.3] » Mapping » Field datatypes » Boolean datatype
            Link
        • Indices Mapping
          • Get Mapping
            • Indices Get Mapping
              Link
          • Put Mapping
            • Elasticsearch Reference [5.0] » Indices APIs » Put Mapping
              Link
      • Indices
        • Delete Index
          • Elasticsearch Reference [2.3] » Indices APIs » Delete Index
          • Elasticsearch Reference [6.3] » Indices APIs » Delete Index
      • Reference
      • Elastic for Apache Hadoop
        • Elasticsearch for Apache Hadoop [6.3] » Elasticsearch for Apache Hadoop » Mapping and Types
          « Apache Storm support
          Link
    • Github
      • elastic/elasticsearch
        • INDEX DELETE with wildcard doesn’t delete all matching indexes #7295
          Link
        • v5.3.1 Entire Bulk request is rejected if containing a single invalid index request #24319
          Link
    • Discuss
      • Index a new document/35281
        Link
  2. Periscope
    • Single Event Tables and Common Analysis Queries
      Link
  3. GitHubGist
    • kuznero
      • kuznero/elasticsearch-curl.md
        Link
  4. StackOverflow
    • curl command to close all indices at once in elastic search
      Link
    • Reindex ElasticSearch index returns “Incorrect HTTP method for uri [/_reindex] and method [GET], allowed: [POST]”
      Link
    • Elasticsearch : Root mapping definition has unsupported parameters index : not_analyzed
      Link
  5. logz.io
    • Daniel Berman
      • ElasticSearch Mapping
        Link

 

ElasticSearch – cURL – Batch Operations – Day 1

Background

In earlier posts we exclusively used Postman to perform Data Entry against Elastic Search.

Batch

Preface

In this post we will discuss using cURL, a command line client tool.

What is cURL?

Wikipedia

Link

cURL is a computer software project providing a library and command-line tool for transferring data using various protocols. The cURL project produces two products, libcurl and cURL. It was first released in 1997. The name stands for “Client URL”.

Get cURL

Please get cURL from here.

Please choose the binary specific to your OS, download and extract it.

No need to install, just reference the folder you extracted it into.

 

Use Scenario

Remove Data

Remove Data – Based on Query

Overview

  1. Command File
    • ElasticSearch
      • URL
        • Index :- bible
        • Type :- biblekjv
        • Method :- _delete_by_query
    • cURL
      • Content-Type: application/json
  2. JSON File
    • Query
      • Match on
        • book=Revelation

Command File


REM Retrieving a document
rem https://www.elastic.co/guide/en/elasticsearch/guide/current/get-doc.html

setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

REM ElasticSearch - URL
set "_url=http://localhost:9200"

REM ElasticSearch - Index,Type,Operation
set "_esindex=bible"
set "_estype=biblekjv"
set "_esoperation=_delete_by_query"
set "_espath=/%_esindex%/%_estype%/%_esoperation%"

REM ElasticSearch - Jsonfile
set "_jsonfile=jsonfile\deleteByQuery_book_Revelation.json"
set "_jsonfileFQDN=%cd%\%_jsonfile%"

REM HTTP - ContentType
set "_contentType=Content-Type: application/json"

%_bin% -i -H "%_contentType%"  %_url%%_espath%?pretty --data-binary @%_jsonfileFQDN%

endlocal 

JSON File

{

  "query":
  {
    "bool":
    {
      "must":
      [
        {
            "match":
            {
                "book":"Revelation"
            }
        }
      ]
    }
  }

}

Output

deleteUsingQuery_20180803_0330PM

Remove Data – Based on Bulk

Overview

  1. Command File
    • ElasticSearch
      • URL
        • Index :- bible
        • Type :- biblekjv
        • Method :- _bulk
    • cURL
      • Content-Type: application/x-ndjson
  2. JSON File
    • Content
      • {“delete”:{“_id”:1000}}
        • Remove record whose ID is 1000

Command File


setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

set "_url=http://localhost:9200"

set "_esindex=bible"
set "_estype=biblekjv"
set "_esoperation=_bulk"
set "_espath=/%_esindex%/%_estype%/%_esoperation%"

set "_jsonfile=jsonldfile\deleteBatch.json"
set "_jsonfileFQDN=%cd%\%_jsonfile%"
set "_contentType=Content-Type: application/x-ndjson"

%_bin% -i -H "%_contentType%" %_url%%_espath%?pretty --data-binary @%_jsonfileFQDN%

endlocal

Json file ( line delimited)


{"delete":{"_id":1000}}

Output

deleteUsingBatch_20180803_0351PM.png

Add Data

Add Batch Data based on JSON file (line delimited)

Overview

  1. Command File
    • ElasticSearch
      • URL
        • Index :- bible
        • Type :- biblekjv
        • Method :- _bulk
    • cURL
      • Content-Type: application/x-ndjson
  2. JSON File
    • Content
      • {“index”:{“_id”:1001}}
        {“bookID”:66,”book”:”Revelation”,”chapterID”:1,”verseID”:1,”passage”:”Revelation 1:1″,”verse”:”The Revelation of Jesus Christ, which God gave unto him, to shew unto his servants things which must shortly come to pass; and he sent and signified it by his angel unto his servant John:”}

        • Record – 1
          • Index ID
        • Record – 2
          • Column Value pair
          • Comma delimited
        • Record – N
          • Empty line
          • Make sure not ended with LF nor CR/LF
Command File

setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

set "_url=http://localhost:9200"

set "_esindex=bible"
set "_estype=biblekjv"
set "_esoperation=_bulk"
set "_espath=/%_esindex%/%_estype%/%_esoperation%"

set "_jsonfile=jsonldfile\revelation.json"
set "_jsonfileFQDN=%cd%\%_jsonfile%"

set "_contentType=Content-Type: application/x-ndjson"

%_bin% -i -H "%_contentType%"  %_url%%_espath%?pretty --data-binary @%_jsonfileFQDN%

endlocal 

XML File

{"index":{"_id":1001}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":1,"passage":"Revelation 1:1","verse":"The Revelation of Jesus Christ, which God gave unto him, to shew unto his servants things which must shortly come to pass; and he sent and signified it by his angel unto his servant John:"}
{"index":{"_id":1002}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":2,"passage":"Revelation 1:2","verse":"2Who bare record of the word of God, and of the testimony of Jesus Christ, and of all things that he saw."}
{"index":{"_id":1003}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":3,"passage":"Revelation 1:3","verse":"3Blessed is he that readeth, and they that hear the words of this prophecy, and keep those things which are written therein: for the time is at hand."}
{"index":{"_id":1004}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":4,"passage":"Revelation 1:4","verse":"4John to the seven churches which are in Asia: Grace be unto you, and peace, from him which is, and which was, and which is to come; and from the seven Spirits which are before his throne; "}
{"index":{"_id":1005}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":5"passage":"Revelation 1:5","verse":"5And from Jesus Christ, who is the faithful witness, and the first begotten of the dead, and the prince of the kings of the earth. Unto him that loved us, and washed us from our sins in his own blood, "}
{"index":{"_id":1006}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":6,"passage":"Revelation 1:6","verse":"6And hath made us kings and priests unto God and his Father; to him be glory and dominion for ever and ever. Amen"}
{"index":{"_id":1007}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":7,"passage":"Revelation 1:7","verse":"7Behold, he cometh with clouds; and every eye shall see him, and they also which pierced him: and all kindreds of the earth shall wail because of him. Even so, Amen."}
{"index":{"_id":1008}}
{"bookID":66,"book":"Revelation","chapterID":1,"verseID":8,"passage":"Revelation 1:8","verse":"8I am Alpha and Omega, the beginning and the ending, saith the Lord, which is, and which was, and which is to come, the Almighty."}

XML File – Image

revelation_20180804_0808AM.png

Output

addUsingBatch_20180803_0403PM

Review Data

Review Data based on JSON file

Command File

setlocal

set "_binfolder=C:\Downloads\Curl\v7.61.0\Windows\extract\curl-7.61.0-win64-mingw\bin"
set "_binapp=curl.exe"
set "_bin=%_binfolder%\%_binapp%"

set "_url=http://localhost:9200"

set "_esindex=bible"
set "_estype=biblekjv"
set "_esoperation=_search"
set "_espath=/%_esindex%/%_estype%/%_esoperation%"

set "_jsonfile=jsonfile\search_book_Revelation.json"
set "_jsonfileFQDN=%cd%\%_jsonfile%"

set "_contentType=Content-Type: application/json"

%_bin% -i -H "%_contentType%"  %_url%%_espath%?pretty --data-binary @%_jsonfileFQDN%

endlocal

XML File

{
  "query":
  {
    "bool":
    {
      "must":
      [
        { "match":
             { "book": "Revelation" }
        }
      ]
    }
  }
  , "sort" :
  {
      "bookID": {"order": "asc"}
    , "chapterID": {"order": "asc"}
    , "verseID": {"order": "asc"}
  }
} 

Output

fetchUsingQuery_20180803_0425PM

Source Code Control

Git

DanielAdeniji/elasticSearchcURLBatch

Link

Elasticsearch / Jörg Prante – JDBC importer

Background

On the ElasticSearch’s web site, one of the tools listed under the “Data Import & Export Validation page” is Jörg Prante’s – JDBC importer.

In this post, we will do a quick assessment.

 

Artifacts

Jörg Prante’s – JDBC importer is open source and it is available on GitHub.

The project’s URL is https://github.com/jprante/elasticsearch-jdbc.

The releases URL is https://github.com/jprante/elasticsearch-jdbc/releases.

As of today (2018-July-30), here is a screen shot.

The most recent release is 2.3.4.1.

The date is 2016-August-28th; a month short of 2 years ago.

Download

Downloaded the latest version.

 

Build

Built the maven project by invoking the gradlew.bat batch file.

 

Prepare

Environment Variable

JAVA_HOME

Prepared the command shell environment by setting the JAVA_HOME Environment variable.

Here we set JAVA_HOME to our installed Java JDK 1.8 folder.

 

set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181

 

Command

 

gradlew clean install 

Output

 

Build Deliverable

The build deliverable is placed in the build\distributions folder.

Based on our Source Code Release Number; the exact file name is elasticsearch-jdbc-2.3.4.1-2.3.4.1-dist.zip.

 

Copy to Lib Folder

Please extract the Lib folder from the *dist.zip into the Lib folder under the main folder.

 

Processing

Preparation

Prepare payload

We chose to package our script into two files.  The main initiator and the JSON configuration.

stackoverflow2010.json


{
    "jdbc" :
	{
        "url" : "jdbc:sqlserver://localhost:1433;databaseName=Stackoverflow2010;"
        , "user" : "stackoverflow"
        , "password" : "hIy8jA2lNl"
        , "sql" :  "select top 1 * from Stackoverflow2010.dbo.Users"
	}

    , "elasticsearch" :
		{
             "cluster" : "elasticsearch"
           , "host" : "localhost"
           , "port" : 9200
        }

    , "index" : "stackoveflow2010"
	, "type" : "user"
}

 

invokeStackoverflow2010.cmd


@echo off

setlocal

set DIR=%~dp0

set LIB=%DIR%..\lib\*

set BIN=%DIR%..\bin

set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181"

"%JAVA_HOME%\bin\java" -cp %LIB% -Dlog4j.configurationFile="%BIN%\log4j2.xml" "org.xbib.tools.Runner" "org.xbib.tools.JDBCImporter"	stackoverflow2010.json

endlocal

Invoke Command File

Invoked Command File.

Nothing shows up on the console.

But,  bad news on the elasticSearch log file.

Text


java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [5.6.0]
	at org.elasticsearch.transport.TcpTransport.ensureVersionCompatibility(TcpTransport.java:1462) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1409) ~[elasticsearch-6.3.2.jar:6.3.2]
	at 

Explanation

  1. Received message from unsupported version: [2.0.0] minimal compatible version is: [5.6.0]

 

Other Issues

Compile \ Build

Java JDK

Version

Unfortunately, when we tried using Java JDK 10x we ran into a ditch.

Prepare Environment

To prepare the environment:


set java_home=C:\Program Files\Java\jdk-10.0.2

Invoke gradlew

Command Issued is:


gradlew --help

Output

and the response received is ….

Image

build_java_jdk_v10_20180731_0812PAM

Textual

>set java_home=C:\Program Files\Java\jdk-10.0.2

>gradlew --help

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine java version from '10.0.2'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

>

Summary

Other issues aside, looks like we fell victim to Client’s API compatibility issue.

Our tool is a bit dated as it is stuck at v 2.0.0

Unfortunately, ElasticSearch requires at minimum Version 5.6.0

ElasticSearch – Error – “index read-only / allow delete (api)”

Background

Ran into an error with ElasticSearch.

I was trying to add a new record.

The error reads

index read-only / allow delete (api) ( Elastic Search – Error )

 

Trouble Shooting

Elastic\Elasticsearch\logs\elasticsearch.log

Reviewed elasticsearch.log and it is reading…

 

Text


rerouting shards: [high disk watermark exceeded on one or more nodes]

high disk watermark [90%] exceeded on
[C:\ProgramData\Elastic\Elasticsearch\data\nodes\0]
free: 12.1gb[5.1%], shards will be relocated away from this node

Image

Explanation

  1. Frequency
    • Every 30 seconds, new log entries are been posted.
  2. Alert
    • high disk watermark [90%] exceeded on
  3. Causation
    • free: 12.1gb[5.1%]
  4. Response
    • Shards will be relocated away from this node

 

Storage Utilization

Operating System ( OS )

Computer Management

Image

Explanation
  1. On our data drive, C:, here is what how utilization looks like
    • Reading
      • Capacity :- 235 GB
      • Free Space :- 12 GB
      • Free % :- 5%

 

Windows Explorer

Image

Explanation
  1. On our data drive, C:, here is what how utilization looks like
    • Reading
      • Capacity :- 235 GB
      • Free Space :- 12.1 GB
    • Deducted
      • Free Ratio :- 0.051
      • Percentile :- 5.1%

 

ElasticSearch

ElasticSearch – Metadata

_nodes/_local/stats/fs
Query
  1. Method :- GET
  2. Payload :- http://localhost:9200/_nodes/_local/stats/fs
Image

 

Explanation
  1. Metrics
    • Availed
      • “total_in_bytes”: 252806361088
      • “free_in_bytes”: 14050291712
      • “available_in_bytes”: 14050291712
    • Calculated
      • Equation :- available_in_bytes / total_in_bytes
      • Number :- 14050291712 / 252806361088
      • Ratio :- 0.0555772871043747
      • % :- 5%

 

Settings

Setting – ElasticSearch

_cluster/settings

Query
  1. Method :- GET
  2. Payload :-
    http://localhost:9200/_cluster/settings?include_defaults=true
Output
Image

Textual
  1. disk
    • threshold_enabled :- true
    • Watermark
      • low :- 85%
      • flood Stage :- 95%
      • High :- 90%

 

What does the numbers mean?

 

Item Definition Implication
cluster.routing.allocation.disk.threshold_enabled Defaults to true. Set to false to disable the disk allocation decider. Determine whether disk metrics are checked
cluster.routing.allocation.disk.watermark.low Controls the low watermark for disk usage. It defaults to 85%, meaning that Elasticsearch will not allocate shards to nodes that have more than 85% disk used. It can also be set to an absolute byte value (like 500mb) to prevent Elasticsearch from allocating shards if less than the specified amount of space is available. This setting has no effect on the primary shards of newly-created indices or, specifically, any shards that have never previously been allocated. New Shards will not be allocated
cluster.routing.allocation.disk.watermark.high Controls the high watermark. It defaults to 90%, meaning that Elasticsearch will attempt to relocate shards away from a node whose disk usage is above 90%. It can also be set to an absolute byte value (similarly to the low watermark) to relocate shards away from a node if it has less than the specified amount of free space. This setting affects the allocation of all shards, whether previously allocated or not. Attempts will be made to move Current Shards
cluster.routing.allocation.disk.watermark.flood_stage Controls the flood stage watermark. It defaults to 95%, meaning that Elasticsearch enforces a read-only index block (index.blocks.read_only_allow_delete) on every index that has one or more shards allocated on the node that has at least one disk exceeding the flood stage. This is a last resort to prevent nodes from running out of disk space. The index block must be released manually once there is enough disk space available to allow indexing operations to continue. Index will marked readonly

Remediation

Outline

  1. File System
    • Make room in File-system

Options Explored

File System

Use OS Tools to make room in File System

Linux

In Linux, use du ( Disk used), df ( Disk free), rm ( remove files ), mv ( move files, and ls ( List files).

 

ElasticSearch

Review default or current disk utilization alert settings and judge whether ripe for changes.

Configuration Setting

C:\ProgramData\Elastic\Elasticsearch\config
Syntax
cluster.routing.allocation.disk.watermark.low: [ratio]
cluster.routing.allocation.disk.watermark.high: [ratio]
cluster.routing.allocation.disk.watermark.flood_stage: [ratio]
Sample
cluster.routing.allocation.disk.watermark.low: 0.99
cluster.routing.allocation.disk.watermark.high: 0.99
cluster.routing.allocation.disk.watermark.flood_stage: 0.99

Index

Index Setting – Read
Syntax
 
http://localhost:9200/index/_settings
Sample
http://localhost:9200/bible/_settings

 

Image

 

 

Index Setting – Set
Syntax
 
{
  
    "index": 
    {
        "blocks": 
        {
              "read_only_allow_delete": "false"
            , "read_only": "false"          
        }
    }
    
}

 

Sample

 

 
{
 
    "index": 
    {
        "blocks": 
        {
              "read_only_allow_delete": "false"
            , "read_only": "false"          
        }
    }
    
}
Image
  1. Method :- PUT
  2. Payload :-
     http://localhost:9200/bible/_settings

 

 

Summary

Quite a bit to go over.

In summary, monitor your disk usage.

If need to, adjust ElasticSearch to scale back by modifying the percentile or hard numbers it responds and starts making accommodation on.

 

References

  1. elastic.co
    • Cluster
      • Disk-based Shard Allocation
        • Elasticsearch Reference [6.0] » Modules » Cluster » Disk-based Shard Allocation
          Link
    • Indices
      • Read
        • Indices – Setting – Read
          Link
      • Update
        • Update Indices Settings
          Link
      • Indices – API – Get Settings
        • Elasticsearch Reference [6.3] » Indices APIs » Get Settings
          « Update Indices Settings
          Link
    • Node
      • Node Stats
        • Elasticsearch Reference [6.3] » Cluster APIs » Nodes Stats « Cluster Update Settings
          Link
    • Issues
      • How to show cluster settings “default” values ? #18645
        Link
      • cluster.routing.allocation.disk.watermark.flood_stage not updateable through cluster api #27406
        Link
  2. Stack-overflow
    • Disk Space Usage
      • ElasticSearch find disk space usage
        Link
    • High disk watermark exceeded even when there is not much data in my index
      Link