SQL Server – v2017 – Install – “Oracle JRE 7 Update 51 (64-bit) is required for Polybase”

Background

Installing SQL Server v2017 and ran into an error that I have seen before.

Error

Error Image

OracleJRE7Update51_x64.2018109.1111AM.PNG

Error Textual

Oracle JRE 7 Update 51 (64-bit) or higher is required for Polybase

Tackled Before

History

Already tackled before here:

SQL Server (v2016) – Installation – Blocked – “Oracle JRE 7 Update 51 (64-bit) or higher is required for Polybase”
here

History

The fix applied was to install Java JRE downloaded from here:

https://www.oracle.com/technetwork/java/javase/downloads/index.html
Link

But, unfortunately even though install applied no help this time.

Trouble Shooting

Let us dig a bit deeper.

SysInternals

Process Monitor

Image

sysInternals.processMonitor.20181009.1010AM.PNG

Explanation

  1. Identified component used by SQL Server Setup
    • Component is ScenarioEngine
  2. ScenarioEngine
    • Log
      • C:\Program Files\Microsoft SQL Server\140\Setup Bootstrap\Log\20181009_092126\Detail.txt

Tail

Tail For Win32

Download

Downloaded Tail For Win32 from here:

Tail for Win32
Link

Usage

Launched “Tail For Win32” and loaded the identified SQL Server Log file ( Detail.txt )

Results
Image
TailforWin32_Usage_20181009_1151AM
Textual

13) 2018-10-09 10:13:06 Slp: Init rule target object: Microsoft.SqlServer.Configuration.Polybase.Polybase_IsMinJavaVersionInstalledFacet
(13) 2018-10-09 10:13:06 SQLPolyBase: Could not find registry setting HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment\CurrentVersion.
(13) 2018-10-09 10:13:06 SQLPolyBase: Minimum version expected: 1.7.51. Java not found.
(13) 2018-10-09 10:13:06 SQLPolyBase: Rule 'Polybase_IsMinJavaVersionInstalled' detection result: IsMinJavaVersionInstalled= False
(13) 2018-10-09 10:13:06 Slp: Evaluating rule        : Polybase_IsMinJavaVersionInstalled
(13) 2018-10-09 10:13:06 Slp: Rule running on machine:
(13) 2018-10-09 10:13:06 Slp: Rule evaluation done   : Failed
(13) 2018-10-09 10:13:06 Slp: Rule evaluation message: This computer does not have the Oracle Java SE Runtime Environment Version 7 Update 51 (64-bit) or higher installed. The Oracle Java SE Runtime Environment is software provided by a third party. Microsoft grants you no rights for such third-party software. You are responsible for and must separately locate, read and accept applicable third-party license terms. To continue, download the Oracle SE Java Runtime Environment from https://go.microsoft.com/fwlink/?LinkId=526030.
(13) 2018-10-09 10:13:06 Slp: Send result to channel : RulesEngineNotificationChannel

Textual
  1. QLPolyBase: Could not find registry setting HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment\CurrentVersion.
    (13) 2018-10-09 10:13:06 SQLPolyBase: Minimum version expected: 1.7.51. Java not found.

Registry

regedit

Images

Regedit – Image – Before
Image

registry_20181009_1038AM.PNG

Textual
  1. Computer\HKEY_LOCAL_MACHINE\Software\JavaSoft
    • Java Plug-in
    • Java Update
    • Java Web Start
    • Java Web Start Caps
    • JRE
Explanation
  1. HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment\CurrentVersion.
    • Java Runtime Environment
      • Missing

Remediation

Outline

Please download Java JRE 8 and install it.

Download

Please download artifacts from here :-

Java SE Runtime Environment 8 Downloads
https://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html
Link

Artifacts

Image

Java SE Runtime Environment 8u181

download_JavaSERuntimeEnvironment_20181009.1221PM.PNG

download_20181009.1026AM.PNG
Explanation
  1. Chose to download Windows X64

 

Install

Images

Destination Folder

download_DestinationFolder_20181009.1031AM.PNG

Installing
Installing -01

install_20181009.1032AM.PNG

java Setup – Complete

install_JavaSetupComplete_20181009.1033AM.PNG

Change in License Terms

download_JavaSERuntimeEnvironment_Roadmap_20181009.1030AM.PNG

Registry

Using regedit, please review the Java’s registry structure.

regedit

Images
Regedit – Image – After
Image

registry_java_jre_1Dot8_20181009_1243PM.PNG

Explanation
  1. HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment\CurrentVersion is now present

 

Summary

It appears that Oracle has changed the folder structure for Java in the Windows Registry.

Specifically what used to be

HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment\CurrentVersion.

is now

HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\JRE\CurrentVersion.

Please be sure to have Java JRE v1.8 ( Java 8) installed for SQL Server v2016\v2017 Polybase.

DBeaver – Database Connection Configuration – IBM – DB2 / UDB JDBC Driver

Background

As always coming back to DBeaver.

This time to connect to a UDB Instance running on Linux.

BTW, UDB means Universal Database.

UDB JDBC Driver

Artifacts

DB2 JDBC Driver Versions and Downloads

The URL for DB2 JDBC Drivers is Link.

Here is a current snapshot.

artifacts_db2drivers_20180816_1236PM

v11.1 FP0 ( GA )

We chose the v11 FP0 ( GA ) which is a baseline install; specifically 4.21.29.

Others might choose v11 M3 FP3; which includes the latest patches.

The URL is Link.

Here is a screen shot:

artifacts_db2drivers_v11_20180816_0110PM

IBM Data Server Driver for JDBC and SQLJ ( JCC Driver )

As all we need is the JDBC Driver, we chose “IBM Data Server Driver for JDBC and SQLJ (JCC Driver)“.

One needs membership account to actually download, so please login in with one if you have already registered.  Else, please create one, as it is free.

Download

In short, we downloaded “IBM Data Server Driver for JDBC and SQLJ (JCC Driver)“.

Extracted

Image

zip_extracted_20180816_0216PM.PNG

Textual

  1. db2jcc.jar
    • JDBC v3.0
  2. db2jcc4.jar
    • JDBC 4.0

Preparation

Network

Port Number

Let us determine the Network Port Number that our DB/2 Instance is listening on.

db2 get dbm cfg

Syntax


db2 get dbm cfg

Sample


db2 get dbm cfg | grep SVCE

Output

db2getdbmcfg_20180816_0230PM.PNG

Databases

Outline

Let us determine the list of databases that are available on our UDB Instance.

db2 list database directory

Syntax


 db2 list database directory

Sample


 db2 list database directory | grep "Database name"

Output

listDatabases_20180816_0242PM.PNG

 

DBeaver

Database

Connections

Steps

Select new connection type
Image

SelectNewConnectionType_20180816_0644AM.PNG

Textual
  1. DB2
    • DB2 LUW
Driver settings – Download driver files
Image

downloadDriverFiles_20180816_0647AM.PNG

Textual
  1. Click on the “Add JARs” button
Driver settings – Edit Driver – Add File – Post
Image

EditDriver_AddFile_After_20180816_0647AM.PNG

Textual
  1. JAR file “db2jcc4.jar” added
Create new connection – DB2 Connection Settings – Tab – General
Image

EditDriver_Tab_Database_20180816_0709AM ( Edited ).PNG

Textual
  1. Host
  2. Port
  3. Database
  4. username
  5. Password
Create new connection – DB2 Connection Settings – finish connection creation
Image

finishConnection_20180816_0719AM ( Edited ).PNG

 

References

  1. IBM
    • Home > DB2 for Linux UNIX and Windows 9.5.0 > Database administration > Administrative interfaces > Administrative SQL routines and views > Supported routines and views > Snapshot routines and views
      • APPLICATIONS administrative view – Retrieve connected database application information
        Link

 

jruby – Get Version Information

Background

As part of a troubleshooting session that I will be undertaken I wanted to start getting familiar with what jruby has in terms of API for discovering the Java Version that it is running under.

Code

Ruby

Version.rb


require 'date'

require 'java'

require 'rbconfig'
include RbConfig

require 'io/console'

java_import java.lang.System

class Version

    # ruby class variables are prefixed with '@@' .
    @@javaRTVersion = "";
    @@rubyPlatform = "";
    @@hostOS = "";
    @@acquired = false;

    #Constructor
    def initialize()

    end

    def self.acquire

        @@javaRTVersion = System.getProperties["java.runtime.version"]

        @@rubyPlatform = RUBY_PLATFORM

        @@hostOS = CONFIG['host_os']

    end 

    # hostOS
    def self.hostOS

        if (@@acquired == false)

            self.acquire()

            @@acquired = true

        end

        return @@hostOS

    end 

    # java runtime version
    def self.javaRTVersion

        if (@@acquired == false)

            self.acquire()

            @@acquired = true

        end

        return @@javaRTVersion

    end     

    # the getter is defined to return the class variable when called
    def self.rubyPlatform

        if (@@acquired == false)

            self.acquire()

            @@acquired = true

        end

        return @@rubyPlatform

    end 

end

def waitForKeyPress

    puts
    puts

    puts "Press enter to continue"

    STDIN.getch()

end 

objV = Version.new()

# Access it's attributes.
puts
puts "hostOS is " + Version.hostOS
puts "javaRTVersion is " + Version.javaRTVersion
puts "rubyPlatform is " + Version.rubyPlatform

waitForKeyPress()

Command File

Objective

Ran again JDK 1.8, 1.9 ( v9 ) and 1.10 ( v10 ).

initiate.cmd


setlocal

REM SET JAVA_HOME to Version 1.8
rem set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181"
rem set "JAVA_HOME=C:\Program Files\Java\jdk-9.0.4"
set "JAVA_HOME=C:\Program Files\Java\jdk-10.0.2"

set "JRUBY_BIN=C:\Downloads\jruby\v9.2\zip\jruby-dist-9.2.0.0-bin\jruby-9.2.0.0\bin"

set "_app=Version.rb"

%JRUBY_BIN%\jruby.bat "%_app%"

endlocal

Output

JDK 1.8

javaVersion_run_1DOT8_20180816_0435AM

JDK 1.9

Image

javaVersion_run_1DOT9_20180816_0439AM

Textual


WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.runtime.encoding.EncodingService to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.jruby.runtime.encoding.EncodingService
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

hostOS is mswin32
javaRTVersion is 9.0.4+11
rubyPlatform is java

JDK 1.10

Image

javaVersion_run_1DOT10_20180816_0445AM

Textual


WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.runtime.encoding.EncodingService to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.jruby.runtime.encoding.EncodingService
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

hostOS is mswin32
javaRTVersion is 10.0.2+13<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;">&#65279;</span>
rubyPlatform is java

jruby – Sample App – Classes – blackBook

Background

Now that we have jruby installation out of the way let us create a simple app that creates a class.

BTW, our jruby Installation post is here.

 

Code

blackbook.rb


###############################################
#Foundational Sample Script
#https://gist.github.com/dougal/24876
###############################################
require 'date'

class Blackbook

    #Declare Accessor Methods   
    attr_accessor :name, :relationship, :dateAdded

    
    # ruby class variables are prefixed with '@@' .
    # set number of entries
    @@iNumberofEntries = 0;
    
    #Constructor
    def initialize(name, relationship, dateAdded)
    
        @name = name
        @relationship = relationship
        @dateAdded = dateAdded
        
        #Increment number of entries
        @@iNumberofEntries = @@iNumberofEntries + 1
        
    end
  
    # Class Method numberofEntries
    # Returns Class Variable @@iNumberofEntries
    def self.numberofEntries
    
        return @@iNumberofEntries
        
    end  
    
end

# Get Current Date
dtNow = Time.now

# Date Formats
dateFormat="%b %d %Y"
dateFormatYYYYMMDD="YYYYMMDD"

# Create the new instance ( Prince Charles)
dob = "19481114"
dobCharles = Date.parse(dob);
entryCharles = Blackbook.new('Charles', 'Prince', dobCharles)

# Access it's attributes.
puts "Entry Name is " + entryCharles.name
puts "Relationship is " + entryCharles.relationship
puts "Date Added is " + entryCharles.dateAdded.strftime(dateFormat);
puts ""

# Create the new instance ( Lady Diana )
dob = "19610701"
dobDiana = Date.parse(dob);
entryDiana = Blackbook.new('Diana', 'Princess', dobDiana)

# Access it's attributes.
puts "Entry Name is " + entryDiana.name
puts "Relationship is " + entryDiana.relationship
puts "Date Added is " + entryDiana.dateAdded.strftime(dateFormat);
puts ""

#Get Number of entries
strNumberofEntries = Blackbook.numberofEntries.to_s
puts "Number of entries in black book " + strNumberofEntries

invoke.cmd


setlocal

REM SET JAVA_HOME to Version 1.8
set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181"
rem set "JAVA_HOME=C:\Program Files\Java\jdk-9.0.4"
REM set "JAVA_HOME=C:\Program Files\Java\jdk-10.0.2"

set "JRUBY_BIN=C:\Downloads\jruby\v9.2\zip\jruby-dist-9.2.0.0-bin\jruby-9.2.0.0\bin"

set "_app=blackbook.rb"

%JRUBY_BIN%\jruby.bat "%_app%"

endlocal

output

jruby – Installation and Hello World App

Background

Because of a side trek down the ElasticSearch road, specifically Logstash, wanted to develop a more congruent understanding of jruby.

Here we go.

jruby

per-requisite

Java JDK

Java JDK v1.8

Please download and ensure that Java’s JDK version 1.8 is downloaded and installed.

BTW, here is the link to Java JDK v1.8.

 

Artifact

jruby is available here.

Here is a current screen shot.

Image

 

Tabulate

Here is a quick break down:

Package OS Platform Packaging Installation
JRuby 9.2.0.0 Binary .tar.gz Linux Binary Unzip/Extract
JRuby 9.2.0.0 Binary .zip Windows Binary Unzip/Extract
JRuby 9.2.0.0 Windows Executable Windows Install4j Wizard  – x32 bit Run Installer
JRuby 9.2.0.0 Windows Executable (x64) Windows Install4j Wizard – x64 bit Run Installer
JRuby 9.2.0.0 Source .zip Multi-Platform Source Code Extract
JRuby 9.2.0.0 Complete .jar Multi-Platform Jar File Copy
JRuby-jars 9.2.0.0 .gem

 

Explanation

  1. Windows Executable
    • Most MS Windows users should go with the Windows Executable
    • Depending on OS bitness, please go with 32 or 64 bit
  2. Binary
    • Infrequent or Evaluation users should go with Binary Zip
  3. Source
    • Developers should go with the Source Code
  4. Complete Jar
    • OEM

 

Choice

We went the “Binary Zip” Route.

Download

Downloaded “jruby-dist-9.2.0.0-bin.zip

 

Extract

Using 7-Zip extracted the zip file.

Here is what our extracted folders look like.

 

Coding

Let us write a very simple script.

Hello World

Code

Ruby Source Code


# Beginner Ruby App
# https://en.wikibooks.org/wiki/Ruby_Programming/Hello_world
puts 'Hello, world!'

# Query Environment Variables
userDomain=ENV["USERDOMAIN"]
username=ENV["USERNAME"]

# Concatenate User Domain and Username
yourName=userDomain + '\\' + username

# Display Output
puts 'Your Full Username is ' + yourName



 

Invoke

Command File to initiate jruby script.


setlocal

REM SET JAVA_HOME to Version 1.8
set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181"
rem set "JAVA_HOME=C:\Program Files\Java\jdk-9.0.4"
rem set "JAVA_HOME=C:\Program Files\Java\jdk-10.0.2"

set "JRUBY_BIN=C:\Downloads\jruby\v9.2\zip\jruby-dist-9.2.0.0-bin\jruby-9.2.0.0\bin"

set "_app=helloWorld.rb"

%JRUBY_BIN%\jruby.bat "%_app%"

endlocal


 

Output

Here is the output of invoking the script.

On MS Windows, Profiling Java Apps Memory Utilization Using VMMap

Background

In our last post, we covered the usage of a few Oracle standard tools to profile Java Applications.

BTW the referenced post is available here.

 

SysInternals

VMMap

In this post, we will briefly cover using SysInternal’s VMMap GUI to track down memory utilization.

 

Artifact

VMMap is available from here.

The current version is v3.21 and it was released a couple of years ago; July 20, 2015 to be exact.

Download & Extract

Download and Extract it; no need for installation.

Usage

ElasticSearch

Select or Launch Process

 

Metrics

 

WebSphere

Select or Launch Process

 

Metric

Profiling Java Apps Using Bundled Tools

Background

Lately I have found myself playing with a lot of Applications that are running inside Java’s JVM.

The responsible next step is to start profiling their consumption patterns.

 

Referenced Work

These days most of my work are begun from a Google Search.

This work started from Karun Subramanian’s well thought effort.

Here are his blog posts:

  1. 5 not so easy ways to monitor the Heap Usage of your Java Application
    Link

 

Constraints

The tools that we will be evaluating have the following constraints:

  1. They should be run under the same account as the Java Processes that will be monitored

 

Tools

In this exercise we will look at our tools from the prism of Interface; Command Line  Interface ( CLI ) or Graphical User Interface.

Outline

  1. Interface
    • Command Line Tools ( CLI )
      • jstat
      • jmap
    • GUI
      • visualvm
      • jconsole

Preparation

jps

Outline

jps is analogous to the ps command in Linux.

While ps list all processes; jps lists all running Java Processes.

 

Syntax


jps

 

Sample


jps | find /I /V "Jps"

 

Output

 

Explanation

  1. The first column is the LVMDID ( process ID)
  2. The second column is the Process Name

 

Command Line Interface ( CLI )

jstat

Artifacts

Upon installing Java’s JDK, jstat is available in the JDK’s HOME bin folder.

Code

syntax

jstat -gc [process-id]

Sample

jstat -gc 13800

Output

jstat_20180811_0913AM

Explanation
Column Meaning Sample
EC Eden Capacity (KB) 69952
EU Eden Space Utilization (KB) 19112.8
OU Old Space Utilization (KB) 120747.8
OC Old Space Capacity (KB) 174784.0

jmap

Artifacts

jmap is available in the JDK’s HOME bin folder.

Code

syntax

jmap -heap [process-id]

Sample

jmap -heap 13800

Output

es.20180811.0943AM

Explanation
Attribute Meaning Sample
MaxHeapSize Maximum Heap Size 256 MB
NewSize Eden Space Utilization 85 MB
MaxNewSize Eden Space Capacity 85 MB
OldSize Old Space Utilization 170 MB

jhsdb

Artifacts

From Java JDK v9, jhsdb is available in the JDK’s HOME bin folder.

Code

syntax

jhsdb jmap -heap --pid [process-id]

Sample

jhsdb jmap --heap --pid 1960

Output

Explanation
Attribute Meaning Sample
G1
Regions 2020
Capacity 2020 MB ( 2 GB )
Used 98 MB
Free 1921 MB ( 1.92 GB)
Used % 4.90%
G1 – Eden Space
Regions 62
Capacity 74 MB
Used 62 MB
Free 12 MB
Used % 83.78%
G1 – Survivor Space
Regions 4
Capacity 4 MB
Used 4 MB
Free 0
Used % 100 %
G1 Old Generation
Regions 34
Capacity 50 MB
Used 32 MB
Free 17 MB
Used % 65.79 %

 

Graphical User Interface ( GUI )

visualVM

Artifacts

Java Version
Java Version v1.8 and Before

Up to Java v1.8 Visual M was bundled with Java JDK.

Once Java’s JDK is installed, please access visualvm.exe from the JDK’s HOME bin folder.

Java Version  9 and Above

Please download VisualVM from here.

Extract the zip file and run visualvm.exe from the bin folder.

Visual

Tab – Overview
Image

es_Tab_Overview_20180811_1004AM.PNG

Textual
  1. JDK
    1. Bitness :- x64
    2. v1.8
  2. JVM Arguments
    • -XX:+UseConcMarkSweepGC
      • Garbage Collection Algorithm
      • Web Links
    • HeapDumpPath = data
    • ErrorFile :- logs/hs_err_pid%p.log
Tab – Monitor
Image

es_Tab_Monitor_20180811_1005AM.PNG

Textual
  1. Heap
    • Max :- 256 MB
    • In use :-  120 MB

 

Tab – Threads
Image

es_Tab_Threads_20180811_1004AM

 

jconsole

Artifacts

JDK is accessible from the bin folder of Java’s JDK HOME folder.

Visual

Tab – Overview
Image

jconsole_Tab_Overview_20180811_1121AM

Textual
  1. Heap Memory Usage :- 137 MB
  2. Live Threads :- 60
  3. Classes :- 16,000
Tab – Memory
Image

jconsole_Tab_Memory_20180811_1112AM

Textual
  1. Used :- 156 MB
  2. Committed :- 256 MB

 

Tab – Threads
Image

jconsole_Tab_Threads_20180811_1128AM

Textual
  1. Live Threads :- 60
  2. Peak :- 63

 

 

References

  • Oracle
    • Home / Java / Oracle JDK 9 Documentation
      • Java Platform, Standard Edition Tools Reference
    • Java
      • jps
        • jps – Java Virtual Machine Process Status Tool
          Link
  • Karun Subramanian
    • 5 not so easy ways to monitor the Heap Usage of your Java Application
      Link
  • Dustin Marx
    • Dzone
      • jhsdb: A New Tool for JDK 9
        Link