On MS Windows, Profiling Java Apps Memory Utilization Using VMMap

Background

In our last post, we covered the usage of a few Oracle standard tools to profile Java Applications.

BTW the referenced post is available here.

 

SysInternals

VMMap

In this post, we will briefly cover using SysInternal’s VMMap GUI to track down memory utilization.

 

Artifact

VMMap is available from here.

The current version is v3.21 and it was released a couple of years ago; July 20, 2015 to be exact.

Download & Extract

Download and Extract it; no need for installation.

Usage

ElasticSearch

Select or Launch Process

 

Metrics

 

WebSphere

Select or Launch Process

 

Metric

Java – Database – Get DB Metadata

Background

Wanted to quickly review our DataSource Connection.

Code


   
    private String getClassName(Object obj)
    {

        Class  objClass;
        String strClassName;    
    
        objClass = obj.getClass();
        
        strClassName = objClass.getName();
         
        return (strClassName);
                    
    }


    private void fetchDBMetadata
    (
        PrintWriter out
    )
    {
        
        String strURL;
        String[] arrayOfToken  = null;
        
        int iLen = 0;
        int i =0;
        String strData = null;      
        
        try 
        {
            
            /*
                Create InitialContext
            */          
            objCTX = new InitialContext();

            Object obj = null;
       
            /*
                Lookup specific context
            */                 
            obj = objCTX.lookup
                    (
                        strDBID
                    );

            if (obj instanceof DataSource)
            {               
                objDS = (DataSource) obj;
            }
            else
            {
                
                strClassName = getClassName(obj);
                 
                strLog = "Unexpected Object type returned from context "
                            + "ID " + strDBID
                            + " Object Type " 
                            + strClassName
                            ;
                            
                throw new Exception(strLog);
                
            }     
            
            try
            {

                objDBConn = objDS.getConnection();
                
                objDBMetadata = objDBConn.getMetaData();
                
                strURL = objDBMetadata.getURL();
                
                out.println("URL :- " + strURL + CHAR_CRLF);

                out.println(CHAR_CRLF + CHAR_CRLF);             
                
                int iProp = 0;
                
                String strURLProp;
                
                iProp = strURL.indexOf(";");
                
                //out.println("iProp :- " + iProp + CHAR_CRLF);             
                
                strURLProp = strURL.substring(iProp + 1);
                
                //out.println("URLProp :- " + strURLProp + CHAR_CRLF);
                
                if (strURLProp != null)
                {
                    
                    // get token    
                    arrayOfToken = strURLProp.split(";");
                    
                    // get token's length                       
                    if (arrayOfToken == null)
                    {
                        iLen = -1;
                    }
                    else
                    {
                        iLen = arrayOfToken.length;                 
                    }   
                    
                    strLog = "Number of Properties " 
                                + iLen
                                + CHAR_CRLF
                                ;
                    
                    out.println(strLog);
                    
                    out.println(CHAR_CRLF);                                 
                
                    for (i = 0; i < iLen; i++)
                    {
                        strData = arrayOfToken[i];
                        
                        if (strData != null)
                        {
                            
                            strLog = CHAR_TAB
                                        + i 
                                        + ")"
                                        + " " 
                                        + strData
                                        ;
                            
                            out.println(strLog + CHAR_CRLF);                    
                            
                        }   
                        
                    }   
                        
                } // for i
                
            }
            catch (Exception ex)
            {
                          
                out.println("Exception in fetchMetadata ");
                    
                ex.printStackTrace();
                
                out.println(ex.getMessage());
            
            }
        
      
            
        } 
        catch (SQLException ex) 
        {
            
           System.out.println(ex.getMessage());         
           
           ex.printStackTrace();
                    
           out.println(ex.getMessage());
           
        }
        catch (NamingException ex) 
        {
           System.out.println(ex.getMessage());         
           
           ex.printStackTrace();
           
           out.println(ex.getMessage());
           
        }
        catch (Exception ex) 
        {
           System.out.println(ex.getMessage());         
           
           ex.printStackTrace();
                    
           out.println(ex.getMessage());
           
        }       
        finally 
        {
            
           if (objDBConn != null)
           {
                try
                {
                    objDBConn.close();
                }
                catch (SQLException ex) 
                {
							
					System.out.println(ex.getMessage());         

                    ex.printStackTrace();
					
					out.println(ex.getMessage());

                } 
                
           }
           
       } // finally
   
    } //fetchDBMetaData

Output

Output – StackOverflow2010

WebSphere/Liberty :- Reading data file

Background

Having to read a data file and wanted to see the options available.

Let us go over some of those options.

Option

  1. API :- File / File Reader
    • Location
      • File System
        • Sample
          • /home/
          • /WEB-INF/datafile/dbo.users.txt
  2. API :- classLoader.getResourceAsStream
    • Location
      • File System
        • relative to class path
        • Sample
          • \datafile\dbo.users.txt

Packaging

Using https://extract.me/, an online WAR file extractor we can visualize our targeted layout.

Image

rearfileDOTwar_20180724__20180724__0827PM

Explanation

  1. WEB-INF
    • Artifacts
      • Read Method
        • API :- File
      • Sample
        • datafile
          • dbo.users.txt
    • Java Classes
      • Read Method
        • API :- Class Loader
      • Sample
        • datafile
          • dbo.users.txt

Configuration

The location of our file can be saved in various locations.

web.xml

In our case we will save in the web.xml file.

Image

webDOTxml_20180724_0402PM.png

Source

Read web.xml and get servlet parameters


 private void getDataFilePath
(
    PrintWriter out
)
{

    ServletConfig config;

    config = getServletConfig();

    String initparamValue;      

    Enumeration enumeration = null;

    String parm;

    enumeration = getInitParameterNames();

    while (enumeration.hasMoreElements())
    {
        parm = enumeration.nextElement();

        strLog = parm
                    + " "
                    + getInitParameter(parm)
                    + ""
                    ;

        out.println(strLog);

    }

    //datafileOS
    initparamValue = config.getInitParameter
                        ("datafileOS");

    strDatafileOS = initparamValue;

    //datafileResource
    initparamValue = config.getInitParameter
                        ("datafileResource");

    strDatafileResource
        = config.getServletContext().getRealPath(initparamValue);

    //datafileOS
    initparamValue = config.getInitParameter
                        ("datafileClass");

    strDatafileClass = initparamValue;

}   

Reading Text File through File API


private BufferedReader getBufferedReaderUsingTraditionalMethod
(
      String strDatafile
    , PrintWriter out
)
{

    BufferedReader bufferedReaderLocal = null;

    try
    {   

        file = new File(strDatafile);

        if (file != null)
        {

            fileReader = new FileReader(file);

            bufferedReaderLocal = new BufferedReader(fileReader);

        }
    }
    catch (Exception ex)
    {

        ex.printStackTrace();

        System.out.println(ex.getMessage());

    }

    return(bufferedReaderLocal);

}

Reading as Class File


private BufferedReader getBufferedReaderUsingClassLoader
(
	  String strDatafile
	, PrintWriter out
)
{

	BufferedReader bufferedReaderLocal = null;

	try
	{

		classLoader = getClass().getClassLoader();

		objInputStream = classLoader.getResourceAsStream(strDatafile);

		if (objInputStream == null)
		{

			return (bufferedReaderLocal);

		}
		else
		{	

			bufferedReaderLocal = new BufferedReader
				(
					new InputStreamReader(objInputStream)
				);

		}

	}
	catch (Exception ex)
	{

		ex.printStackTrace();

		System.out.println(ex.getMessage());

		out.println(ex.getMessage());

	}

	return (bufferedReaderLocal);

} 	

Read data file


private void readDatafile(PrintWriter out )
{

    BufferedReader bufferedReader = null;

    StringBuffer stringBuffer = null;
    String line;
    long lNumberofRecordsRead = 0;

    Collection objListDisplayName = new ArrayList();    

    try
    {

        strDatafile = strDatafileOS;

        // Read OS File
        bufferedReader = getBufferedReaderUsingTraditionalMethod
                         (
                              strDatafile
                            , out
                         );

        // If unable to read OS File
        // read data file from web.inf/
        if (bufferedReader == null)
        {

            strDatafile = strDatafileResource;

            bufferedReader = getBufferedReaderUsingTraditionalMethod
                                (
                                      strDatafile
                                    , out
                                );

        }

        //servlet class file
        if (bufferedReader == null)
        {

            strDatafile = strDatafileClass;

            bufferedReader = getBufferedReaderUsingClassLoader
                                (
                                      strDatafile
                                    , out
                                );

        }           

        if (bufferedReader == null)
        {
            return;
        }

        stringBuffer = new StringBuffer();

        while ((line = bufferedReader.readLine()) != null)
        {

            objListDisplayName.add(line);

            lNumberofRecordsRead = lNumberofRecordsRead + 1;

        }

        if (fileReader != null)
        {
            fileReader.close();
        }

        if ( classLoader != null)
        {
            classLoader = null;
        }

        if (objInputStream != null)
        {
            objInputStream = null;
        }

        if (bDebug)
        {
            strData = lNumberofRecordsRead
                        + " record(s) read";

            out.println(strData);

        }

    }
    catch (IOException ex)
    {

        ex.printStackTrace();

        System.out.println(ex.getMessage());

        out.println(ex.getMessage());

    }

}

DisplayReadFile

private void displayReadFile(PrintWriter out )
{

    long i = 0;

    iterator = objListDisplayName.iterator();

    String strData;

    strData = "";

    out.println(strData);

    // while loop
    while (iterator.hasNext())
    {

        i = i + 1;

        strData = "
                    + iterator.next()
                    + ""
                    ;

        out.println(strData);

        if (i == 6)
        {
            break;
        }

    }

}

Source Code

DanielAdeniji/javaReadFileText
Link

Dedicated

Dedicated to Ivan S. & Joe L.

I was really struggling with this.

But, these brothers wrestled with me and made sure I did not get it wrong.

And, so here is gratitude for brothers who take time and think we deserve better; especially when we clearly don’t.

Ivan S.

ivanS_20180724_0456PM

WebSphere Liberty :- Simple Servlet ( DB )

 

Preparation

WebSphere / Liberty

Java Database Connectivity ( JDBC )

Download

Download JDBC Driver from your Database Software provider.

For us Microsoft, and we get the JDBC Driver from here :-

Docs / SQL / Connect your client to SQL / JDBC
Download Microsoft JDBC Driver for SQL Server
Link

Configuration

Server.xml

Data Source

Outline
  1. Data Source
    • jndi
      • Unique name within each application
      • Will be referred to by the Application
      • Sample :- jdbc/inform
    • jdbcDriver
      • libraryRef
        • Sample :- MSJDBCLib
    • properties.microsoft.sqlserver
      • serverName=”localhost”
      • portNumber=”1433″
      • databaseName=”StackOverflow2010″
      • user=”stackoverflow”
      • password=”password”
  2. Library
    • id
      • Sample :- MSJDBCLib
      • file name :- C:\Downloads\Microsoft\Java\jdbc\v6.0.8112.200\extract\sqljdbc_6.0\enu\jre8\sqljdbc42.jar
Sample
Image

serverDotxml_20180723_0819AM

 

Code

	<dataSource
		id="jdbc/inform"
		jndiName="jdbc/inform"
	>
		<jdbcDriver
			libraryRef="MSJDBCLib"
		/>

		<properties.microsoft.sqlserver
			serverName="localhost"
			portNumber="1433"
			databaseName="StackOverflow2010"
			user="stackoverflow"
			password="hIy8jA2lNl"
		/>

	</dataSource>

	<library id="MSJDBCLib">
		<file
			name="C:\Downloads\Microsoft\Java\jdbc\v6.0.8112.200\extract\sqljdbc_6.0\enu\jre8\sqljdbc42.jar"
		/>
	</library>

Code

Outline

  1. table.java
    • java source code
  2. compileDBInform.cmd
    • Command file for compiling Java Source Code
  3. web.xml
    • web.xml file
  4.  packageDBInform.cmd
    • Package java class files into War file
  5. deployDBInform.cmd
    • Copyy War file unto WebSphere Server dropins folder

Source Code

table.java


package dbInform;

import java.io.*;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import javax.annotation.Resource;
import javax.servlet.*;
import javax.servlet.http.*;
import javax.servlet.annotation.WebServlet;

import javax.sql.DataSource;

import javax.naming.*;

public class table
    extends HttpServlet
{

    private static final long serialVersionUID = 1L;

    private DataSource objDS;
    private Connection objDBConn = null;

    private static final String DB = "jdbc/inform";  

    private static final String QUERY_TABLE
        = "select * "
            + " from INFORMATION_SCHEMA.TABLES tblIST "
            + " order by TABLE_SCHEMA, TABLE_NAME"
            ;

    private String strData;
    private String strRow;

    private String strBGColor = "";
    private int    iRowNumber = 0;

    private String TR_COLUMN_EVEN = " bgcolor = '#AF601A'";
    private String TR_COLUMN_ODD = " bgcolor = '#1D8348'";

    private String strTableSchema;
    private String strTableName;
    private String strTableType; 

    public table()
    {
        super();
    }
    public void doGet
    (
          HttpServletRequest request
        , HttpServletResponse response
    )
        throws
               ServletException
             , IOException
    {

        response.setContentType("text/html");

        PrintWriter out = response.getWriter();

        out.println("
<H1>Inform.table</H1>

\n");

        try
        {

            Context objCTX = new InitialContext();

            Object obj = null;

            obj = objCTX.lookup
                    (
                        DB
                    );

            objDS = (DataSource) obj;

            objDBConn = objDS.getConnection();

            Statement stmt = null;

            stmt = objDBConn.createStatement();

            ResultSet rs = stmt.executeQuery
                            (
                                QUERY_TABLE
                            );

            out.println("
<TABLE BORDER="1" WIDTH='80%'>");    

            strData =
                          "
<TH>" + "SCHEMA" + "</TH>

"
                        + "
<TH>" + "TABLE" + "</TH>

"
                        + "
<TH>" + "TYPE" + "</TH>

"
                  ;

        strRow = "
<TR>" + strData + "</TH>

";

        out.println(strRow);

       while (rs.next())
       {

            iRowNumber = iRowNumber + 1;

            if ((iRowNumber %2) == 0)
            {
                strBGColor = TR_COLUMN_EVEN;
            }
            else
            {
                strBGColor = TR_COLUMN_ODD;
            }

            strTableSchema = rs.getString("TABLE_SCHEMA");
            strTableName = rs.getString("TABLE_NAME");
            strTableType = rs.getString("TABLE_TYPE");

            strData =
                        "
<TD>" + strTableSchema + "</TD>

"
                      + "
<TD>" + strTableName + "</TD>

"
                      + "
<TD>" + strTableType + "</TD>

"

                      ;

            strRow = "<TR>"
                        + strData
                        + "</TR>

";

            out.println(strRow);

        }

       out.println("</TABLE>

"); 

    }
    catch (SQLException e)
    {
       e.printStackTrace();
    }
    catch (NamingException e)
    {
       e.printStackTrace();
    }
    finally
    {

       if (objDBConn != null)
       {
            try
            {
                objDBConn.close();
            }
            catch (SQLException e)
            {
                e.printStackTrace();
            } 

           }
       }
    }
}

Compile Java Source Code

compileDBInform.cmd


setlocal

set "_folderCompiler=C:\Program Files\Java\jdk-10.0.2\bin"

set "_relVer=8"

set "_relTag= --release %_relVer%"

set "_folderJar=C:\IBM\WebSphere\Liberty\v18.0.0.2\extract\\wlp\dev\api\spec"

set "_fileServlet=com.ibm.websphere.javaee.servlet.3.1_1.0.21.jar"

set "_fileAnnotation=com.ibm.websphere.javaee.annotation.1.3_1.0.21.jar"

set "_folderDeploy=C:\IBM\WebSphere\Liberty\v18.0.0.2\extract\wlp\usr\servers\defaultServer\dropins"

set "_targetFolder=WEB-INF\classes"

"%_folderCompiler%\javac" -cp %_folderJar%\%_fileServlet%;%_folderJar%\%_fileAnnotation%  %_relTag% -d %_targetFolder% table.java

endlocal

Configuration Files

xml.web

Image

webDotxml_20180723_0813AM

Code


<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app xmlns="http://java.sun.com/xml/ns/j2ee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"
    version="2.4">

    <display-name>
        DBInform
    </display-name>

    <description>
    Simple Application that shows database schema via INFORMATION_SCHEMA.
    </description>

    <!--
        dbInformTable
            dbInform.table
    -->
    <servlet>
        <servlet-name>table</servlet-name>
        <servlet-class>dbInform.table</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>table</servlet-name>
        <url-pattern>/table</url-pattern>
    </servlet-mapping>

</web-app>

Package

packageDBInform.cmd


setlocal

set "_folderCompiler=C:\Program Files\Java\jdk-10.0.2\bin"
set "_targetFolder=WEB-INF\classes\"

"%_folderCompiler%\jar" cvf dbInform.war WEB-INF\web.xml %_targetFolder%\dbInform\table.class

endlocal

Deploy WAR File

deployDBInform.cmd


setlocal

set "_folderBase=C:\IBM\WebSphere\Liberty\v18.0.0.2\extract\wlp\usr\servers\defaultServer\dropins"

echo F | xcopy dbInform.war %_folderBase%\dbInform.war /d /Y /Q 

endlocal

Browser

Launched browser and pointed it at ( https://localhost:9443/dbInform/table )

and we got a list of the tables in our database.

dbInform_table_20180722_1054PM

Source Code Control

Github

DanielAdeniji/javaDBInform
link

Error 404: javax.servlet.UnavailableException: SRVE0202E: Servlet was found, but is corrupted: SRVE0227E:

Background

Playing around with Liberty, but not getting far.

One of the errors that I am seeing is the one posted below.

Error

Error Image

SRVE0202E_20180721_0626PM

Error Text


 Error 404: javax.servlet.UnavailableException: 

SRVE0202E: Servlet [helloWorld]: wasdev.helloWorld was found, but is corrupted: 

SRVE0227E: Check that the class resides in the proper package directory. 

SRVE0228E: Check that the classname has been defined in the server using the proper case and fully qualified package. 

SRVE0229E: Check that the class was transferred to the filesystem using a binary transfer mode. 

SRVE0230E: Check that the class was compiled using the proper case (as defined in the class definition). 

SRVE0231E: Check that the class file was not renamed after it was compiled.

Trouble Shooting

Class File Name & Folder Structure

Here are the steps we took to validate the file:

  1. Checked to ensure that file is indeed present in the war file
    • The War is a simple archive file and was able to use 7-Zip to review the file’s location
      • Package name matches folder structure

Environment

Java Run-time

Task Manager

Using MS Windows Task Manager, reviewed the java run-time.

Task Manager – Details

taskManager_javaw_20180721_0655PM

Task Manager – Process – javaw.exe – Tab – Details
Image

javaw_20180721_0654PM

Image
  1. File Version :- 8.0.1210.13
  2. Product Version :- 8.0.1210.13

Compiler

Java Compiler

javac

Reviewed the java compiler that we are using and noticed that we are using Version 10.

Sample

"C:\Program Files\Java\jdk-10.0.2\bin\javac" -version

Output

javac_Version_20180721_0713PM

Remediation

Java Compiler

Stay with later model Java Compiler and use -release option

Update the script to use the “–release” option.


 >"C:\Program Files\Java\jdk-10.0.2\bin\javac" -cp C:\IBM\WebSphere\Liberty\v18.0.0.2\extract\\wlp\dev\api\spec\com.ibm.websphere.javaee.servlet.3.1_1.0.21.jar;C:\IBM\WebSphere\Liberty\v18.0.0.2\extract\\wlp\dev\api\spec\com.ibm.websphere.javaee.annotation.1.3_1.0.21.jar   --release 8 -d WEB-INF\classes helloWorld.java

Align Java Compiler

Downloaded JDK v8, Installed it, and changed our script from using javac 10 to now use java 8.

Credit

Will have to come back and credit where I got this salient advice from.

As I always say, the whole point of writing is to cite proven sources.

WebSphere – Edition – Liberty ( v18 ) – Installation

Background

It is good to revisit Java J2EE after a long hiatus for me personally.

WebSphere – Liberty

Thankfully IBM avails WebSphere via its Liberty Edition.

Liberty is free and as such it is very accessible.

Version 18.x.y.z

Download

The current version of WebSphere Liberty is v18.0.0.2 and it is available here.

Please download it.

Extract

Extract the downloaded package.

Copy

Copy the extracted files into a neutral location where it will live and be delivered from.

Install

Install Additional features

Here are the additional features that we will be installing:

  1. AdminCenter
AdminCenter
Command
Command – Linux

bin/installUtility install adminCenter-1.0

Command – Windows

bin\installUtility install adminCenter-1.0

Output

adminCenter-1Dot0_20180720_0145PM

Configuration

Security

Server.xml
Security

Please access the server.xml file and add entries that look like what is pasted below.

Server.xml – Security
  1. basicRegistry
    • list each user and password
  2. administrator-role
    • list each user that should be part of the administrator group

    <basicRegistry id='basic'>
      <user name='admin' password='websphere' />
      <user name='guest' password='guest' />
    </basicRegistry>

    <administrator-role>
      <user>admin</user>
    </administrator-role>

    <keyStore 
        id='defaultKeyStore' 
        password='Liberty' 
    />
Features – Enable AdminCenter

     <!-- Enable features -->
    <featureManager>
        <feature>webProfile-8.0</feature>
        
        <!--
            2018-07-20 2:02 PM dadeniji
        -->
        <feature>adminCenter-1.0</feature>
        
    </featureManager>

Control

To control the web server, please launch a command shell and issue commands against the bin folder.

Start

Code

bin\server start
Output

serverStart_20180720_0141PM.png

Stop

Code

bin\server stop
Output

serverStop_20180720_0208PM.png

Web Access

Port Numbers & Host

Server.xml

Get Network Port Numbers and host from server.xml

      <!-- To access this server from a remote client 
       add a host attribute to the following element, 
       e.g. host="*" 
   -->
    <httpEndpoint id="defaultHttpEndpoint"
                  httpPort="9080"
                  httpsPort="9443" 
    />

Username & password

server.xml

Also from server.xml, please get user and password.

    <!-- Define an Administrator and non-Administrator -->
    <basicRegistry id="basic">
      <user name="admin" password="websphere" />
      <user name="guest" password="guest" />
    </basicRegistry>

    <!-- Assign 'admin' to Administrator -->
    <administrator-role>
      <user>admin</user>
    </administrator-role>

Browser

Launch a web browser and enter the URL and Port Number for Admin Center.

As we are logged on locally, please enter http://localhost:9443/AdminCenter.

If trying to connect from remote host, please edit your server.xml.

And, add host=* to the httpEndPoint section.

adminCenter_login_20180720_0212PM.png

 

References

  1. IBM
    • IBM Knowledge Center
      • Home > WebSphere Application Server Liberty base. . .> Administering Liberty using Admin Center
        • Setting up Admin Center
          Link
        • Logging in to Admin Center
          Link