NUMA – Get Current Configuration ( On Linux )

Background

Reading through database documentation and understanding that some database platforms do not really support NUMA, as well as others.

What is NUMA?

Wikipedia

Link

Modern CPUs operate considerably faster than the main memory they use.

CPUs increasingly have found themselves “starved for data” and having to stall while waiting for data to arrive from memory.

Many supercomputer designs of the 1980s and 1990s focused on providing high-speed memory access as opposed to faster processors, allowing the computers to work on large data sets at speeds other systems could not approach.

Limiting the number of memory accesses provided the key to extracting high performance from a modern computer.

For commodity processors, this meant installing an ever-increasing amount of high-speed cache memory and using increasingly sophisticated algorithms to avoid cache misses.

But the dramatic increase in size of the operating systems and of the applications run on them has generally overwhelmed these cache-processing improvements. Multi-processor systems without NUMA make the problem considerably worse. Now a system can starve several processors at the same time, notably because only one processor can access the computer’s memory at a time.

NUMA attempts to address this problem by providing separate memory for each processor, avoiding the performance hit when several processors attempt to address the same memory.

For problems involving spread data (common for servers and similar applications), NUMA can improve the performance over a single shared memory by a factor of roughly the number of processors (or separate memory banks).

Of course, not all data ends up confined to a single task, which means that more than one processor may require the same data. To handle these cases, NUMA systems include additional hardware or software to move data between memory banks.

This operation slows the processors attached to those banks, so the overall speed increase due to NUMA depends heavily on the nature of the running tasks.

Linux

Review Configuration

Processor Name

Get Processor Name

/proc/cpuinfo

Code

cat /proc/cpuinfo | grep -i "model name" | sort | uniq

Output

processorName.01.20190804.0538PM

Processors / CPU

Get Number of CPUs and Cores

Number of Cores

/proc/cpuinfo
Code

cat /proc/cpuinfo | grep -c 'core'

Output

core.01.20190804.0456PM

Number of processors

/proc/cpuinfo
Number of processors
Code

cat /proc/cpuinfo | grep -c 'processor'

Output

processor.01.20190804.0438PM.PNG

 

Number of Sockets

lscpu
Number of sockets
Code

lscpu | grep -i 'socket'

Output

socket.01.20190804.0448PM

NUMA Grouping

Get CPU NUMA grouping

lscpu

Code

lscpu | grep -i 'NUMA'
Output

numa.lscpu.01.20190804.0501PM

Explanation
  1. Here we have a lone NUMA Group
  2. All six (6) CPUs are within the lone grouping

 

numactl

Code

numactl -- hardware
Output

numactl.hardware.01.20190804.0550PM

Explanation
  1. Single NUMA Node
    • CPUs => 0,1,2,3,4,5
    • Size => 132 GB
    • free => 600 MB

 

References

  1. James Green
    • Sockets vs. Cores on a VMware VM Config?
      Link
  2. Indiana University
    • About processors, chips, sockets, and cores
      Link
  3. Stack Overflow
    • How to obtain the number of CPUs/cores in Linux from the command line?
      Link
  4. FibreVillage
    • How to tell if CPU hyperthreading enabled on Linux
      Link

Samba :- Linux & Windows Integration

Background

Mid last week as I was returning home received a word.

It read :-

The Linux Host is not accessible and the Linux Subsystem on our Window 10 desktop is not working either.

 

Working Through

Windows 10 Linux Subsystem

Looked at the Windows 10 Subsystem and the issue appears to be newbie issues.

It looks like a few of the commands from the Linux Study Guide that my family member  is following is not available in the Windows Subsystem.

 

Linux Host

The Linux host was more complicated.

Rebooted

Tried to connect to the Linux Host, but it wouldn’t respond

Due to lack of affection and attention and it needed a reboot badly.

 

Applied Patches

Waiting for me was a trek load of patches.

Applied them.

 

SSH

Tried to ssh it from one of our Windows systems.  But, I couldn’t get a response.

Wireless Access Point

Dynamic IP Address

The Linux host is connected to our network via a Wireless connection.

Connected to the WAP via a browser.

Confirmed that the IP Address had changed from the one registered with the DNS server.

 

DHCP Reservation

Tried to see whether the WAP supports DHCP Reservation.

DHCP Reservation basically states that a given MAC Address should be granted exclusive access to an IP Address.

DNS

Knowing the the WAP could end up offering a different IP Address, it was time to untie our Linux host from a permanent IP Address.

 

SAMBA

Years ago, played around with SAMBA.

But, that was yearns ago.

Vladyslav Baidak @ Scalified

Googled on the right terms.

And, ended up finding a quality post from Vladyslav Baidak @ Scalified.

That post is here :-

Devops: Configuring Host Name Resolution
Link

 

Linked In

Had a few suggestions and reached out to Vladyslav Baidak via Linked In.

Textual

Hi Daniel! Thanks for your interest. I’m really glad I could help you. Indeed that was the reason for writing this mini article – I had the same question some time ago.   As for the title – that’s a typo for sure, we’ll fix it soon

Daniel, we fixed the title and added possibility to add comments as you asked.

 

Image

VladyslavBaidak.linkedIn.20190715.1120AM

 

 

 

 

 

 

Samba :- Name Resolution – From Linux, Resolve Windows Host

Background

Here is how to use Samba to perform name resolution from a Linux Host to MS Windows computers.

Lineage

  1. Windows
    • WINS
      • Installing & Configuring WINS
        Link
  2. Linux
    • Samba
      • Samba – Host Name Resolution
        Link

Steps

Outline

  1. Package Installation
    • samba-winbind
  2. Network Configuration
    • Names Service Switch
      • /etc/nsswitch.conf
    • Samba
      • etc/smb.conf
  3. Firewall Configuration
    • Firewall – firewalld
      • Allow tcp/137 and ucp/137
  4. Restart Services
    • winbind

Package Installation

samba-winbind

yum

yum – is package installed?
Objective

Is samba-winbind installed?

Code

sudo yum list installed | grep samba

Output

package.install.yum.list.01.20190711.png

yum – Install Package
Objective

Install samba-winbind

Code

sudo yum -y install samba-winbind

Output

Image – 1

package.install.yum.install.01.20190711.0801PM.png

Image – 2

package.install.yum.install.01.20190711.0802PM.png

Network Configuration

Files

File :- etc/nsswitch.conf
Outline
  1. Load the /etc/nsswitch.conf file in an editor
  2. Ensure that wins is included in the list of hosts to consult for name resolution requests
    • Original Entry
      • hosts : files dns myhostname
    • Revised Entry
      • hosts : files dns myhostname wins
Images
Image – 01

nsswitch.conf.01.20190711.png

Image – 02

nsswitch.conf.03.20190711.0807PM.png

File :- etc/smb.conf
Outline
  1. Load the /etc/smb.conf file in an editor
  2. Sections
    • Section – Global
      • Configuration Entry
        • workgroup
          • Please have your network domain name
        • netbios name
          • Your computer’s host name
        • wins server
          • Please ensure that your wins server is referenced
Images

smb.conf.01.20190712.0436AM.PNG

 

Network Firewall Configuration

Firewalld

Our OS is centOS and our current firewall is firewalld.

Configuration
Command – Syntax

sudo firewall-cmd --zone={zone} --add-port={port-number}/{protocol} --permanent

Command – Sample

sudo firewall-cmd --zone=public --add-port=137/udp --permanent

sudo firewall-cmd --zone=public --add-port=137/tcp --permanent

Images

firewall.allow.01.20190711.png

Restart Services

Outline

Please restart relevant services.  Those are :-

  1. winbind

 

Configuration

Command – Syntax
sudo service {service} restart
Command – Sample

sudo service winbind restart

Troubleshooting

Issues

ping: unknown host <hostname>

  1. Restart Services
    • Restart winbind
  2. Configuration
    • Linux
      • Configuration – File –
        • /etc/smb.conf
          • Ensure that your WINS Server is referenced

Referenced Work

Vladyslav Baidak
Backend Engineer at Scalified
Devops: Configuring Host Name Resolution
Link
September 24, 2017

References

  1. StackExchange
    • Ask Ubuntu
      • Windows Hostnames are not resolved
        Link
      • Why can’t my machine resolve local windows hostnames?
        Link

Samba :- Name Resolution – From Windows, Resolve Linux Host Names

Background

Upon trying to connect to the lone Linux host on our LAB network found it inaccessible.

Later found out that the IP Address received from the DHCP Server has changed.

 

Goal

Need to be able to use name resolution and find the Linux host from a windows box.

 

Network Ports

  1. Netbios Name Service
    • UDP Port 137
  2. Netbios Datagram Service
    • UDP Port 138
  3. SMB/CIFS Network Traffic
    • Windows NT 4.0 and Below
      • TCP Port 139
      • UDP Port 139
    • Windows 2000 and above
      • TCP Port 445

Access Linux Hosts from Windows

Outline

  1. Samba Package
    • Is Samba Package Installed?
    • If not, install Samba Package
  2. Configuration
    • /etc/samba/smb.conf
      • netbios name
  3. Start and Enable Services
    • smb
      • Start & Enable for auto-start
      • Check Status
  4. Firewall
    • Netbios Naming Service
      • UDP port 137
      • TCP port 137
  5. Confirm
    • Linux
      • Confirm listening network ports
    • Windows
      • Confirm windows hosts name resolution

Tasks

Samba Package

Is Samba Installed?

yum
Syntax
sudo yum list installed

Sample
sudo yum list installed | grep samba

Output – Image

source.windows.01.20190709.0847PM

Explanation
  1. Samba Installed

Install Samba

If Samba has not yet been installed, we install it.

yum
Syntax
sudo yum -y install {package}
Sample
sudo yum -y install samba

Samba Configuration

Outline

  1. /etc/samba/smb.conf
    • Domain
    • Host
Editor
Syntax
sudo vi {file}
Sample
sudo vi /etc/samba/smb.conf

Output – Image

smb.conf.02.20190709.0946PM.PNG

Service

Service Start and Enable for auto-start

Syntax
sudo systemctl start {service}
sudo systemctl enable {service}
Sample
sudo systemctl start nmb
sudo systemctl enable nmb

 

Service Status

Syntax
sudo service {service} status -l
Sample
sudo service nmb status -l
Output

service.nmb.status.01.20190709.0913PM.PNG

Explanation
  1. Local Master Browser
    • Our Linux host takes upon the role of a local master browser
    • The role is for a specific subnet based on the Interface’s IP Address

Firewall

Firewall – Open
Syntax
sudo firewall-cmd --zone=public --add-port={port}/{protocol} --permanent

Sample
sudo firewall-cmd --zone=public --add-port=137/udp --permanent
sudo firewall-cmd --zone=public --add-port=137/tdp --permanent

Output – Image

firewall.137.01.20190709.0902PM

Explanation
  1. TCP Port 137
    • Opened
  2. UDP Port 137
    • Opened

Confirmation

Confirm Listening Ports

On Linux Hosts, confirm listening network ports

netstat
Syntax
netstat
Sample
netstat -lu | grep netbios
Output – Image

netstat.listening.01.20190710.0218M

Explanation
  1. The listening ports are listed
    • Network Ports
      • They are udp ports
        • netbios-ns
        • netbios-dgm
    • Services
      • /etc/services
        • Please consult the /etc/services file for corresponding network port numbers for the listed service

 

Confirm Windows Host Name Resolution

Confirm that windows host able to perform name resolution against Linux Hosts

Command Line Tools
Syntax
ping {linux-host-name}
Sample
ping linuxdbhost
Output – Image

ping.windows.02.20190710.0144AM

Explanation
  1. The windows host is able to perform name resolution against the windows host

 

Referenced Work

Vladyslav Baidak
Backend Engineer at Scalified
Devops: Configuring Host Name Resolution
Link
September 24, 2017

References

  1. Samba.org
    • Firewalling Samba
      Link
  2. Centrify
    • Configuring the Samba integration
      • Modifying the Samba smb.conf configuration file
        Link
  3. Stack Exchange
    • superuser
      • What is NetBIOS? Does Windows need its ports 137 and 138 open?
        Link
  4. Rackspace
    • How To
      • Check running services on Linux
        Link
  5. Makandra Cards
    • netstat: show listening ports
      Link
  6. Red Hat
    • Archives
      • Re: What is NMB ?
        Link
  7. OS Technix
    • How To Find The Port Number Of A Service In Linux
      Link

IBM Divestiture – 2019

Background

IBM recently completed its purchase of Raleigh’s North Carolina Linux Powerhouse, Red Hat.

As part of that corporate merger, it appears regulars whether here overseas are concerned of allowing too much vertical technology ( Application software, Operating System, and Integration Software ) to rest in a single entity.

 

Affected Product Lines

HCL Software

  1. Collaboration
    • Notes Domino
  2. Application Security
    • AppScan
  3. Application Delivery
    • Bigfix
  4. Marketing
    • Unica
  5. Digital Experience

 

Impact

The current customer base of the products will obviously be affected.

Impacts will include license renewals, support issue tracking, etc.

Customer facing support staff will also have to respond graciously concerns raised by customers.

Referenced Material

  1. HCL
    • HCL TECHNOLOGIES ANNOUNCES CLOSE OF ACQUISITION OF SELECT IBM PRODUCTS
      Link

 

AWS/EC2 – Instance – Metadata

Background

Continuing with our study on AWS/EC2.

Let us quickly cover how we go about querying the EC2 instance for rudimentary system information.

Environment

On the instance itself, please send a payload against Link-local address; specifically 169.254.169.254.  The full URL is http://169.254.169.254/latest/meta-data/

Tool

We are on MS Windows and we have browsers loaded and we will use them as simple HTTP Client.

if you are on Linux and all you have is a terminal mode connection, please use curl ( Client URL ).

Top Level

Here is our top level node.

Browser

Block Device Mapping

Lists Storage Devices; please note only block devices and not RAW Storage Devices.

Please also keep in mind detailed storage info is not available; merely names.

Images

Image-Block Device Mapping

hostname

Images

Image-hostname

instance-id

Images

Image-instance-id

instance-type

Images

Image-instance-type

hostname

Images

hostname

local-hostname

Images

Image-Localhostname

local-ipv4

Images

Image-Local IP4 Address

Mac

Please read further; specifically under Network\Interface\Macs.

Network

interface

Macs

Network Mac Addresses which can be fairly useful for network troubleshooting, etc.

Images
Network Mac -01

Placement

Availability Zone

Here we talk about Country/Region/Availability Group.

In our case we are in

  1. Country :- US
  2. Region :- East Coast
  3. Availability Zone :- 1C

Images

Image.us-east-1c

Security-Groups

Security Groups are the Security groups each instance is assigned.

In the screen shots below we have two distinct results.

First is workplace for AWS Workplace node and the second is one of our custom security groups.

Images

Image – workplace

Image – taskRunner

Summary

Quick summary.

Instance metadata allows a bit of introspection on each EC/2 instance.

Akin to Windows Management Interface, WMI, in MS Windows.

References

  1. Amazon
    • AWS Documentation » Amazon EC2 » User Guide for Windows Instances » Amazon EC2 Instances » Configuring Your Windows Instance
      • Instance Metadata and User Data
        Link
    • AWS Documentation » Amazon EC2 » User Guide for Windows Instances » Monitoring Amazon EC2 » Monitoring the Status of Your Instances
      • Scheduled Events for You
        Link
  2. Curl for Windows

Eclipse/Hibernate – Installation – Day/01

 

Outline

  1. Download & Install Eclipse
  2. JDBC Driver
    • Microsoft JDBC Driver
      • Download Microsoft JDBC Driver
  3. Configure Database
    • Microsoft SQL Server
      • Provision Database
        • WideWorldImporters ( Sample Database )
          • Restore Database
      • Provision Principal
        • Create SQL Server Principal
        • Grant Principal access to database
        • Grant principal access to database objects
  4. Configure Eclipse
    • Install Components
      • JBoss Tools
        • Hibernate

Tasks

Download & Install Eclipse

Please peruse this post per installing Eclipse :-

Eclipse IDE – Installation On MS Windows
Link

JDBC Drivers

Our database back-end is Microsoft SQL Server and so we will reach for Microsoft SQL Server JDBC Drivers.

Microsoft

SQL Server

JDBC Driver
Links
  1. Microsoft JDBC Driver for SQL Server
    Link
  2. Download Microsoft JDBC Driver for SQL Server
    Link
Available

Here are all the drivers available as of 2019-May-6th :-drivers.microsoft.jdbc.20190506.1231PM

We are on 7.0.

And, we will cover that in this post!

Database

Again, our database back-end is Microsoft SQL Server.  There are a few sample databases on that platform; the latest official is WorldWideImporters.

Microsoft

SQL Server

Worldwide Importers
Links
  1. SQL Samples
    Link
  2. Wide World Importers
    • Wide World Importers sample database v1.0
      Link
Brochure

Brochure

microsoft.sample.20190506.1246PM

Choice

We can go with traditional database backup files ( bak ).

Or with bacpac.

We chose :-

SQL Server 2016 SP1 (or later) Any Edition aside from LocalDB; SQL Server 2016 RTM (or later) Evaluation/Developer/Enterprise Edition
WideWorldImporters-Full.bak

Restore database

Launched SQL Server Management Studio and restored the backup file


USE [master]
go

exec master..xp_create_subdir N'C:\Microsoft\SQLServer\Datafiles\WideWorldImporters\'
go

exec master..xp_create_subdir N'C:\Microsoft\SQLServer\Logfiles\WideWorldImporters\'
go

RESTORE DATABASE [WideWorldImporters]
FROM  DISK = N'C:\downloads\Microsoft\SQLServer\Sample\WideWorldImporters\WideWorldImporters-Full.bak'
WITH
       FILE = 1
    ,  MOVE N'WWI_Primary' TO N'C:\Microsoft\SQLServer\Datafiles\WideWorldImporters\WideWorldImporters.mdf'
    ,  MOVE N'WWI_UserData' TO N'C:\Microsoft\SQLServer\Datafiles\WideWorldImporters\WideWorldImporters_UserData.ndf'
    ,  MOVE N'WWI_Log' TO N'C:\Microsoft\SQLServer\Logfiles\WideWorldImporters\WideWorldImporters.ldf'
    ,  MOVE N'WWI_InMemory_Data_1' TO N'C:\Microsoft\SQLServer\Datafiles\WideWorldImporters\WideWorldImporters_InMemory_Data_1'
    ,  NOUNLOAD
    ,  STATS = 5
    ,  REPLACE

GO
Create Database Account

Let us create SQL Server Account


use [WideWorldImporters]
go

/*
    SQL Server Principal - helloDB
*/
if suser_id('helloDB') is null
begin

    create login [helloDB]
    with password = 'Nu3Hu5RuSt'
       , DEFAULT_DATABASE = [WideWorldImporters]
       , CHECK_EXPIRATION = OFF
       , CHECK_POLICY = OFF

end
go

/*
    SQL Server Principal - helloDB
*/
if user_id('helloDB') is null
begin

    create user [helloDB]
    from login [helloDB];

end
go

grant select, insert, update, delete on schema::[Application] to [helloDB]
go

grant select, insert, update, delete on schema::[Purchasing] to [helloDB]
go

grant select, insert, update, delete on schema::[Sales] to [helloDB]
go

grant select, insert, update, delete on schema::[Warehouse] to [helloDB]
go

jboss.org

Available Software

In the Work With entry box, please enter :-

http://download.jboss.org/jbosstools/photon/development/updates

Once entered please click the “Add…” button

install.AvailableSoftware.02.WorkWith.20190505.0225PM.PNG

Add Repository

Please give a Name for the Repository we are adding.

In our case we chose jboss.

install.AvailableSoftware.03.WorkWith.20190505.0225PM.PNG

 

Check the items that you wish to install

Using Web Services, jboss.org is reached.

Available Items are returned and displayed.

install.AvailableSoftware.05.AddRepository.20190505.0228PM.PNG

Item – jBoss Data Services Development

We selected the following items :-

  1. JBoss Data Services Development
    • Hibernate Tools

install.AvailableSoftware.06.AddRepository.20190505.0229PM.PNG

Item – jBoss Data Services Development – Install Details

Please review the selected items :-

  1. JBoss Data Services Development
    • Hibernate Tools

install.AvailableSoftware.07.AddRepository.20190505.0230PM.PNG

 

Item – Review Licenses

Please review License Agreement

  1. License text(for Hibernate Tools 5.4.4.v20190328-1505)

 

Item – Review Licenses

Please review License Agreement

  1. License text(for Hibernate Tools 5.4.4.v20190328-1505)

Item – Installing Software

 

Item – Security Warning

Security Warning – The authenticity or validity of the software cannot be established

Even Hibernate.org is shipping software without SSL Certificates?

 

It is good, please click “Install anyway”.

Restart

Please restart Eclipse ID to effect the changes.

Restarted

Upon restart, we are asked to opt in for anonymous usage statistics.

Summary

Enough shenanigans for Day 01.