12/5/2013
Java on HP NonStop Update
for BITUG BIGSIG
Moore Ewing
HP EMEA NonStop PreSales
December 2013
1
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The unique value of HP NonStop
Designed from the ground up for 100% application availability
The HP NonStop Integrated Stack
NonStop Operating System
Database and transactions
• Fault-tolerant, scalable, shared-nothing
architecture
• Single-system-image clustering up to 255
systems
• Uncompromising data integrity
• Highly scalable, clustered SQL database with
transactional integrity
• Mixed workload support – OLTP, queries and
batch
Security
• Built-in system and application security
• Enterprise security integration
• Software to help with Security audits
APPLICATIONS
Modern Application Development
Middleware
Database and Transaction Management
System Management and Control
System management
Security
• HP Mission-Critical Converged
Infrastructure integration
• Industry standard, enterprise class
NonStop Operating System
2
Middleware
• Transparent application fault tolerance and
scalability through standard middleware
• Service-Oriented Architecture (SOA)
• Java frameworks for fast deployment
Modern app development
• Java, C/C++, ODBC, JDBC, SOAP, XML
• Eclipse
Hardware
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
1
12/5/2013
Architecture evolution (Hardware)
- from Bus&Channel technology to Server-Area-Network
NonStop 1 to K-Series
NonStop S- and NS-Series
ServerNet
Shared-nothing architecture
2 to 16 CPUs per system
Fault-tolerant hardware & software
Bus-based system interconnect
Any-to-any IPC
Constrained I/O connectivity
• Shared-nothing architecture
• Replaced Bus and Channel
by low-latency network CPUs per Any-to-any
IPC
T/16 CISC chips -> MIPS RISC chips ------
Itanium 1-core -
Itanium n-core
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
NonStop is primarily a software product
NonStop OS + TMF + Pathway + NSSQL + RDF
+iTPWebserver + NSJ + NSSOAP + NSJSP
TNS1
CLX
CYCLONE
S-Series
NS-Series
NSBlades
The hardware has constantly changed to new technology ,
the architecture of how the software uses it has stayed the same !
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
2
12/5/2013
NonStop Architecture is Scalable and Fault-Tolerant
How does an HP NonStop server do it?
Share-nothing, clustered, single system image architecture
•
Creates an Available and Scalable system with online expansion of resources (CPUs, disc, etc.)
Hardware and software fault-tolerance
•
Non-redundant duplication of hardware plus stateful takeover by process-pairs
Single system image across network of clusters
•
Supports local expansion, distributed applications and disaster recovery
Fail-fast fault containment
•
Combines with share-nothing model to prevent propagation of errors
Online management of the system and database
•
Avoids application downtime while administration is taking place
Evolving technology
•
Intra-system connectivity: bus/channel – ServerNet (CPU and I/O network)
•
Processor chips: Tandem CISC – MIPS RISC – Intel Itanium EPIC
5
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
NonStop system architecture is MPP not SMP
ServerNet Physical Interconnect
NOS
Single System Image
NOS
SQL/MX
TCP/IPv6
Process Management
NOS
NOS
6
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
3
12/5/2013
HP NonStop
The critical role of “TS/MP”
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
NonStop TS/MP
Scalability and availability
Requestor
Requestor
Requestor
Requestor
Requestor
Requestor
Requestor
Requestor
PATHSEND
PATHSEND
App 2
Ap 1
App 1
App 1
Ap 1
App 1
App 1
Requestor
PATHSEND
SC-APP2
TS/MP
App 2
App 2
App 2
App 2
App 2
App 2
PATHSEND
TS/MP
Ap 1
App 1
App 1
SC-APP1
Requestor
App 2
Ap 1
App 1
App 1
4
12/5/2013
TS/MP and Requester-Serverclass structure
- Application Virtualization in and across NonStop Servers
Application is divided into
- Requester processes - Presentation. Navigation , Context Management
•
- ServerClasses – pools of replicated processes
doing data-related business logic and data access
Application load balancing within & across NonStop Servers
•
Application persistence
Transaction management within & across NonStop Servers
•
Instances of a Server Class balanced across CPUs
•
Location of data transparent to application
•
On-line upgrade to new version of Server
Classes
•
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Bringing Java to
Integrity NonStop
About more than just a language ........... About Enterprise quality transactional applications
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
5
12/5/2013
Java EE for Transactional Online Processing Applications
• APIs and TECHNOLOGY SPECIFICATIONs
– A collection / integration of various enterprise
application functions and APIs to access them
•
•
•
•
Development and Deployment platform
– A unified platform for server-side development
Reference Implementation
– Implements the Java EE specification & demonstrates its viability
Compatibility tests
– Certifies a Java EE product, confirms application portability
Java EE Blueprints
– Programming model, patterns, guidelines, best practices
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
JEE functionalities
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
6
12/5/2013
Bringing Java to Integrity NonStop
• Technical challenges in NS system environment
- a uni-processor system without threading
- 32-bit addressing with a 2GB user data limit in the process
- Static binding without DLLs
• “Religious wars” in the JEE world
- EJB Entity beans very “ugly” and expensive
- Servlet/JSP + Opensource projects satisfied many (most) applications
• Implemented
- POSIX User level threading for JVM
- Opensource Web Container (Tomcat) and popular Frameworks .
-Have added 64-bit addressing and EJB3 container (JBoss/Wildfly))
• Had to bridge the SMP-MPP threading gap………
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
NonStop Server for Java
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
7
12/5/2013
NSJ features – summary
NSJ 6 features
NSJ 7 features
• Based on Oracle JDK 1.6.0_24 version
• Based on Oracle JDK 1.7.0_01 version
• Supports all headless JDK APIs
• First NSJ release with 64-bit support
• Supports a very large Java heap
– Size is only limited by the physical memory of the
system
• < 1.2 GB Java heap size
• Current version = NSJ 6 Update 3
• Improved garbage collection
– Parallel and concurrent mark sweep GC
15
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
This is a rolling (up to three year) Roadmap and is subject to change without notice
.
Garbage collection and NSJ 7.0
New feature in future release of NSJ
For a JVM running on NSMA (J-series)
• GC Policy Commands now supported
CPU n
-XX:+UseSerialGC
-XX:+UseParallelGC
IPU 0
IPU 1
IPU 2
GC Process
GC Process
IPU 3
JVM
-XX:+UseParNewGC
-XX:+UseParallelOldGC
GC Process
GC Process
-XX:+UseConcMarkSweepGC
• Garbage collection processes created in each IPU
associated with each JVM
• Essentially working as GC “threads” similar to SMP
implementations (e.g. HP-UX)
SHARED MEMORY
C-heap Java Heap Java Thread Stacks
• Can be used in either 32-bit or 64-bit JVM
For a JVM running on single-core (H-series)
• GC policy remains as Serial
• JVM will not start if any other GC Policy Command is used
16
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
This is a rolling (up to three year) Statement of Direction and is subject to change without notice.
8
12/5/2013
NonStop Extensions for
Java
• JToolkit
To integrate Java modernisations with other NonStop application environments
• JI ( Java Infrastructure)
Allow Java APIs to use TS/MP functionality
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
JToolkit – Legacy Integration with Java
Supports JPathsend
messages <= 2MB
Pathway
serverclass
JPathsend API
Enscribe
JEnscribe API
18
Pathway
Supports $RECEIVE
serverclass messages <= 2MB
JPathway API
Standalone
Server
processes
JRequester API
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
9
12/5/2013
JI
(Java Infrastructure)
- Motivation for creating it
WHY LEARN NONSTOP API ?
•Why should a java programmer learn NonStop API while java apps are meant to be
platform agnostic
PORTING
•A big chunk of NonStop specific code in NSJSP (port of Apache Tomcat) is interfacing with
$RECEIVE
PROVIDE SOMETHING TO BRIDGE THE Java-NonStop GAP
•Give customers and ISVs something new that will make it easier to exploit NonStop systems
REDUCE EFFORT and COST
•Help more open source products (JBoss etc) to be ported onto NonStop
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Product Overview
Standard Interface
•Provides Java standard face to NonStop API for IPC (Pathsend or FS)
•Use java.net.Socket and java.nio.SocketChannel for Pathsend and FS IPC
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
10
12/5/2013
Advantages
Develop on desktop, Deploy on NonStop
DEVELOPMENT
•The application
can be developed
on desktop using
an IDE of
developer’s choice
•During
development no
NonStop libraries
are needed
TESTING
•The application
can be tested on
desktop using any
of the various
testing frameworks
•Testing too does
not require any
NonStop specific
libraries
DEPLOYMENT
•The application
can be deployed
as a TS/MP server
class
•The application
inherits all the
goodness of the
platform and
TS/MP
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Architecture – Modes of operation
Request-response
• Client initiates data exchange
• Server responds to client request
• Server can ONLY respond to client
message
Bi-directional
• Default mode of operation in JI
• Client and Server can each send data
independent of the other
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
11
12/5/2013
Use cases – NSASJ (JBoss port on TS/MP)
JBoss code modification to run it as TS/MP server class
• NIL
Open Source Component.
Uses JI to act as pathsend
client
JBoss AS. Uses JI to run as
a TS/MP server class
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
JEE functionalities
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
12
12/5/2013
NonStop iTP WebServer
and
NonStop Servlets for
JavaServer Pages
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
Tomcat vs NSJSP
NSJSP: Standard Tomcat, optimized for scale & HA on NonStop
Tomcat Server
Apache Web Server
Web Apps
Web Apps
Web Apps
AJP Protocol
Servlet Container
(Catalina)
NSJSP Environment
iTP Web Server
NSJSP Server
Pathsend
iTP Web
Server
NSJSP
Web Apps
Apps
ServerWeb
Web Apps
Servlet Container
(Catalina)
TS/MP
Serverclasses
26
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
13
12/5/2013
NonStop Servlets for JavaServer Pages (NSJSP)
Value-add port of Apache Tomcat to NonStop
Request
Response
Implemented to leverage
scalability and fault-tolerance
provided by TS/MP
TS/MP
$ZWEB
$YWEB
NSJSP Manager
NSJSP
NSJSP
JVM
CPU 0
27
PATHMON
iTP
WebServer
PATHMON
Can leverage Pathway Domains
feature of TS/MP 2.4 and later
releases to implement an “On-line
Upgrade” configuration of NSJSP.
NSJSP
NSJSP
JVM
JVM
CPU 1
CPU 2
JVM
CPU n
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Parallel TCP/IP Sub System
iTP WebServer and NonStop JSP
SERVLET TS/MP
Serverclass
Server Process
JVM
HTTPD
Server Process
HTTPD
Server Process
TS/MP Load
Balancing
HTTPD
Server Process
HTTPD
NonStop Server
uses TS/MP for management
and PTCP/IP for request routing :
really a replicated application
NSJSP - uses TS/MP for management and initial
© Copyright 2012
Hewlett-Packard
Development(new
Company, L.P.
The information
contained herein is subject to change without notice.
request
routing
Servlet
session)
HTTPD -
29
HTTPD TS/MP
Serverclass
Server Process
Server Process
JVM
Server Process
JVM
Server Process
Server Process
JVM
JVM
TS/MP
& IPC
Handler
User
Classes
Tomcat
Container
Classes
14
12/5/2013
Open Source Java Frameworks on NonStop
–
Enterprise Java on NonStop
NSJ (Java VM)
NSJSP (Tomcat)
•
No changes were required except...
• Special “dialect” file for Hibernate
•
Executes in NSJSP container
•
•
Inherits NonStop fundamentals with NSJSP
Integrate existing NonStop Apps via JToolkit (Java)
•
Fully integrated and tested by HP
•
End user documentation provided
Fully supported by HP
•
MyFaces
Spring MVC
Axis2
Spring WS
Spring
POJO
POJO
JPA/Hibernate
JDBC / JTA
SQL/MX
31
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Where do SASH frameworks fit?
Presentation
Presentation
JEE
Business Logic
BusinessLogic
DataAccess
NonStop
SQL/MX
Services
NSJSP (Tomcat)
Scalable and available
SASH execution container
NonStop TS/MP
NonStop OS
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
15
12/5/2013
Java Ecosystem on HP NonStop
NSJ
SASH
myFaces
Servlets
Hibernate
64-bit
log4j
Axis2
Advanced GC
JMS
EMS appender
Spring
JToolkit
JSF
Eclipse
JPA
JDBC T2
EJB
NSJSP
(Tomcat) JSP
JDBC T4
SPJ
Dynamic JVM languages
NonStop OS
33
©2009 HP Confidential
NSASJ 1.0 = JBOSS EJB Container + Enterprise RAS
Financial
Services
Applications using EJB
Communication
Media
Entertainment
Manufacturing
Healthcare &
Public Sector
JBOSS EJB Container (Value-added port)
Certified Java SE Platform (JDK and JVM)
Clustering
Availability Scalability
NonStop TS/MP
Transactions
NonStop TMF
HP NonStop
NonStop OS
35
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
This is a rolling (up to three year) Roadmap and is subject to change without notice
.
16
12/5/2013
Wildfly(Jboss) and NSASJ Deployments
NSASJ derived from JBoss AS
JBoss AS Domain Deployment
36
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
Architecture & Components
SERVER
•JEE container and TS/MP serverclass
Post Master
•Transfer data from remote client to
SERVER
•A pathsend client to SERVER
Host Controller
•Management Component
Cache server
•Stores state information (example
stateful beans)
37
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
17
12/5/2013
Integration with NSJSP
NonStop Server
Path
send
NSASJ EJB Container
JI Socket
JI Socket
EJB Client in
Servlet
NSJSP
Container
App EJB
Post Master
Local
Socket
JI
Socket
Path
send
• Two connectivity options shown in figure
− Using JI and without JI
• NSJSP servlet can use the NSASJ provided client library to invoke the EJBs
• Servlet can also integrate with the JTA library provided by NSASJ
− EJB invocations by the servlet are within a transaction context
• If JI used then TMF context too can be propagated to the EJBs
38
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
JTA and TMF Integration
JTA Library
• Integrated with TMF using XARM library
• Seamless integration with TMF
39
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
18
12/5/2013
Integration with NSMQ
NonStop Server
NSASJ EJB Container
App
EJB
NSMQ
Client RA
NSMQ
• NSMQ provides a resource adaptor which can be deployed & configured in NSASJ
• NSASJ instances can be configured to register with NSMQ either as consumer or producer
− Consumers will be Message Driven Beans (MDB)
− SLSB & SFSB will be the producers
• NSMQ parameters can be configured such that inbound messages are evenly distributed across the consumer
NSASJ instances
• Similarly for outbound messages, the NSMQ cluster manager will handle the even distribution
• NSASJ transactions involving NSMQ interactions are part of the global TMF transaction
40
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
NSMQ Architecture and Components
A client application creates and
sends JMS messages.
The Parallel I/O sub system of
TCP/IPv6 balances the load across
the NSMQ brokers
The Cluster of NSMQ brokers.
Messages are stored in SQL/MX
database to prevent data loss.
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
A client application which receives
and processes messages
19
12/5/2013
Scalability
NSMQ brokers are
interconnected to handle
additional load
All the brokers establish a duplex
connection with other brokers
Messages are exchanged
between brokers dynamically
Consumers connected to any of
the broker has access to the
messages residing in any broker.
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Fault Tolerance and High Availability
Every NSMQ broker is run as a
master and slave processes.
The master process caters to the
requests of the clients.
In the event of master failure, the
client takes over the
responsibilities of the master.
The messages are stored in the
SQL/MX database before
dispatch.
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
20
12/5/2013
JEE functionality on NonStop
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Questions Raised
( by Customer Newbies to NonStop )
•
•
•
•
Scaling OLTP for cpus/cores?
The File Loader problem?
Cost of Java compared to COBOL?
Need to learn TS/MP and JToolkit?
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
21
12/5/2013
Questions Answered
• Scaling OLTP for cpus/cores?
JEE …..
iTPWS/NSJSP NSASJ NSMQ all scale by configuration
without programmer code
RYO (Custom) …..
Use Farm/Cluster distributed model
Mapto TS/MP if possible
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Questions Answered
•
The File Loader design problem?
HP accept that for Utility/Batch type programs MPP requires
more design effort due to lack of Globally shared memory.
A trade-off for the benefits of NonStop Architecture with
Continuously Available production OLTP applications.
Techniques such as disc-based context would be recommended.
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
22
12/5/2013
Questions Answered
• Cost of Java compared to COBOL?
Good analysis by them of their test case
Small Benchmarks = OK
Full Applications
- dynamic managed environment vs static data language = NOK
- extensive use of libraries = ?/NOK
JDBC
- should use Prepare/Execute
- MFC helps reduce compilation and start-up costs
- cost of parameter/results marshalling
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Questions Answered
• Cost of Java compared to COBOL?
(Good analysis of test case)
With TS/MP
- JPathway marshalling can be expensive
(They identified this)
- JI should be less expensive
(but Java clients only)
- Java /COBOL/C++ = not technical, a user trade-off
( what they recognised)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
23
12/5/2013
Questions Answered
• Need to learn TS/MP and JToolkit?
JI (again)
But only supports Java Clients !
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Using Java on Integrity NonStop
•
•
•
•
•
The application should be capable of being deployed on a
Architecture is the Key -------------farm/cluster of systems, by either replication of instances or
by distribution of the components of a single instance.
- exploit Clustered IP and TS/MP
All components of the application should be replicable
within or across instances to ensure availability and
JVM Startup hints
scalability of the application.
- Class Loading and .war unpacking
There should be no reliance upon global shared memory
between processor units nor upon sequenced serialization
(WLS could be very bad)
such as allocation of strictly sequential identifiers or rigid
- JDBC preparation, use pools and MFC time-ordered processing.
- artificial warm-up to prime JIT
NSJSP Sessions, avoid if possible, keep small
- MyFaces can be expensive , consider GWT
Review use of libraries
Manage object lifecycles for gc , very short or very long
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
24
12/5/2013
Modern NonStop middleware stack
Communication
Media
Entertainment
Financial
Services
Jtoolkit
NSDEE
NSJSP
(Eclipse)
(Tomcat Servlets)
Healthcare &
Public Sector
Manufacturing
NSASJ
(EJB of JBoss)
NSMQ
SASH
(ActiveMQ)
(Open Source Java)
Cache*
SOAP
All Modern
JI
Certified Java SE Platform (JDK and JVM) (NSJ)
Clustering
Availability
iTP Web Server
Scalability
All Standard
Transactions
NonStop TMF
NonStop TS/MP
All NonStop
NonStop OS
53
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The Full Java Ecosystem now on HP NonStop
Java Infrastructure
the new key to TS/MP for Java
Thank you
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
25