Test & Evaluation/Science & Technology Program
Net-Centric Systems Test (NST)
Test Technology Area
Virtual Management Information Base (VMIB) Overview
Scott Oppenlander (Principal Investigator)
March 2012
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Acknowledgement
This project is funded by the Test Resource Management Center (TRMC) Test
and Evaluation/Science & Technology (T&E/S&T) Program through the U.S.
Army Program Executive Office for Simulation, Training and Instrumentation
(PEO STRI) under contract number W900KK-10-C-0022.
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
2
What is the business problem?
•
The emphasis on development and integration of net centric programs has resulted
in diverse and interrelated web services to test and troubleshoot.
•
Debugging complex SOA environments requires costly and scarce on-site subject
matter experts (SME).
•
Test range time with live assets is expensive and scheduled months in advance.
Time spent investigating web service failures instead of testing interoperability with
live assets is wasted funding.
•
The additional time required to debug complex SOA environments directly drives the
schedule and budget overruns when deploying a system. This leads to delays in
providing the warfighter with the tools required to accomplish their mission.
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
3
Example – Document Retrieval
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
4
Pitfalls of net-centricity
•
Applications that invoke web services must be written to gracefully handle all
possible return values and exceptions.
•
Target web services must be written both to gracefully handle all conditions as well
as to return meaningful information in the case of exception conditions.
•
Chained web services (i.e. web services that themselves invoke web services) do
not return the root cause of failure
•
Different web services are frequently coded and maintained by different vendors and
agencies, each with variable coding practices, support procedures, opacity, and
general helpfulness.
•
Susceptible to time synchronization, server certificate or trust store issues.
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
5
Consequence of Failure
org.mc4j.ems.connection.EmsInvocationException: Exception on invocation of
[findMonitorDeadlockedThreads]org.mc4j.ems.connection.EmsConnectException: Connection failure null
at org.mc4j.ems.impl.jmx.connection.bean.operation.DOperation.invoke(DOperation.java:127)
at org.rhq.plugins.jmx.MBeanResourceComponent.invokeOperation(MBeanResourceComponent.java:547)
at org.rhq.plugins.jmx.MBeanResourceComponent.invokeOperation(MBeanResourceComponent.java:517)
at org.rhq.plugins.jmx.ThreadDataMeasurementComponent.invokeOperation(ThreadDataMeasurementComponent.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:525)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: MBeanException: null Cause: javax.management.InstanceNotFoundException: java.lang:type=Threading is not registered.
at org.jboss.mx.interceptor.ReflectedDispatcher.handleInvocationExceptions(ReflectedDispatcher.java:180)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:163)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
... 3 more
Caused by: javax.management.InstanceNotFoundException: java.lang:type=Threading is not registered.
at org.jboss.mx.server.registry.BasicMBeanRegistry.get(BasicMBeanRegistry.java:523)
at org.jboss.mx.server.registry.BasicMBeanRegistry.getValue(BasicMBeanRegistry.java:561)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
... 48 more
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
6
How can VMIB help?
• Visibility inside the encryption boundary to examine messages
• Detection and diagnosis of failure in chained web services
• Open interface allows “bring your own” analytics
• Simple to install and back out
• Pairing of SOAP requests and responses
• No code or service endpoint modifications required. No proxy
agents to configure.
• Toolkit allows users to write custom instrumentation
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
7
Chained web services
JVM
SO
AP
re
sp
on
se
VMIB agent
VMIB agent
SOAP response
AP
SO
JVM
se
on
sp
JVM
re
VMIB agent
VMIB agent
JVM
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
8
Installation
• Shut down JVM
• Copy VMIB probe .jar files
VMIB client
• Update $JAVA_OPTS
• Restart JVM
VMIB probe
JVM
• Deploy VMIB client application
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
9
Concept of Operations
Other analytics
HSQLDB
(in-memory
database)
VMIB client
application
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Other analytics
VMIB
10
VMIB Components
• Probe – captures information about classes and exceptions as
specified on the command line. The JVM under test is modified in
memory by the VMIB probe to provide instrumentation.
• Log4VMIB – a small, high speed bridge class that accepts data from
the VMIB probe and inserts it into the HSQL database.
• HSQL (hyper structured query language database) – a small, high
performance, in memory database that acts as a buffer between the
probe and the agent.
• Agent – an application that extracts instrumentation from the HSQL
database and publishes it via web service.
• Hub – VMIB analytics and visualization on a separate server.
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
11
Supported Platforms and Protocols
• Java 5 and 6
• JBoss-WS 4.2.2
• JAX-WS 2.1
• Apache CXF 2.2 and 2.3
• Weblogic 10.3
• SOAP
• JSON
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
12
Future Features and Issues
• Disconnected operations
• Limited bandwidth
• Prioritization of traffic
• Bandwidth throttling
• Tiered hubs
• Address accreditation questions
• Investigate automated generation of instrumentation
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
13
Contact Information
Scott Oppenlander
soppenlander@lce.com
(214) 766-1804
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
14
Questions
?
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
15