Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
XML Validation
Test Suites
with CAMV
NIEM Team, Oracle Public Sector
Technology Introduction – September 2011
Disclaimer Notice
The following is not intended to outline Oracle general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any features
or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 2
Test Suites Introduction
• A key need in working with XML information exchanges,
is developing examples and matching templates
including rules, context, and roles for correctly managing
information exchange content between partners. The
W3C Schema by themselves are not sufficient.
• Learn how to use open source tooling, XPath rules, and
the OASIS CAM templates standard to create automated
testing tools.
• Integrating SQL table lookup validation rules
• The CAMV validation engine can also be used in
production environments to manage live information
exchanges.
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 3
Contents
• Part 1
•
•
•
•
The XML Validation Framework
XPath Rules Techniques
Code lists and SQL lookups
Rules use cases
• Part 2
– Example CAM templates
– CAMV validation examples
– NIEM example test suite
– Elections example test suite
– SQL table database lookups
– Running validations from visual CAM Editor
– Results handling techniques
– Summary
* National Information Exchange Model (NIEM) – see http://www.niem.gov
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 4
Deploy
Requirements
CAMV
XML Validation Framework
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
Architecture and Deployment
XML Validation Framework
Structure
Rules
Code Lists
SQL Query
Template
(compiled)
Database
Lookup
(optional)
Results XML
XML
instance
CAMV
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Java Handler
Errors and
Warnings
Slide 6
CAMV Engine
CAMV Java (OneJar)
JDOM XML
JDBC
Spring
XPath
API
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Error
Handler
Slide 7
CAMV Validation Engine
• Standalone XML validator using XPath rules
• Context aware and declarative rules approach
• Integrated with CAM editor menus for instant testing
• External code list and SQL table lookup support
• Callable via Spring framework or Java API; thread safe
and server container deployable
• Configurable error levels – error, warning
• Java error results object returned
• Embedded validation results attributes in XML results
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 8
CAM Editor / CAMV Environment
Single XML
Test Instance
CAM Editor
XSLT
CAMV
CAM
template
Reports
XML Validation
Result
CAM Editor
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 9
CAMV / ANT Script Environment
CAM
template(s)
CAMV Control
Script XML
Reports
XML Test
Instances
2
ANT
1
XSLT
CAMV
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
XML Validation
Results
Slide 10
CAMV Test Control Script XML (single)
Singleton Test Case Example
<camvTestSuite>
Test case name
<testCase>
<Example-Test-v19-3>
<cxxfile>Test-Template-v19.cxx</cxxfile>
<xmlfile>Test-Instance-v19-3.xml</xmlfile>
Validation Template
<verbose>true</verbose>
Test XML instance
<inlineErrors>true</inlineErrors>
<parameters>
Optional items
<abc>Param1</abc>
<cde>Param2</cde>
</parameters>
</Example-Test-v19-3>
</testCase>
</camvTestSuite>
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 11
CAMV Test Control Script XML (multi-instance)
Multi-Test Folder Example
Test case name
<camvTestSuite>
Validation Template
<testCase>
<Example-Test-With-Folder>
<cxxfile>../samples/templates/EML-330-list-v7.cxx</cxxfile>
<testdir>../samples/XMLsamples/UOCAVA/330</testdir>
<verbose>true</verbose>
<inlineErrors>true</inlineErrors>
<parameters>
<abc>Param1</abc>
<cde>Param2</cde>
</parameters>
</Example-Test-With-Folder>
</testCase>
</camvTestSuite>
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Test XML instances folder
Optional items
Slide 12
Linux GTK ANT Configuration
• Linux distributions (e.g. Ubuntu) has ANT preinstalled in /usr/share/ant/lib
• Download Ant-contrib from
• http://ant-contrib.sourceforge.net/
• Copy ant-contrib jar file to /lib directory inside ANT
installation
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 13
Mac OS ANT Configuration
• Mac OS X 1.7+ “Lion” comes pre-installed - use
command : ant –version and it will self-configure
• Mac OS X 1.6 and earlier – need to install “Xcode”
development software first
• Download Ant-contrib from
• http://ant-contrib.sourceforge.net/
• Copy ant-contrib jar file to /lib directory inside ANT
installation
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 14
ANT – Windows configuration
• Download ANT from
• http://ant.apache.org
• Download Ant-contrib from
• http://ant-contrib.sourceforge.net/
• Copy ant-contrib jar file to /lib directory inside ANT
installation
• Set the ANT_HOME environment variable to the
directory where you installed ANT
• Add %ANT_HOME%/bin folder to executable path
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 15
Windows System Path Settings
1
2
3
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 16
Running CAMV with ANT
• Go to CAMV location on Sourceforge
• http://www.cameditor.org/#CAMV_Testing
• Download CAMV release JAR and Test example ZIP
• Unpack test example from ZIP and put CAMV jar into
folder
• Go to the command line (run cmd)
• Switch to folder where CAMV test suite is located
• ANT runAll
• Output is automatically placed in folder called CAMVTestResults
located within the test instance location folder(s)
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 17
Reviewing Validation Results HTML
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 18
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
XPath Rules Techniques
Quick Syntax Primer
Controlling Scope, Context, Evaluation
Dynamic Structure Components
Useful XPath Resources
XPath Syntax Primer
• XPath is a simple assertion language where
expressions reference components hierarchically
within an XML instance structure – e.g.
• /animals/canines/domestic/dogs/poodle[1]
• /animals/canines/domestic/*
• CAMV uses the XPath v2.0 Java library
• In CAM templates XPath is used extensively to
provide rules and content details for XML structure
components
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 20
XPath Functions
XPath Function sampler
logical
concat(string,string,…)
substring(string,start,len)
string-length(string)
normalize-space(string)
upper-case(string)
contains(string1,string2)
Yes
starts-with(string1,string2)
Yes
not(arg)
Yes
exists(item,item,...)
Yes
count((item,item,...))
/bookstore/book[price>35]/title
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 21
CAM XPath Extensions (most used)
Structure Functions
Content Functions
makeOptional(xpath…)
setNumberMask(xpath,arg)
makeRepeatable(xpath…)
setStringMask(xpath,arg)
makeMandatory(xpath…)
restrictValues(xpath,args)
excludeTree(xpath…)
setDateMask(xpath,arg)
makeNillable(xpath…)
datatype(xpath,arg)
setChoice(xpath…)
setLength(xpath,arg)
setLimit(xpath…)
lookup(xpath,args)
setValue(xpath,arg)
printMessage(xpath,arg)
setNumberRange(xpath,args)
setDefault (xpath,arg)
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 22
CAM templates and XPath expressions
• Within a CAM template XPath expressions are found
in the <as:BusinessUseContext> section of the
template.
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 23
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
Rules Use Cases
Typical Business Needs
Quick Examples
Standard Pattern for Business Rules
• Claim – An English statement of a business rule
that conveys its meaning.
Example: A Chapter MUST have a Title.
• Test – A formal statement of a business rule
that can be parsed and processed by machine.
Example: <Assert test="count(para) &gt;=1">
• Diagnostic – An English statement that identifies and
explains what happened or was found when a Test fails.
Example: Chapter "X" does not have a Title.
• Outcome – Error, Warning, Pass
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 25
Controlling Scope and Context
Condition controls if and when the content rule action applies
Condition can control structure as well; optional, mandatory, exclude
Condition can reference different component(s) for contextual rule actions
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 26
Some more examples
Business Rule: Items with color attributes don't need a tray number—the
color is sufficient for picking the item:
<as:constraint condition="exists(//Item/@color)"
action="makeOptional(//Item/TrayNumber)" />
Business Rule: If order weight exceeds 25 kg, purchase order must specify a
freight carrier:
<as:constraint condition="//Item/@weight > 25"
action="makeMandatory(//Item/FreightHandler)">
Using XPath axis referencing (when things are not adjacent in
hierarchy):
<as:constraint condition="exists(ancestor::Item/@color)"
action="makeOptional(//Item/TrayNumber)" />
* Excerpted from Michael Sorens DEVX CAM article examples
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 27
Useful XPath Resources
• Michael Sorens article on using CAM from DEVX.com
• http://www.devx.com/xml/Article/41150
• Martin Roberts – tutorial on writing CAM rules
• http://merryflame.co.uk/jcam/site/tutorials/CAMTemplateBasics.htm
• XPath tutorial resources
• Search on XPath tutorial
• Books available
• Tools – XPath expression constructors
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 28
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
Code Lists and
SQL Lookups
Introduction to CAM code lists
Advantages
Tools and Hints
Code lists
• CAM lookup() function allows referencing to external XML
lists of values
• Can replace use of restrictValues() static lists
• Database - CAM lookup() function can reference SQL table
for dynamic value checking
• Where to get code lists?
•
•
•
•
Import XSD enumeration schema into CAM lookup lists
Convert UBL Genericode into CAM lookup lists
Dump from database into lookup list XML format
Direct via SQL table lookup
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 30
CAM Code list XML format
Code list name
Code value
Textual
value
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 31
Combining Rules and Code lists
• Real power of code lists and CAM – ability to select code list
values based on XPath rule criteria to handle different partner
configurations, versioning and cross-table validations
• e.g. Ford / Mustang | Chevy / Corvette
• Message version id = 1.6 – use code list XYZ-1-6
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 32
Code lists examples
• Load lookup list XML into CAM editor
• Assign lookup() rule to XML component
• Using SQL lookups – need to define connectivity to database
and the SQL query to perform – we will do this part in the next
section
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 33
Code Lists Summary
• Lookups can be to external XML file instances
• Simple XML format for optimized performance and
readability
• Dynamic runtime control via XPath rules to select lists
to be applied
• Lookup can reference SQL statement for database
content checking
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 34
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
Part 2
Example CAM templates
Running CAMV validations
Results handling techniques
Summary
CAM templates
• Here we introduce CAM templates and understanding
their component sections
• Template is abstraction layer over schema structure
and rules
• Template can be automatically built from existing XSD
schema
• Add additional XPath validation rules
• Compiled templates provide the validation instructions
for the CAMV validation engine
• Using CAMV validations and handling results
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 36
CAM Templates Overview
• Consists of 4 functional sections:
• Header declares properties and parameters for the CAM process to reference
• AssemblyStructure captures the business information exchange structures
• BusinessUseContext provides the rules to apply during validation
• Extension points to the annotations and external lookup tables
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 37
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
Example CAM templates
Examples Reviewed
Quick walk through of important details
Available Test Suite Examples
Experimental SAR Examples
• Two test example templates and XML instances:
• Baseline SAR with just structure and content usage rules,
schema parlance “cardinality, facets and enumerations”
• Baseline SAR advanced – logical business rules added to
check consistency of information and structure components
• Each test set contains examples that pass and those
that fail depending on the template validation applied
OASIS EML V7 – UOCAVA Testing
• Set of test examples for Election Management:
• Both individual test cases and folder based multiple examples
• Each test uses different template and validation scenario
* SAR – Suspicious Activity Report NIEM exchange
* EML – Election Markup Language Standard
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 39
Experimental SAR Overview
Areas of Interest:
-Drugs
-Fraud
-Gangs
-Internet
-Pornography
-Property
-Community
-Weapons/Chemicals
-Prescriptions
-Travel/Immigration
-Terrorism
* SAR – Suspicious Activity Report
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 40
Visual Template Rules Wizard Tools
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 41
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
Running CAMV Validation
Illustrative examples – pass, fail, warnings
Default structure handling
Data type and Content checking
Cross-field validation rules
SQL database lookup
Example Advanced Rules
Business Rule: Providing warning if PurgeDate is earlier than PurgeReviewDate
<as:constraint
condition="//sbl:PrivacyDetails/sbl:PrivacyPurgeDate < //sbl:PrivacyDetails/sbl:PrivacyPurgeReviewDate"
action="printMessage(//sbl:PrivacyDetails/sbl:PrivacyPurgeReviewDate,Warning: Review date not before
purge date)">
<as:annotation/>
</as:constraint>
Business Rule: Location must contain an address or a highway or coordinate set
<as:constraint
condition="not(./nc:LocationAddress) and(not(./nc:LocationHighway))
and(not(./nc:LocationTwoDimensionalGeographicCoordinate))"
action="printMessage(//j:TargetLocation/sbl:Location,Empty Location information)">
<as:annotation>
<as:documentation type="documentation">Example of advanced rule</as:documentation>
</as:annotation>
</as:constraint>
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 43
Default Structure Handling
By default all structure components are required and string content
Content Masks
Structure control
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 44
Running CAMV details
Batch Command Line Execution Window
Viewing Same Template in Visual Editor
From within the Test Suite folder on the command line execute > Ant runALL
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 45
Sample SAR Validation Results HTML
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 46
Installing and using SQL lookup example
Contains test sample
and template
DB lookup sample install folder contents
Run start and stop
database scripts
Simply download and unpack the ZIP archive and drop the
DatasourceExample package into the samples Test Suite folder
Then add the camv-test-DbList.xml to the testsuites folder
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 47
SQL DB configuration details
Modify to match your
local directory
structure
camv-test-DbList.xml contains data source reference and connection configuration
Modify inside dbProps.props file to match
your local directory structure
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 48
Sample SQL
DB Test Run
• Start the
database
instance
(slide 47)
• From within
the Test
Suite folder
on the
command
line execute
> Ant runALL
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 49
DB lookup test results rendered as HTML
SQL DB lookup result
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 50
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
CAM Editor Visual Tools
SQL lookup entry wizard
Run CAMV validation wizard
Visual entry of SQL connection details
Visual review of validation results
CAM template SQL lookup rule definition
This is the myLists.cam example template
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 52
CAM template SQL lookup rule wizard
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 53
Test from CAM Editor validation Run Wizard
Pick test case XML
file
1
Choose JDBC driver
Confirm driver name
2
3
4
jdbc:derby://localhost:1527/firstdb
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Replace default
connection with
actual JDBC string
NOTE: this is the
same information as
contained in the
dbProp.props file
Slide 54
Sample Visual Test Results
SQL lookup result
displayed
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 55
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
More Results Handling
XSLT script post-processing
XML Diff considerations
Regression Testing with CAMV
Middleware Integration
Default XSLT rendering of results
HTML output
Finds CAM
validation
results
attributes
Formats
and outputs
error details
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 57
Post-Processing Tips and Tricks
• XSLT post-processing not only for HTML reporting
• Create response error messaging directly from CAMV output
• Create receipt notification directly from CAMV output
• Can use simple DIFF to compare generic HTML report
output to detect regression test changes
• Rule based checking is superior to simple result content
change testing
• Java API and Spring supports direct integration of CAMV
and returning validation results directly to middleware
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 58
Deploy
Requirements
CAMV
Model
Test<Insert Picture Here>
Data
Build
Exchange
Exchange
Development
Generate
Dictionary
Summary
Review
Next Steps
Future enhancements
Resources and Links
Review and Value Proposition
• Testing and verification of your exchange XML examples
• SQL database lookups and flexible code lists handling
• Business rule checking
• Post-processing options and reporting with XSLT scripts
• Regression testing with batch processing
• Flexible deployment of XML validation framework
• Integration via Spring API to middleware
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 60
What are we working on?
• Improvements
• “CAM for web services” – currently exchange-centric
• Enhancing ANT capabilities and soapUI integration
• New features
• Dictionary integration with NIEM Domain Update Tool
• Evaluation tools for dictionaries and components
• Enhancing XML example generation with synthetic data
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 61
CAMV Summary
• Open source, open public standards based, crossplatform deployment written using Java
• Uses Java APIs, Spring API, JDOM, etc.
• Validation features:
• XPath V2.0 support
• Looks ups
• 20 built-in functions
• Reporting
• Embedded CAMERROR, CAMWARN tags
• Spring API and results set handler
• Optional sysout test result log
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 62
Links and References
DOWNLOADS http://cameditor.org/ #CAMV_Testing
NIEM MATERIALS Oracle NIEM website
http://www.oracle.com/goto/niem
Oracle XMLOrb blog site
http://blogs.oracle.com/xmlorb
Copyright ©2011, Oracle. All rights reserved.
Oracle Draft Materials – Limited Circulation
Slide 63