Agile Enterprise Data Management:
Terms, Models, Universal
Warehouses
Phasic Systems Inc
www.phasicsystemsinc.com
703-945-1378
2
Phasic Systems Inc Management
• Geoffrey Malafsky, Ph.D, Founder and CEO
▫ Research scientist
▫ Supported many organizations in their quest to access the right
information at the right time
• Tim Traverso, SVP Federal
▫ Former Technical Director, Navy Deputy CIO
• Kevin Moran VADM (Ret), COO
▫ Former Deputy Chief of Naval Personnel
• Marshall Maglothin, SVP HealthCare
• Deborah Malafsky SVP Business Development
3
Our Agile Methods
• Why be Agile?
▫ Provide flexibility and adaptability to changing business needs while
maintaining accuracy and commonality
▫ Segmented approach is too slow, rigid, and costly
• How?
▫ Treat data lifecycle as one continuous operation from governance to
modeling to integration to warehouses to Business Intelligence
▫ Emphasize value produced at each step and overall coordination
▫ Seamlessly fit with existing organization, procedures, tools but add Agility,
commonality, flexibility, and reduced cost and time
• We are Agile and comprehensive
▫ Typical 60-90 day engagement
▫ Deliver completed products not just plans or partial results
4
Methods and Tools
• DataStar Discovery: Agile system modeling, data modeling, data semantics
▫ Policy, governance, standards and design
▫ Fully integrated metadata: each element, table includes attributes for
business, IT, and BI
▫ Point-select data models, codes, rules, in Data Lifecycle Model
▫ Easily build common term standards using all variations with clear
understanding of business use scope, meaning
▫ Rapid collaborative agreement; line-of-sight to operations
• DataStar Unifier: Agile Implementation
▫ Agile warehousing and aggregation
▫ Simplified, common semantics using Corporate NoSQL™
▫ Aggregate data using all use case and system variations simply and easily
into standard or NoSQL databases
▫ Universal warehouse with flexibility, extensibility, adaptability
5
PSI Customer Testimonial: VADM (ret) J. “Kevin” Moran
“As a COO of a Wall Street firm and a former US Navy
Vice Admiral in charge of a large integrated organization of
thousands of people and numerous IT systems, I have seen
firsthand the critical role that high-quality enterprise data plays
in day-to-day operations of an organization. Without timely
access to reliable and trusted data all of our operations were
vulnerable to poor decision making, weak performance, and a
failure to compete. With Phasic Systems Inc.’s agile
methodology and technology, we were finally able to solve our
data challenges at a fraction of the time, cost, and
organizational turmoil that all the previous and more
expensive, time-consuming approaches failed to do.”
6
Agility Across the Data Life Cycle
Traditional
►Slow (9-24
months)
► Fractious
(business,
IT, BI)
► Expensive
► Inflexible
► Poor practical
semantic mapping
Agile EDM
►Fast (1-3 months)
► Coordinated
(business, IT, BI)
► Large decrease $
► Flexible (daily)
► Easy, adaptive
semantic mapping
Products
►Data models (all)
► Unifying, concise
Data System Model
► Glossaries
► Codes
►Semantic
mapping (data, XML,
Legacy, SOA)
► Feed MDM, BI
► Universal Data
Warehouse
►Managed
governance –
integration - analytics
7
Agile: Overcome Hurdles
• Group rivalry
▫ Embrace important business variations; recognize no valid reason
to force everyone to use only one view exclusively.
• Terminology confusion
▫ Use a guided framework of well-known concepts to rapidly identify,
and implement variations as related entities.
• Poor knowledge sharing
▫ Use integrated metadata where important products (business
models, data models, glossaries, code lists, and integration rules)
are visible, coordinated, and referenceable
• Inflexible designs
▫ Use a hybrid approach (Corporate NoSQL™) for Agile
warehousing and integration blending traditional tables and
NoSQL for its immense flexibility and inherent speed
Data Meaning Not Just Metadata
Governance
Integration
Design
MDM
CEO/CFO/CIO
Sales,
Accounting
Which Value? Whose?
SAP/IBM/ORACLE
Ontologies
My “customer” or your “customer”?
How is data used?
Must be agile in order to adapt quickly to new business needs
▫ Continuous change is norm: requirements, consolidation
▫ We must use all the important business variations of key terms (e.g.
account, client, policy) – No such thing as single version for all!
9
Data System Model
10
Real Estate Listing Example
• Seems simple and well-defined
▫ Each house has a type, id, address, etc..
▫ Industry standards: OSCRE, RETS
• Yet, data systems are very different
▫ Data model tied tightly to business workflow
▫ Extensions and “make-it-work” changes added over time
• Similar to customer relationship mgmt, ERP, and many
other fields
11
Main Hurdle to Enterprise Data Standards and Warehouse: Data Values
Not Synchronized with Metadata
Different Meanings (Legal and
Business Activities)
NKY
HomeSeekers
Texas
12
13
Fully Integrated Metadata for Business, IT, and BI
14
15
16
Corporate NoSQL™
• NoSQL gives large systems flexibility & high performance
▫ Simple key-value pairs: little data modeling
▫ Inherent hardware performance: no database joins (10000x faster)
▫ But, poorly suited for corporate use – lacks connection to business
• Corporate NoSQLTM
▫
▫
▫
▫
▫
▫
Blends traditional techniques and NoSQL
Tables provide direct alignment to business concepts
Key-value pairs eliminate need to delineate every attribute
Business driven terminology in production model
Easily handles semantic variations
Updates do not require changes to data models or physical stores
17
Position Data Model
Results
• Applied to production data:
▫ Fully cleaned & integrated data governance approved
 Requirement: 500,000 records in 2 hrs on Sun E25K
 Actual: 50 minutes on 3 year low-cost server
• Governance documents produced and approved
▫ Legacy data models – first time in ten years
▫ Common data model – directly derived from ontology.
Position-Resume model
• Standing governance board created with short decisionmaking monthly meetings
▫ Position-Resume Governance Board
• Process approach and technology applied to new IT
systems