© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
Business value with patterns
Questions to be answered:
1. Can patterns deliver business
value (“quality at speed”) for you?
2. “Data model” and “Data Vault”
patterns
 How can benefits be realised?
 Are benefits diminished or increased
if you adopt both?
3. Can they support your IM strategy?
1
Agenda
1. A challenging case study
(if they can succeed, your chances are pretty good!)
2.
3.
4.
5.
6.
7.
Data Model Patterns – claims & applicability
“Doc, I just don’t feel well” – need for diagnosis
The diagnosis results, and a “get-well” plan
Data Vault – why, what, & (some) how
“Universal Data Vault” – combining patterns
Hints & helpers for you
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
2
© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
1. A challenging case study
(if they can succeed, your chances are pretty good!)
2. Data Model Patterns – claims & applicability
3. “Doc, I just don’t feel well” – need for diagnosis
4. The diagnosis results, and a “get-well” plan
5. Data Vault – why, what, & (some) how
6. “Universal Data Vault” – combining patterns
7. Hints & helpers for you
3
Special Needs Assistant/Practitioner
Regulation (SNAPR)





Registration, renewal
Random spot-audits
Complaints
Offence case management
Monitoring of conditions
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
4
Pre-nationalisation nightmare
 83 management organisations
(By profession &/or state)
 One person, multiple professions
 Career (e.g. Home Care  Rehab  Teacher)
 Concurrent
 Plus student training records
 Complaints potentially across all above
 Risks e.g.
 Deregistered by one, “problem” relocates
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
5
Simple (!!!) solution
– one national organisation!
 Federal Act of parliament
 Delegation
 Closure of 83 organisations on one day
 National commencement of SNAPR the next day
 Wonderful people, a massive effort, but …
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
6
A day in the life …:
- Receipt of a complaint
 Any prior complaints against this person?
 Prior organisations
• Several of the 83 !
• Aging platforms, licences, knowledge workers
 Since SNAPR formed, now got 84th data source!
 Plus all of the student records at multiple institutes
 Ask all of the above for prior professions, if any
 Possible conflicts of interest
… and that’s for just one the complaint
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
7
Information Management (IM)
problems we all may face
 IM challenges – the usual suspects
 Data owner / custodian / steward = ?
 Data currency
 Data accuracy
 Data accessibility
 Audit (including reading the data!)
 etc.
 For highlight on data integration woes
… but an out-of-date IM strategy wasn’t helping
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
8
Exercise: What if you’re the CIO?
A guess at an Info. Mgt. solution set?
Consider:
 Top-down versus bottom-up
 Strategic versus tactical
 Rebuild versus refactor
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
9
© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
1. A challenging case study
2. Data Model Patterns – claims & applicability
 The “sales pitch” for data model patterns
 Applicability for you
• How big is your organisation?
• How wide is your project’s scope?
• What parts of your architecture can be pattern-based?
3. “Doc, I just don’t feel well” – need for diagnosis
4. The diagnosis results, and a “get-well” plan
5. Data Vault – why, what, & (some) how
6. “Universal Data Vault” – combining patterns
7. Hints & helpers for you
10
The sales pitch for data model
patterns
“… using simpler and more generic models, we will find they stand the test of time better,
are cheaper to implement and maintain, and often cater to changes in the business not
known about initially” (Data Model Patterns by David Hay)
“Strict modeling of the real world leads to a system that reflects today’s realities but
not necessarily tomorrow’s. The abstractions that emerge during design are key to
making a design flexible” (Design Patterns by Gamma, Helm, Johnson & Vlissides)
“… simple models are always worth the effort. … they make things easier to build,
but more importantly they make them easier to maintain and extend in the future.
That’s why it’s worth replacing software that works with simpler software that also works.”
(Analysis Patterns by Martin Fowler)
Summary: Flexibility, cost effectiveness, adaptability (for today & tomorrow)
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
11
Might patterns work for you?
- Organisation size
 Continuum of large-to-small organisation sizes
 International not-for-profit (local, but generic)
 Major Australian civil engineering
 2nd-tier bank
 Telecommunications subsidiary(VPN niche)
 2-person conference management
 A third-world company that runs on Excel?!
 … and it works – Several of these clients
volunteered comments for The Nimble Elephant
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
12
Might patterns work for you?
- Initiative size
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
13
Might patterns work for you?
- Architectural areas
Enterprise
logical
data model
Application
logical
data model
Relational
database
COTS
package
-Data Warehouse design
-Services’ architecture
-Services’ XML
-Master data management
-Reference data management
-Business rules vocabulary
-OO class design
-etc.
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
14
© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
1. A challenging case study
2. Data Model Patterns – claims & applicability
3. “Doc, I just don’t feel well” – need for diagnosis
 Embarrassing questions
 The plan for getting diagnostics
 How long to get test results?
4. The diagnosis results, and a “get-well” plan
5. Data Vault – why, what, & (some) how
6. “Universal Data Vault” – combining patterns
7. Hints & helpers for you
15
Embarrassing questions
- When “business” isn’t happy
 Why is it so hard to answer a simple question?
 Why are there so many definitions for core
concepts?
 Why do we get different answers to the same
question?
 Across different reports
 Worse still, when we re-run the same report
 Why does IT/BI cost so much?
 Why, why, why!
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
16
Scheduling a diagnosis
SNAPR plan:
1. Develop a pattern-based Enterprise Logical
Data Model (ELDM)
2. Develop an updated IM strategy
(Based on the ELDM)
3. Act on recommendations
(whatever they might be!)
… so let’s look at Step 1 
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
17
ELDM timing expectations:
- How long to get test results?
 Urgent mobilisation of strategy
 ELDM
 Ideally first, but no time
 In parallel, but framework needed in 5 days
 ELDM “project plan”
 Start-of-week: Familiarisation
 Thursday: Formal “pattern” training (IT & business)
 Friday: IT & business develop the “framework!!!
 (then a few weeks to document, refine, …)
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
18
A palette of “Assembly patterns”
Account
Task
Agreement
Resource
/ Asset
Document
Product
Event
Party & Role
Location
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
19
(One) Drill-down for “Party”
subclassification
0..*
For example:
-Employment
-Marriage
-Ownership
-etc.
PartyToPartyRelationshipType
partyToPartyRelationshipTypeCode
partyToPartyRelationshipTypeDescription
participant1Role
participant2Role
0..*
0..*
partyTypeCode
partyTypeDescription
classification
0..*
Party
1
participant2
effectivePeriod
nameDisplayString
fulfilment
0..*
1
roleTypeCode
roleTypeDescription
classification
0..*
Role
(link)
(link)
effectivePeriod
status
Organization
PartyIdentifier
effectivePeriod
partyIdentifierType
partyIdentifierValue 1..*
gender
containment
containment
1
1
1..*
containment
PersonName
effectivePeriod
personNameType
salutation
familyName
postNominal
OrganizationName
1
0..*
PersonGivenName
givenName
sequenceNumber
effectivePeriod
orgNameType
orgName
1
Address
effectivePeriod
addressDisplayString
0..*
Person
Role's Status can be:
-Planned
-Actual
1
1
1
identification
0..1
RoleType
PartyToPartyRelationship
effectivePeriod
subclassification
0..*
0..1
1
participant1
[Classification mechanism
typically applied to
Organizations, not to
Persons.]
PartyType
0..*
classification
1
0..1
0..*
0..*
AddressService
PhysicalAddress
effectivePeriod
addressUsageType
flatIdentifier
floorIdentifier
propertyName
locationDescription
streetNumber
streetName
town
state
postcode
country
PhoneNumber
countryPrefix
areaPrefix
localNumber
Emailddress
emailAddressString
PostalAddress
postalDeliveryDescription
town
state
postcode
country
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
20
Exercise: Develop a framework
(SNAPR: 1 day). You get ¼ hour!
1. Regist’n, renewal
2. Complaints
3. Random
spot-audits
4. Offence case mgt. Task
5. Monitoring
of conditions Resource
Account
copy
filed
as
constrained
by
Agreement
Document
/ Asset
sited
at
participates in,
signatory to, ...
Product
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
Event
Party &
Role
Location
21
2
A framework
appears
Roles can include:
-Licensed services provider
-Student
-Complainant
-Assessor
-etc.
relates to
responsible for
Service provider details
extended by recording of
competencies (languages,
qualifications, etc.)
Party & Role
adjusts balance of
Account
participates in
relates to
participates in
participates in
contains,
depends on
triggers
relates to
produces
Triggering
Event
milestone noted as
constrains
Activity
(calendar
entry)
Examples include:
-Payment for service
-Applic’n for Regist’n
-Applic’n for Renewal
-Lodgement of Complaint
-etc.
referenced in
Agreement
has as outcome
Examples include:
-Registration
-Restriction
-Endorsement
-etc.
documentary evidence
documentary evidence
relates to
documentary evidence
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
Document
22
© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
1.
2.
3.
4.
A challenging case study
Data Model Patterns – claims & applicability
“Doc, I just don’t feel well” – need for diagnosis
The diagnosis results, and a “get-well” plan
 The pain-point results come in
 A recommendation emerges
5. Data Vault – why, what, & (some) how
6. “Universal Data Vault” – combining patterns
7. Hints & helpers for you
23
The pain-point results come in
 Party/Role profiles needed, but can’t be
delivered
 Increasing pressures for BI integration
 …. and the Data Marts were too successful!
 Increasing pressures for operational integration
 Legacy is dragging SNAPR down
 …
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
24
(Revisiting the Diagnosis plan:)
1. Develop a pattern-based Enterprise Logical
Data Model (ELDM)
2. Develop an updated IM strategy
(Based on the ELDM)
3. Act on recommendations
(whatever they might be!)
… now let’s look at Step 2’s recommendations 
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
25
A recommendation emerges
 EDW (in addition to Data Marts) to address:
 Data Mart growth
 BI (cross-Data Mart) integration
 Single repository to collate
• legacy (83 pre-nationalisation)
• SNAPR (post-nationalisation)
 EDW + ESB/SOA to address:
 Operational integration (with EDW as one source)
 ELDM’s Party/Role pattern to be applied to all
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
26
© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
1.
2.
3.
4.
5.
A challenging case study
Data Model Patterns – claims & applicability
“Doc, I just don’t feel well” – need for diagnosis
The diagnosis results, and a “get-well” plan
Data Vault – why, what, & (some) how
 Business drivers for DV
 What is Data Vault (DV)?
 DV is flexible
… that’s good news
… & it’s “not-so-bad” news, too
6. “Universal Data Vault” – combining patterns
7. Hints & helpers for you
27
The sales pitch for the Data Vault
“pattern”




Auditability
Scalability
Resilience to change
Dan Linstedt: “All the data, all the time”
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
28
DV in context
Operational
source systems
Data Marts
“Client” (Carer)
Relationship
Manager system
Complaints
“cube”
Registration
system
 Data Marts
(Kimball)
 “3NF” (Inmon)
 Anchor Modeling
(Rӧnnbӓck)
 Data Vault
(Linstedt)
Finance
“cube”
Complaint
system
(etc.)
(etc.)
Operational
source systems
Data Marts
“Client” (Carer)
Relationship
Manager system
Complaints
“cube”
Registration
system
Complaint
system
Enterprise Data
Warehouse
Integration Layer
Finance
“cube”
(etc.)
(etc.)
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
29
What is DV? Start with operational
“Normal”(ised) view
Employee
PK
EmployeeNumber
EmployeeName
WorkPhoneNumber
Mobile1
Mobile2
UserId
Salary
Care Worker managed by
assigned Employee
CareWorker
PK
CareWorkerNumber
FK1
CareWorkerName
Address
PhoneNumber
Mobile
EmployeeNumber
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
30
What is DV?
- simply a direct transposition?
Hub_Employee
Hub_CareWorker
PK
Emp_HubId
PK
CW_HubId
U1
EmployeeNumber
LoadDateTime
RecordSource
U1
CareWorkerNumber
LoadDateTime
RecordSource
Link_CareWorker_Employee
Sat_Employee
PK,FK1
PK
Emp_HubId
LoadDateTime
PK
CW_Emp_LinkId
FK1,U1
FK2,U1
CW_HubId
Emp_HubId
LoadDateTime
RecordSource
Sat_CareWorker
PK,FK1
PK
EmployeeName
WorkPhoneNumber
Mobile1
Mobile2
UserId
Salary
RecordSource
CW_HubId
LoadDateTime
CareWorkerName
Address
PhoneNumber
Mobile
RecordSource
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
31
It’s that simple, really.
Country Endeavours
“Creative Solutions for Difficult Problems”
©
© 2014
D C Comics – Time Warner
32
What’s “Conceptual”
Party - Simplified
name
address
phone
1 fulfilment
0..*
Role
Employee
Customer
effectivePeriod
name
address
phone
salary
superannuationFund
effectivePeriod
name
address
phone
creditLimit
industryCode
effectivePeriod
EmployeeRole
salary
superannuationFund
CustomerRole
creditLimit
industryCode
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
33
How: DV is flexible – it gives you
choices (that you may have to make!)
 Split Satellites (by source, churn, …)
 Same-as-Links e.g. for merging “concepts”
 Granularity of Links
 Effectivity Satellites (temporal, bi-temporal)
 Reference data as Hubs
 SNAPR wanted audit on ESB/SOA data reads!
 Raw DV vs. Business DV …
(and Generalisation / Specialisation) 
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
34
© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
1.
2.
3.
4.
5.
6.
A challenging case study
Data Model Patterns – claims & applicability
“Doc, I just don’t feel well” – need for diagnosis
The diagnosis results, and a “get-well” plan
Data Vault – why, what, & (some) how
“Universal Data Vault” – combining patterns
 Why would you combine the patterns?
 Cancelling or complementary?
 … and what does UDV look like?
7. Hints & helpers for you
35
Business-speak continuum
Frequency-of-usage continuum
10
9
8
7
6
5
4
3
2
1
0
Thing
Agreement, Role,
…
Agreed
Restriction,
Agreed
Endorsement, …
Gender-related
Restriction, Agerelated
restriction, …
Supervised-only
Gender-related
Restriction, …
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
[Concept courtesy Hans Hultgren]
36
Logical model continuum
sub-classification
0..*
0..1
RoleType
roleTypeCode
roleTypeName
roleTypeDescription
Party::Party
nameDisplayString
effectivePeriod
domain identification
0..*
1
RoleStatusType
1 classification
fulfilment
roleStatusTypeCode
roleStatusTypeName
roleStatusTypeDescription
1
0..*
participant 1 as role
0..*
participant 2 as role
0..*
0..*
1
classification
status recording
1
1
Role
effectivePeriod
1
0..*
PartyRelationships::PartyRoleInterrelationship
0..*
RoleStatus
effectivePeriod
effectivePeriod
CareWorker
RegisteredTrainingOrganisation
Teacher
RehabProfessional
HomeCarer
Complainant
CareRecipient
Employee
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
37
A subset (!!!) of the
“Generalisation” problem
Agreement
Role
 Employee
 Care Worker
•
•
•
Teacher
Rehab
Home carer
 Care Recipient
 Supervisor
 Licence
•
•
•
Teaching Licence
Rehab Licence
Home care Licence
 Restriction
•
•
Teaching Restriction
(etc.)
 Endorsement
•
•
Teaching Endorse’t
(etc.)
 Supervision
•
•
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
Teaching Supervis’n
(etc.)
Event
 Application
•
•




Initial
Renewal
Audit
Complaint
Offence
Monitoring
Activities e.g.
 Preliminary review
 Suitability review
•
•
•
Identity Check
Health Check
(etc.)
38
Exercise: Design some queries
1. List all Care Workers who have ever had a
Complaint
2. Identify all Roles filled by John Smith
(employee, board member, teacher, home
carer, …)
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
39
A clash of Pattern cultures?
Christopher
Alexander
Hay,
Silverston
Nimble
elephants?!
Linstedt,
(etc.)
Kimball
Inmon
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
40
Case study solution:
– “Universal Data Vault”
… let the credits roll:
Hay,
Silverston
Lin ste d t,
(e tc.)
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
41
Standard (specialised) DV model
sample
H_Teacher
H_TeachingRestriction
PK
H_Teacher_SID
U1
TeacherRegistrationNumber
Load DateTime
RecordSource
H_TeachingRestriction_SID
U1
RestrictionReferenceNumber
LoadDateTime
RecordSource
L_Teacher_Restriction
S_Teacher
PK,FK1
PK
PK
H_Teacher_SID
LoadDateTime
TeacherName
WorkPhoneNumber
Mobile1
Mobile2
RecordSource
PK
L_Teacher_Restriction_SID
FK1,U1
FK2,U1
H_Teacher_SID
H_TeachingRestriction_SID
LoadDateTime
RecordSource
S_TeachingRestriction
PK,FK1
PK
S_TeachingRestriction_SID
LoadDateTime
RestrictionFromDate
RestrictionToDate
SupervisionIndicator
RecordSource
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
42
SNAPR: “Universal Data Vault” as
a DV extension
RoleType
PK
RoleTypeCode
FK1
RoleTypeName
RoleTypeDescription
AbstractOrConcrete
ParentRoleTypeCode
AgreementType
PK
AgreementTypeCode
FK1
AgreementTypeName
AgreementTypeDescription
AbstractOrConcrete
ParentAgreementTypeCode
RoleToAgreementLinkTypeCode
RoleToAgreementLinkTypeName
RoleToAgreementLinkTypeDescription
H_Role
H_Role_SID
PK
H_Agreement_SID
FK1,U1
U1
RoleTypeCode
RoleBusinessCode
Load DateTime
RecordSource
FK1,U1
U1
AgreementTypeCode
AgreementBusinessCode
LoadDateTime
RecordSource
L_RoleToAgreement
PK,FK1
PK
H_Teacher_SID
LoadDateTime
AgreementToAgreementLinkTypeCode
H_Agreement
PK
S_Teacher
PK
AgreementToAgreementLinkTypeName
AgreementToAgreementLinkTypeDescription
Participant1TypeCode
Participant1TypeName
Participant1TypeDescription
Participant2TypeCode
Participant2TypeName
Participant2TypeDescription
RoleToAgreementLinkType
PK
AgreementToAgreementLinkType
PK
L_RoleToAgreement_SID
FK3,U1
FK1,U1
FK2,U1
RoleToAgreementLinkTypeCode
H_Role_SID
H_Agreement_SID
LoadDateTime
RecordSource
TeacherName
WorkPhoneNumber
Mobile1
Mobile2
“Creative
Solutions for Difficult Problems”
RecordSource
Country Endeavours
© 2014
L_AgreementToAgreement
PK
L_AgreementToAgreement_SID
FK1,U1
FK2,U1
FK3,U1
AgreementToAgreementLinkTypeCode
Participant1_H_Agreement_SID
Participant2_H_Agreement_SID
LoadDateTime
RecordSource
S_TeachingRestriction
PK,FK1
PK
S_TeachingRestriction_SID
LoadDateTime
RestrictionFromDate
RestrictionToDate
SupervisionIndicator
RecordSource
43
Exercise: A Universal Data Vault
for you (or SNAPR)
Roles can include:
-Licensed services provider
-Student
-Complainant
-Assessor
-etc.
relates to
responsible for
Service provider details
extended by recording of
competencies (languages,
qualifications, etc.)
Party & Role
adjusts balance of
Account
relates to
participates in
participates in
participates in
contains,
depends on
triggers
relates to
produces
Triggering
Event
milestone noted as
constrains
Activity
(calendar
entry)
Examples include:
-Payment for service
-Applic’n for Regist’n
-Applic’n for Renewal
-Lodgement of Complaint
-etc.
referenced in
Agreement
has as outcome
Examples include:
-Registration
-Restriction
-Endorsement
-etc.
documentary evidence
documentary evidence
relates to
documentary evidence
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
Document
44
© 2014
Country Endeavours
“Creative Solutions for Difficult Problems”
1.
2.
3.
4.
5.
6.
7.
A challenging case study
Data Model Patterns – claims & applicability
“Doc, I just don’t feel well” – need for diagnosis
The diagnosis results, and a “get-well” plan
Data Vault – why, what, & (some) how
“Universal Data Vault” – combining patterns
Hints & helpers for you
45
Learn the
Data Model patterns
 Recognise the pattern types (elementary,
assembly, integration)
 Get familiar with published patterns
 “Data” – David Hay, Len Silverston, …
 “OO” – Arlow & Neustadt, Fowler, Larman, …
 “Industry” – TM Forum, HL7, …
 Get familiar with patterns & agility
 Integrating the patterns – John Giles
 Building the agile database – Larry Burns
 Extreme Scoping [EDW, BI] – Larissa Moss
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
46
Data Vault observations
 “What’s the catch?”
 Start design with a logical model
 Consider a “Universal Data Vault”
… but as with any variation to the DV standards,
always use a consistent pattern
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
47
Leverage the ELDM
Enterprise
logical
data model
Application
logical
data model
Relational
database
COTS
package
-Data Warehouse design
-Services’ architecture
-Services’ XML
-Master data management
-Reference data management
-Business rules vocabulary
-OO class design
-etc.
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
48
Refactor the enterprise LDM
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
49
Get agile: - The T-model approach
to using the enterprise logical model
 Mile-wide, inch-deep (touch points at least)
 Drill-down only where required
 Iterate within each project, & across projects
Account
High-level logical
subject areas

Agreement Document

Standard logical
assembly patterns

Refined logical
assembly patterns


Party/Role

(etc.)


Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
50
Thanks
… and questions?
Country Endeavours
“Creative Solutions for Difficult Problems”
© 2014
51