Loading data into CMDB - Best practices for
the entire process
Shivraj Chavan
Anand Ahire
BMC Software
Agenda
Why you should never do CMDB only project
Guidance on – ‘Should this be in the CMDB?’
The Life of a CI
Various best practices
Q&A
© 2013 WWRUG Canada Inc. All Rights Reserved
2
Typical Failed CMDB Project
“We need to have a CMDB”
-
Why? … because
Avoid doing a CMDB
only project
Let’s load data into it
-
What data? … whatever data we have laying around
So, that took a long time!
-
And the CMDB is big, out of date, and isn’t bringing any value
See, I told you that CMDB thing was complex and useless hype
-
Another big data store offering no value is obviously not the desire
© 2013 WWRUG Canada Inc. All Rights Reserved
3
CONSUMERS vs. Providers
Although providers supply the data for the CMDB, the important
players for the CMDB are really the consumers
Consumers do interesting and useful things with the data
Providers simply load data
Without consumers – who cares what data is loaded
-
In fact, if no one consumes the data, it shouldn’t be loaded
© 2013 WWRUG Canada Inc. All Rights Reserved
4
Have an XYZ project, that includes using the CMDB
(for XYZ substitute – Incident, Change, Problem, …)
We need to improve our Change Management process
-
The CMDB is not an end in itself, it is an enabler for other processes
-
You must have a goal and a focus for how you want to USE the CMDB
Change Management needs to know about servers, applications, services, and
their relationships
-
If no one is consuming a piece of data, it should not be in the CMDB
-
When in doubt, DO NOT put data into the CMDB until someone asks for it
Look at the improvements in the Change Management process
-
Failed changes and disruption to service because of change are down
-
I can see how the CMDB makes Change Management better
Let’s look at the Incident Management process; how can we improve?
-
There will be many different XYZ projects that all increase content and use of that
content in the CMDB
The CMDB is a long journey; but there is incremental value at every step
along the way
© 2013 WWRUG Canada Inc. All Rights Reserved
5
Choose your data sources wisely
Good data providers do the following:
-
Provides data for CDM classes you need to populate in the CMDB
-
Provides data that is not already provided by a different data source
-
Can populate attribute values which can uniquely identify CI
-
-
CMDB provides context
NOT content
Periodically updates data
Periodically flags data as no longer present in the environment
Indicates when the data was last updated
Updates, maintains, and deletes relationships as well as CIs
Manual Data entry:
-
Example: Asset Sandbox in ITSM
-
There are some classes we expect to populate manually, like Business
Service
© 2013 WWRUG Canada Inc. All Rights Reserved
6
Automated Discovery is a Requirement
Without automated discovery
processes, data accuracy
CANNOT be maintained
Data is inaccurate before you can complete
loading it
© 2013 WWRUG Canada Inc. All Rights Reserved
7
Value Path
Atrium CMDB
HighValue
Incident, Problem, Change, Config
Services
Applications
Running Software
Virtual Layer: Virtual Machines
Less Value
Physical Layer: Servers, Network Devices
= CI, CI Attributes, CI Relationships Auto maintained by likes of ADDM in Atrium CMDB
= CI, CI Attributes, CI Relationships Maintained by Atrium CMDB
= CI
= Relationship
© 2013 WWRUG Canada Inc. All Rights Reserved
8
The Life of a CI
Transform
Extract
Load
Cleanse and Reconcile
Atrium CMDB
Consume
• Only load data that you
need!
• Define dataset per provider
ADDM
Dataset
ADDM
CIs
SCCM
Dataset
MS SCCM
..
..
..
.
CIs
Atrium
Integrator
IMPORT
Dataset
Any Data
Source
• Have different plan for
Initial vs delta loads
• Run multiple copies of key
steps like CMDBOutput
step in spoon
• Think about error handling
especially for custom jobs
CIs
© 2013 WWRUG Canada Inc. All Rights Reserved
9
The Life of a CI
Extract
Transform
Load
• Normalize before you
Identify
• Don’t normalize all classes
• Batch mode – initial or large
data, Continuous – steady
state
Product
Catalog
ADDM
Dataset
CIs
SCCM
Dataset
..
.
IMPORT
Dataset
• Use Suite Rollup / Version
rollup for SWLM
• Always use Reconciliation,
even for a single source
Atrium CMDB
..
CIs
.
• Use Impact Normalization
for Change Mgmt or BPPM
Consume
Cleanse and Reconcile
CIs
N
O
R
M
A
L
I
Z
A
T
I
O
N
R
E
C
O
N
C
I
L
I
A
T
I
O
N
• Keep your data clean,
normalized, and identified
• Use qualifications to filter
data
Production
Dataset
• Use Standard Identification
and Merge Rules
• Put your most specific
identification rule first
© 2013 WWRUG Canada Inc. All Rights Reserved
10
The Life of a CI
Extract
Transform
Load
Cleanse and Reconcile
Consume
• Do not modify data in
production dataset directly.
Atrium CMDB
• Always use sandbox
datasets for manual
changes
ITSM
SIM
• If no one consumes the
data, it shouldn’t be loaded
• Periodically check for
duplicates and take
remediation action
ITBM
Production
Dataset
Dashboards
..
..
BPPM
© 2013 WWRUG Canada Inc. All Rights Reserved
11
The Life of a CI
Transform
Extract
Load
Consume
Cleanse and Reconcile
Atrium CMDB
CIs
CIs
SCCM
Dataset
MS SCCM
..
..
..
.
Product
Catalog
ADDM
Dataset
ADDM
CIs
Atrium
Integrator
..
CIs
.
..
.
IMPORT
Dataset
Any Data
Source
CIs
CIs
N
O
R
M
A
L
I
Z
A
T
I
O
N
R
E
C
O
N
C
I
L
I
A
T
I
O
N
ITSM
SIM
ITBM
Production
Dataset
Dashboards
..
..
BPPM
© 2013 WWRUG Canada Inc. All Rights Reserved
12
Normalization and Reconciliation example
Normalized Data
Data Source 1
Host Name: John Smith Laptop
Model: MB134B/A
Software: MSWord
Version: 2004
Database
Web
Services
Host Name: John Smith Laptop
Model: Apple MacBook Pro 15"
Software: Microsoft Word
Version: 2004
Data Source 2
Host Name: John Smith Laptop
Model: Apple MacBook Pro 15"
Software: MSWD
Version: 11.3.8
Host Name: John Smith Laptop
Model: Apple MacBook Pro 15"
Software: Microsoft Word
Version: 11.3.8
Reconciled Data
Host Name: John Smith Laptop
Model: Apple MacBook Pro 15“
Software: Microsoft Word
Version: 11.3.8
Atrium CMDB
Production Dataset
© 2013 WWRUG Canada Inc. All Rights Reserved
13
Performance considerations
Establish an Integration Server
In many cases when performance is an issue, poor database
configuration and / or indexing is the cause
Consider indexing attributes used in Identification rules
Check query plans, review and correct them
Are DB backups happening when Reconciliation jobs are running?
Use qualifications whenever possible to filter your data
“Fine tune” thread settings and use Private Queue
© 2013 WWRUG Canada Inc. All Rights Reserved
14
Summary
Don’t do standalone CMDB project, CMDB is a means to ends
Approach CMDB project from consumer side not provider
Don’t boil the ocean
-
Start small, prove value and iterate
-
but there is incremental value at every step along the way
Normalize before you reconcile
Always reconcile and use sandbox for manual editing
Service orientation is where real value lies; model services NOW
© 2013 WWRUG Canada Inc. All Rights Reserved
15
Q&A
Anand Ahire
Principal Product Manager – Atrium Core
anand_ahire@bmc.com
© 2013 WWRUG Canada Inc. All Rights Reserved
16
You are Allowed to Extend the CDM – BUT DON’T
Do EVERYTHING possible to design using the CMDB default data
model
-
There is a mapping paper on the web site to help with mapping decisions
-
https://communities.bmc.com/docs/DOC-16471
If there is a request to extend, really evaluate whether there is
really no existing class that it would be appropriate to map things
into
If you do extend the model, make sure you follow best practices
-
Model for the CONSUMER not the provider
-
Add as few extensions as possible
-
Consider that not all consumers can see a new class
© 2013 WWRUG Canada Inc. All Rights Reserved
17
References
Hardware Requirements and Sizing – Documentation
Best Practices for CMDB Design & Architecture – Webinar
What CIs should I push into my CMDB? – Documentation
Understanding Atrium Integrator – Webinar
Understanding Normalization and the Product Catalog – Webinar
Importing custom Product Catalog data – Documentation
Understanding Reconciliation – Webinar
Common Data Model and mapping data to CMDB –
Documentation
Fine tuning ARS for CMDB applications like NE, RE, etc. – KA
https://docs.bmc.com/docs/display/public/ac81/Investigating+CM
DB+Data+Issues
© 2013 WWRUG Canada Inc. All Rights Reserved
18