PG&E Enterprise Information Management (EIM) Strategy
Sendil Thangavelu
Lead Principle, Information Architecture
PG&E
Executive Summary
Current Situation
•Changing Business Landscape: Changing regulations, Customer expectations & Smart Grid
requirements around data will need new capabilities and systems with reduced latencies and time
to market
•Multiple BI systems: Multiple BI systems with data fragmented, light self service foot print, resulting in
inconsistent Information, high TCO and low adoption
•Integration: Point to Point Integrations. Some existing Integration systems are outdated and in some
cases out of support.
•Lack of Best Practices: Metadata, Master Data, Data Governance and holistic Information
Architecture that help drive consistency, reduce delivery cycles and cost are non existent
•Information centric Initiatives: Although related, often initiatives are implemented in silo adding to
cost and fragmentation. Implementing data centric initiatives are proving to be expensive
•Organizational paradigms: Functional group specific business processes and requirements ignore
other cross functional enterprise impacts
•Conclusion: There is significant need to rationalize data management systems and introduce
innovative capabilities.
2
Executive Summary
Recommendations
•Develop holistic Information Management capability: Implement linked, incremental, foundational
information capability that can be leveraged by LOB initiatives. Create additional capabilities in a
phased, business driven manner. Some of the capabilities have a steep learning curve for both IT
and business. Start sooner rather than later.
•Establish Information Management as a priority: With the help of business stakeholders establish that
Information is an Enterprise Asset. Form a Business-IT Steering committee with Senior Management to
prioritize initiatives and track progress
•Consolidate BI capabilities: Identify overlapping and redundant systems and consolidate. Define
standards for future BI Platform
•Move Beyond silo Reporting to Intelligent Enterprise: Implement cradle to grave, pro active Data
Management
•Establish Data Governance: Implement Pragmatic, nimble, purpose driven, repeatable framework
and capability
•Track Accountability: Reduce applications that are developed for short term goals. Track
accountability for lifetime system maintenance cost. Enhance the existing governance structures for
overall DSM.
Next Steps
•Get executive buy in to move forward with Information Management Foundational Phase
•Prioritize Initiatives in Partnership with the Business
•Define a roadmap for a “managed evolution” rather than big bang approach to adding
functionality.
•Show tangible results to the business
3
Scope of EIM capabilities
Key Business Drivers
Business
drivers and
over arching
Principles
Enterprise Information Management
Information Architecture ( Strategies, Models, Standards, Patterns)
Enterprise Semantic Model, Metadata, Lineage
Governance
(Framework, Data Stewardship)
Information
Capabilities
Capability
Enablement
Information Delivery
Management
Data Quality
Management
Enterprise Content Management
Master Data
Management
Data Historian
DW
BI
Visualization
Email
Document
Analytics
Complex Event
Processing
Web
Data Integration
(Extract Transform Load, Service Oriented Architecture, Enterprise Service Bus)
Information Lifecycle, Security and Privacy
(Access, Classifications, Auditing, Protection)
Information Technology Management
(Hardware, Software, Application, Tools, Repositories, Storage)
Information strategies from conceptual value to operational impact
4
‘As-is’ Capabilities and Gap
Key Business Drivers
Business
drivers and
over arching
Principles
Enterprise Information Management
Custom built Legacy
Systems, Packaged
Information
Architecture ( Strategies, Models, Standards,
Patterns)
Apps, Enterprise
14 BI Systems,
Fragmented,
Foundation and
Multiple Technology Stack
migration underway..
Limited Footprint
Enterprise Semantic Model, Metadata, Lineage
Governance
(Framework, Data Stewardship)
Information
Capabilities
Information Delivery
Management
Data Quality
Management
Enterprise Content Management
Master Data
Management
Point to Point interfaces,Out
of support technologies
Data Historian
DW
BI
Visualization
Email
Document
Analytics
Complex Event
Processing
Web
Data Integration
(Extract Transform Load, Service Oriented Architecture, Enterprise Service Bus)
Information Lifecycle, Security and Privacy
(Access, Classifications, Auditing, Protection)
Capability
Enablement
Information Technology Management
(Hardware, Software, Application, Tools, Repositories, Storage)
Infrastructure will
not scale for data
volume growth
and types of
usage
Data stratification based on
usage, value and sensititvity
Information strategies from conceptual value to operational impact
5
As –is Key Information Challenges
Issues
Fragmented and Outdated Integration
• Many interfaces (often redundant)
cause higher costs
Data distributed along fragmented
application landscape
•Inconsistent data
•Cross functional data usage
complex
•Manual reconciliation
processes
•Multiple Data Warehouses
Risks
 Implementation of new functionality
and app interfacing more complex
and expensive
 Difficulty in implementing crossfunctional projects
 Complexity of various point-to-point
interfaces almost not manageable
 Cost and effort increases while
issues continue to persist
 Multitude of interfaces drive
maintenance cost
• Future Cross-Functional or data
intensive business requirements
would be hard to manage or
implement
• Complexity of managing data will
increase significantly with data
intense projects
• With further development of data
complexity, consistency becomes
unmanageable
6
As-is Key Information Challenges
Issues
Multiple Technology stacks
•Not necessarily different capabilities
No footprint of Foundational capabilities
•Master Data Management
•Enterprise Data Quality and
Governance
•Meta Data Management
No Enterprise Information Architecture
New capabilities need to be stood up over time
•Advanced Visualization
•Complex Event Processing
•BI as a Platform with consistent and
complementing capabilities
Risks
•Increased TCO
Licensing costs
Resources
Training
Evolution and upgrades
 Increased project costs due to
Siloed and repeated efforts
 Inconsistencies across projects
 Fragmented & Redundant efforts
 Multiple independent projects to
address similar ‘Information needs’
 No IA governance on these
projects leading to increased costs
and further fragmentation
 Smart Grid will create a data
deluge
 Acquiring, managing and
converting data into actionable,
reliable Information will need these
capabilities to be rolled out in
phases
7
Key Information Challenges
Issues
Organizational Issues
•Project versus Enterprise mind set
•Information intensive Projects
implemented largely based on
outsourced advice.
•Shelf ware of Software products
•Skill set gap/readiness to deploy new
capabilities
•Due to lack of in house skills sourcing
and support model needs to be
evaluated
Risks
 Continue to propagate redundant
projects and assets
 Information is inherently cross
functional as such
 Outsourced advise is a function of
skill sets, not the best solution
8
Future State Information Requirements
Organic Data Volume Growth
Ability to handle a significant increases in the number of operational data
sources and associated data volume
Increasing reliance on data analytics and visualization capabilities due to
significant increases in data volume
Devices with processors and two way communication that will enable
collection of more information, decision making and coordination.
Higher, two way collaboration and business process integration between
users, businesses, individual customers and a variety of technology systems,
resources and intelligent devices.
9
Future State Information Requirements
Business Requirements
Increasing need to move, secure, analyze and act on Information for a
wider range of stakeholders and significantly reduced latency
Deployment of data for use by an increasing number of stakeholders
Increasing range of data latency and availability requirements
Utilization of operational data to make real-time decisions as well as for
planning, scheduling and dispatch
Greater integration of operational and business system data
Regulatory Requirements
Improved data security and an increase in user authorization levels /
schemes
Increasing need to provide data for 3rd party reporting (e.g., regulatory
reporting)
Compliance with CPUC mandated Open ADE
Compliance with FERC 2004 Access controls to reports around usage data
10
Future State Information Requirements
Technology Requirements
• Information Systems must be architected and designed to be adaptive
and resilient to autonomous, independent, potentially unexpected or nonresponsive behavior of the new participants. Example Distributed
Generation
11
Target Set of capabilities
Based on future State Requirements and the identified Information challenges, a
Target set of capabilities need to be stood up in a phased manner. Linked,
Incremental capability build out with tangible Business benefits are being
proposed.
While some efforts seem large, they can be implemented in a smaller scale, yet
with an Enterprise view to iron out issues after which they can be propagated.
Once established, these capabilities can:
•
•
•
Be leveraged by Line of Business initiatives
These initiatives will also assure consistent, reliable Information
Be re used in multiple projects
12
EIM Capabilities-Phased approach
Information Architecture Phase III
•
•
•
•
•
•
DW/BI Rationalization
BI Unified Platform
Complex Event Processing
Analytics
Advanced Visualization
Train of thought analysis
Information Architecture–Phase II
• Enterprise Data Integration with Mash upsInformation as Service Paradigm
• Multi Domain Master Data Management
(Incubator of many EIM disciplines
• Enterprise Data Layer
• SOA and Enterprise Service Bus
Capability
Phase III
Capability-Phase II
Information Architecture-Foundational Phase I
•
•
•
•
•
•
•
Enterprise Semantic Model
Enterprise Meta Data Management
Enterprise Data Profiling and Quality
Enterprise Data Governance
Industry Standards (CIM)
Information Lifecycle Management
Best Practices
Foundation- Phase I
13
EIM Capabilities-Phased approach
Information Architecture-Foundational Phase I
•
•
•
•
•
•
•
Enterprise Semantic Model
Enterprise Meta Data Management
Enterprise Data Profiling and Quality
Enterprise Data Governance
Industry Standards (CIM)
Information Lifecycle Management
Best Practices
Information Architecture Phase III
Information Architecture–Phase II
• Enterprise Data Integration with Mash upsInformation as Service Paradigm
• Multi Domain Master Data Management
(Incubator of many EIM disciplines
• Enterprise Data Layer
• SOA and Enterprise Service Bus
•
•
•
•
•
•
DW/BI Rationalization
BI Unified Platform
Complex Event Processing
Analytics
Advanced Visualization
Train of thought analysis
14
Benefits of EIM
Key Best Practice
Capabilities
Direct
Benefits
Stakeholder
Benefits
• Increased data accuracy, completeness,
Master Data, Meta
Data and Data
Quality
conformity, consistency, and integrity
• Standard for data retrieval
• Effective remediation processes
• Future-proof people, process and tech to
Data Integration
Architecture
Complex Event
Processing
Capabilities
meet uncertain regulatory/industry factors
• Data as a Service
• Data-Store once use many times
• Higher predictability and reliability
• Align system processes with business
processes
• Reduce upstream workload volumes
• Ability to transform large amounts of data to
Advanced
Visualization &
Analytics
Capabilities
useful, comprehensible information
• Improve customer relationships through
targeted demand response programs
• Enhance environmental and regulatory
compliance through more effective tracking
• Achieve greater network reliability and
resilience through real-time performance
updates
Customer
Increased reliability,
increased quality of
available information
and reduced cost to
serve
Regulators
Streamlined
information gathering
process and
reporting
Shareholders
Reduction in overall
cost / improvement
in EPS
15
EIM Capabilities-Foundation phase
Information Architecture Phase III
•
•
•
•
•
•
DW/BI Rationalization
BI as a Platform
Complex Event Processing
Analytics
Advanced Visualization
Train of thought analysis
Information Architecture–Phase II
• Enterprise Data Integration with Mash upsInformation as Service Paradigm
• Multi Domain Master Data Management
(Incubator of many EIM disciplines
• Enterprise Data Layer
• SOA and Enterprise Service Bus
Capability
Phase III
Capability-Phase II
Information ArchitectureFoundational Phase I
• Enterprise Semantic Model
• Enterprise Meta Data
Management
• Enterprise Data Profiling and
Quality
• Enterprise Data Governance
• Industry Standards (CIM)
• Information Lifecycle
Management
• Best Practices
Foundation Phase I
16
Foundational Phase
•Enterprise Semantic Model
It is a model driven approach to managing Data, Information, Intelligence
and Integration. It helps us understand how different pieces of information
relate to each other in a consistent manner.
It helps us achieve consistency from a conceptual model level all the way
to run time artifacts
Don’t model subjects individually
Customer
Rate
Asset
Model for Enterprise
Customer, Rate, Asset,
Programs, Demand
Response,Vendor,Employee
17
Enterprise Semantic Integration
Benefits of Semantic Integration
Message
based
Integration
Enterprise
Semantic Model
Enable broad data interface integration
Forces “semantic coherency”
across all interoperable data
interfaces
Easily “Plug-in” additional data interfaces
Describe new interfaces in terms of a
“business-like” conceptual model
Data bases
Business
Intelligence
Lingua-franca for the business
Business, Analysts, developers,
architects, data stewards can
understand
Data Governance, Business
Processes, Risk Visibility enabled
Easily “Plug-in” additional data interfaces
Describe new interfaces in terms of a
“business-like” conceptual model
18
Semantic Modeling Process
Semantic formation Inputs
Semantic consistency and
standardisation
Semantic outputs for run
time consumption
XML Message
Metadata
Existing Models
Reference
Models
Consistent,
Semantic
Model
RDF
Canonical
Output
Generation
IEC-CIM
OWL
Business
Vocabulary
DDL
Closed loop change management
Governance, Tooling, Training, Change Management
OWL-Web Ontology Language
RDF-Resource Description Framework
DDL-Data Definition Language
XML-extensible Markup Language
19
Enterprise Semantic Model Eco System
Semantic
Assets
Create/Edit
View
Discover
Enterprise Semantic Model (CIM Inspired)
Metadata
Repository
Mapping
Mapping
Mapping
Governance
User
Interaction
Implementation
Layer
Text
Text
Data Sources/Services
20
Foundational Phase
Enterprise Meta Data Management
Metadata is data about data. Describing a resource with metadata allows it to be
understood by both humans and machines in ways that promote interoperability and re use.
Metadata is structured information that describes, locates and makes it easier to retrieve,
use, or manage an information resource.
Types of Meta data Include: Business, Technical & Operational
What does this data mean ?
Where did it come from ?
How did it get there ?
Why is my report showing different data than
your report?
Who’s data is right ?
21
Foundational Phase
Benefits
•
•
•
•
•
Increased confidence in data
Assert Lineage, Quality and Fit for purpose
Foster Discovery, Self service, adoption, sharing and Re use
Helps machine to machine interaction such as Data Integration
Reduce support needs and costs
22
Metadata Management Environment
Inputs
Presentation
Meta Engine Layer
Information Models
Services Registry
Repository
Meta Data
Integration
Meta Data
Repository
Analysis
Search
Data bases
Central
Repository
Business Intelligence
Extract Transform
Load
·
·
·
·
·
·
·
·
Impact
Usage
Lineage
Quality
Accuracy
Trends
Timeliness
Availability
Discover
View
Direct Entry &
Update
Data Quality
23
Metadata at PG&E - Example
Data Element: Productive Time
Business Name: Productive Time
Data Definition: Employee wages paid while the employee is
at work …
Abbreviation:
PrdTm
Data Source:
SAP/Time Keeping
Business Rule: Productive time (physical time at PG&E)
can be non-billable for emails, meeting …
Mouse over or right click to
see Metadata “pop-up” with
information about a specific
data element
Mouse over or right click
to see metadata “pop-up”
24
Foundational Phase
Enterprise Data Quality
Data quality is an assessment of fitness of the data to serve its purpose in a
given context. Aspects of Data Quality include
Accuracy
Completeness
Timeliness
Relevance
Reliability
Viability of business decisions are contingent on good data...
Good data is contingent on an effective approach to Data Quality
Management
25
Foundational Phase
Enterprise Data Quality approaches
Reactive: addresses problems that already exist
deal with inherent data problems, integration issues,
merger and acquisition challenges
Proactive: diminishes the potential for new problems
to arise Governance, roles and responsibilities, quality
expectations, supporting business practices,
specialized tools.
Both approaches are needed. Profiling and quality management should
be taken as upstream as possible in the data creation process
26
Data Quality-Iterative implementation approach
27
Data Profiling Architecture
28
Enterprise Data Governance
Enterprise Data Governance
Is an Organizational capability that oversees the use and usability of Data.
It involves people, process and Technology
Benefits
•Increase consistency & confidence in decision making
 Decrease the risk of regulatory fines
 Improve data security
 Achieve consistent information quality across the organization
 Designate accountability for information quality
 Semantic modeling will lend itself to Data Governance
29
Data Governance Paradigm Shift
From
From
To
BU or functional group specific business
processes and requirements ignore other cross
functional enterprise impacts
Data Governance forum to ensure end to end
impact assessment of all information
management efforts
Lack of Business ownership
Sponsorship and accountability
Data not managed as a priority
Data Managed as a Enterprise Asset
Bottom up IT development places low priority on
data management objectives
Development efforts that affect critical data
include top-down data stewardship
Source: Forrester
Source: Forrester
30
Standards and Information Management
The Smart Grid ecosystem will require a wide variety of information to be exchanged,
managed, accessed and analyzed. Standards specify object models that are the basis for
efficient exchanges of Information between applications within and among grid domains.
Broad implementation of these standards will enhance interoperability of applications and
reduce the time and expense required to integrate new technologies and systems.
Standards are a moving Target for Information Management. Certifications process is still
nascent.
At the core of many IEC standards is the IEC Common Information Model (CIM).
 CIM has been officially adopted to allow application software to
exchange information about the configuration and status of an
electrical network
 Some of the standards such as IEC 61850(Substatation Automation),
IEC 61968 (Distribution) and IEC 61970 (Transmission), 60870 (Exchange
of Information between control centers) are series with multiple parts,
where some parts may be appropriate, or may only be in a proposed
or draft form
 Domain models provided by the CIM may be leveraged by PG&E as
starter inputs for Enterprise Semantic Model
31
Logical Relationship amongst Standards
Open O&M
OPC
IEC-62541
IEC-61850
OAGIS
CIM
IEC-61968, 61970
Open GIS
Open ADR
32
Data Integration and Possible Standards
WS,61968
Portal
Customers,
partners
WS,Mutispeak,
Proprietary
8,61
970
B2B
Market
Integration
6196
,
70
08 R
,6 D
P3 A
N en
D Op
Measurement
Historian
Gateway
Services
Query
DMS
ETL
P
61 3 ,60
85 87
0 0,
Mobile
Gateway
Message Bus
61968,Multispeak
EMS
DN
Control Center
Bus
W
61 S ,
96 OP
8, C
97
0
PC
,O
WS 1970
6
OPC,61968
P3
61 ,608
85 70
,
0
tes,
Upda
Metering
Systems
DN
Field Area Network
Application
WS
,
61 OPC
97
0
Substations
y
HAN
Devices
ta r
Application
,
ak
pe
is
ut 68
M
S, 6 1 9
W
AN
ie
pr
ro
P
,
SI
Application
WS,Mutispeak,OPC
61968,970
Standards and Data Integration
Meter Data
Management
Gateways
ETL
Data
Warehouse
Master Data
Management
33
Information Lifecycle Management
The policies, processes , practices, services and tools used to align the
business value of Information with cost efficient and appropriate
Infrastructure from the time information is created to its final disposition
Source: SNIA
•Information has value, and that value changes over time
•Older DOES NOT necessarily mean lower value for Information
•A key Objective of ILM is to ensure cost of ownership to be commensurate
with value of Information
34
EIM Capabilities-Phased approach
Information Architecture Phase III
•
•
•
•
•
•
DW/BI Rationalization
BI as a Platform
Complex Event Processing
Analytics
Advanced Visualization
Train of thought analysis
Information Architecture–Phase II
• Enterprise Data Integration with Mash
ups- Information as Service Paradigm
• Multi Domain Master Data
Management (Incubator of many EIM
disciplines
• Enterprise Data Layer
• SOA and Enterprise Service Bus
Capability
Phase III
Capability-Phase II
Information Architecture-Foundational Phase I
•
•
•
•
•
•
Enterprise Semantic Model
Enterprise Meta Data Management
Enterprise Data Profiling and Quality
Enterprise Data Governance
Industry Standards (CIM)
Information Lifecycle Management
Foundation Phase I
35