Overcoming Communication,
Distributed Systems, and Simulation
Challenges: A Case Study Involving the Protection and Control of the
Electric Power Grid
Kenneth Hopkinson
Cornell University
Department of Computer Science

Motivating
Example

Research Goals
• Explore Methods to Identify and Avoid
Potential Pitfalls in Using a Utility
Intranet
• Investigate Potential Power System
Protection and Control Applications that can Benefit from IP Networking
• Evaluate Communication Protocols that have the Ability to Support Applications with Real-Time Requirements

Goal I: Explore Methods to
Identify and Avoid Potential
Pitfalls of Using a Utility
Intranet

Motivation: Power Scenarios that Involve a Utility Intranet
• Power systems are becoming increasingly complex
– Increasing communication
– High-level power control
– Low-level power protection
• High-quality simulators exist
– ns2: communications
– PSCAD/EMTDC: electromagnetic
– PSLF: electromechanical
• Needed: A process that combines simulators with minimal disruption

EPOCHS
• Problem: Costly and time consuming to create single, combined simulation
Agent
• EPOCHS:
– three domain-specific simulators
Unified
View
Agent
Agent
Legend
Simulators
Federated
Communication
Combined
System
– combines events and timing
Custom
Modules
– uses non-intrusive techniques
– adds agent support
– presents unified view
RTI
NS2
150K LOC
PSLF PSCAD/EMTD

Agents
• Autonomous - take independent action
• Interact - sense and modify environment
• Communicate - ability exchange information

Sample Agent

Agents within the Utility Intranet
Utility
WAN Power Plant - Control Center - Substation
Router
Ethernet LAN
Protection
IED
Protection
IED
Control
IED
Control
IED
Substation
Power Plant
Control
Center
Agent
Communication
Control and
Protection
Schemes
Substation LAN
Actuator
Output
Sensor
Input

Goal II: Investigate Potential
Power System Protection and
Control Applications that can
Benefit from IP Networking

A List of Scenarios
• Backup Protection Relays
• Differential Relays for T-Junction Lines
• Remedial Action Scheme
• Bilateral Load Following
• Voltage Collapse

Case study: Remedial Action Schemes
(RAS)
• Detect abnormal system conditions
• Take corrective action to maintain system stability and voltage levels
• Current RAS inadequate

RAS
Example

50 Generator IEEE Test Case
• Published Power System
• Modified to Emulate Western United
States Remedial Action Scenario

Northwest System
• Rotor angles undergo dangerous increase in northwest after line loss
• System stable after generation rejection
Without Agent RAS With Agent RAS

Southeast System
• Southeast system undergoes unacceptable frequency decrease
• Remote load shedding maintains frequency above preset level
Without Agent RAS With Agent RAS

Backup Protection Systems
Agents improve total system response by 87%!!

Goal III: Evaluate
Communication Protocols that have the Ability to Support
Applications with Real-Time
Requirements

Strengthening QoS Properties of
IP-based Networks
• QoS Community Largely Focuses on
Protocols Requiring Router Modification
– RSVP
– DiffServ
• These Protocols Have Not Been
Deployed in Practice
• Needed : Method to Strengthen QoS
Properties without Router Modification
• First Step : Gravitational Gossip

Gossip Example

Gossip Example

Gossip Example

Gossip Example

Gossip Example

Gravitational Gossip:
Percent Information Desired
25%
50%
99%
75%

Adaptive Gossip Weights
• Used Multiple Methods for Comparison
• First was the Secant Method Taken
From Numerical Analysis x k

1
 x k
 f ( x k
) f x k
( x k
)

 x k

1 f ( x k

1
)
• Determine Next Gossip Weight Based on History

Control Theory
• Proportionate Controller
• Proportionate Controller with Delay

Test System
• 20 Nodes Per Group x 4 Groups
• 1 Data Source
• Arranged in a Mesh Architecture
100%
75%
50%
25%
Data Source

Experimental Results
• Performed control tests on a uniform grid (impulse, ramp response, sustained oscillation, unit step, unit step disturbance, communication): Error Results
• P Controller with Delay Best Performer When Network
Conditions Changing

Experimental Results
• Performed control tests on a uniform grid (impulse, ramp response, sustained oscillation, unit step, unit step disturbance, communication): Signal Shift Results
• Newton’s Method is Best When Data Targets are Changing

Test System Revised: 100%
Information Target in All Cases
Gossip Vs. TCP Vs. UDP
100%
100%
100%
100%
Data Source

Messages Sent
• UDP: ~20 Msgs/Rnd, TCP: ~1 Msg/Rnd,
• Gossip: ~ 6-8 Msgs/Rnd

Latency and Error Rates
• TCP Latency is very high, UDP and Gossip
Comparable
• TCP and UDP have comparatively high error rates, Gossip error is near 0.

Sample Application:
Bilateral Load Following
• Goal: Allow variable power contracts between loads and generators located in different control areas
• Communication between loads, generators, and control centers in the electricity’s path are needed
• Currently creating a test case combining published power systems that are 9, 14, 30, 39, and 57 buses in size respectively.

The WECC

Representative Power
Contracts in the WECC

WECC Power Grid with
Contracts

Degree of Interest Decreases with
Distance from Power Contract
Corridors

Superimposition of Gossip and Power Maps

Future Work
• Interdependence between IT and physical infrastructure likely to grow over time
– Monitoring critical infrastructures
– Military command and control networks
– Other large publish/subscribe systems
• Better understanding QoS requirements of key applications could have a big impact

Future Work
• Traditional QoS protocols use SLA’s enforced through RSVP or DiffServ in a static fashion
• Gravitational gossip adapts to network conditions in a comparatively nimble fashion
• A more detailed comparison of the strengths and weaknesses of these approaches could lead to new innovations

Future Work
• Co-Optimize Communication and
Electric Power Requirements to Reach a Global “Best”
• Use Recorded Information from Real
Disturbances to Validate EPOCHS
• Use Communication Methods to Stop
Such Disturbances

My Colleagues
• Ken Birman , Computer Science, Advisor,
Cornell University
• Jim Thorp , Electrical and Computer
Engineering, Committee Member, Cornell
University
• Xiaoru Wang , Electrical Engineering,
Southwest Jiaotong University
• Renan Giovanini , Electrical Engineering,
University of Sao Paulo at Sao Carlos
• Denis Coury , Electrical Engineering,
University of Sao Paulo at Sao Carlos
• Kate Jenkins , Computer Science, Cornell
University

My Contributions
• Developed concept and bulk of code in
EPOCHS
• Co-Developer of electric power cases
• Co-creator of static version of graviational gossip
• Primary inventor of adaptive gravitational gossip

Related Work
• Use of Agents to model entities within a simulated world
– Lee S., A. Pritchett, D. Goldman 2001. Hybrid Agentbased Simulation for Analyzing the National Airspace
System. In Proceedings of the Winter Simulation
Conference , Arlington, VA, USA, 1029-1037.
• Combining commercial software with limited source-code availability is still quite rare. A prominent example:
– Strabburger, “On the HLA-based Coupling of
Simulation Tools”, European Simulation
Multiconference 1999
• Epidemic Communication Protocols
– Birman, K., Hayden M., Ozkasap, O., Xiao, Z., Budiu,
M., and Minsky, Y. Bimodal Multicast. ACM
Transactions on Computer Systems 17, 2 (May 1999),
41 – 88.

Conclusion
• The EPOCHS testbed allows us to examine agent-based scenarios under realistic conditions
• Early work shows promise for the use of agents in the electric power grid
• Future work will concentrate on studying the effect of background traffic on power systems with real-time requirements

Questions?