Keeping the lights on Keeping the lights on Keeping the lights on Keeping the lights on Keeping t Keeping the lights on Keeping the lights on Keeping the lights on PAC.MARCH.2010 by Xi-cai ZHAO, Shu-chao WANG, Shao-jun CHEN, NR Electric Co., China Data Acquisition SCADA 46 47 in Substation for the Future the lights on Today, almost everyone is talking about Smart Grid. However, data and information are the basis for the operating of the electric power system, and data acquisition is the key issue. There are several independent systems between substations and control centers : SCADA system Relay Protection & Fault Information Management System (PMIS) Special Protection Scheme (SPS) Wide Area Monitoring System (WAMS) Tele-metering system (TMS), etc. Each system is for a unique purpose and has its own data acquisition devices in the substation, dedicated communication channel between substations and control centers, and dedicated processing workstation in control centers. Furthermore, each system is maintained by a separate team, thus the maintenance of common system data models, parameters, and data itself is done repeatedly, and there is inconsistency between these systems. With the application of electronic instrument transformers and IEC 61850, it is possible to integrate these systems, optimize hardware configuration, re-allocate functions, unify data models and the communication interface, so as to share the power grid models, parameters, data and other resources, make it easy to maintenance or add new functions, and save investment. The roadmap to achieve a universal data acquisition platform proposed in this article includes the following steps: Update the conventional bay control unit to include functions of Phasor Measurement Unit (PMU) and that of Disturbance & Fault Recorder (DFR) Update substation gateway interfaced with control centers to handle all the data collected by substation automation systems Design a new communication protocol to transmit all kinds of data between substations and control centers via a single communication channel (IEC 61850-90-2 may be the answer) PAC.MARCH.2010 Data Acquisition SCADA 48 Data acquisition is the key issue. It is feasible to re-design substation automation system, Update the front end terminals at control centers to handle all the data for different purposes Set up a universal platform at control centers for data acquisition, data sharing, data display, and various applications, which are part of smart EMS (Energy Management System) Reliability plays a significant part in the design process of a Universal Data Acquisition Platform, which should be taken into consideration during the hardware and software design of the whole system and can be achieved in four aspects: redundancy, security, robustness and resilience. Data requirements for different purpose Today, there are several departments in control center that care about real time data from substations. In China they are: Operating Operation Planning Market and Trading Protection Automation Communication Departments. Other utilities may have different organization structure, but with similar functions. For operators on duty at control centers, the data or information should assist them to manage the entire system during steady state, fault clearance period, and post-fault restoration. The data required for the monitoring and control of the power system can be classified into three types: Real Time Data Real time data from the physical process, including alarms, events, status, analogue measures of generation, load, flow, voltage, current, etc. Real time data generated by data processing and by 11 operator input, including derived values operator controls/commands/comments, output of close-loop algorithms, etc. Real time data imported from other information systems, e.g. from other SCADA nodes, from external SCADA systems, etc. Historical/Archived Data Long term storage of selective real time data for trending, analysis and planning purposes Semi-static Data Data describing the physical processes such as power system topology, asset parameters, etc. Data supporting the processing functions such as conversion parameters, displays, etc. On the other hand, protection people want to upload protection parameters, fault recordings from substations to do post-fault analysis, protection setting check and protection performance evaluation. Recently, many literatures researched feasibility of system modeling, state monitoring, state estimation, stability analysis, post-fault analysis or even close-loop control for stability purpose based on data collected by PMUs. The requirements on refreshing cycles and propagation delay of PMU data for different purposes are shown in Table 1. Roadmap to achieve a Universal Data Acquisition Platform The main idea of a universal data acquisition platform is to merge the existing independent systems and at the same time fulfill the various requirements on data acquisition Existing systems between substations and control center Control Center networks between EMS PMIS AAC SPS TMR FE FE FE FE FE Proprietary IEC 60870-5-102 substations and control centers, and automation system in the control center, IEC 60870-5-101 IEC 60870-5-104 Proprietary IEEE C37.118 Substation Gateway Gateway PDC Gateway Gateway DFR PMU SPS MET to have better protection, BCU PROT monitoring and SCADA control of the electric power system. AAC EMS PDC PMIS SCADA Analysis and Application Center Energy Management System Phasor Data Concentrator Relay Protection & Fault Information Management System Supervisory Control and Data Acquisition PAC.MARCH.2010 PMIS BCU FE PMU SPS WAMS Bay Control Unit Front End Phasor Measurment Unit Special Protection Scheme SPS DFR MET PROT TMR WMAS TMR Disturbance & Fault Recorder Meter Protection Tele-Metering Reading System Wide Area Management System by Xi-cai ZHAO, Shu-chao WANG, Shao-jun CHEN, NR Electric Co., China 49 from system operation point of view. After investigating the existing systems, a roadmap has been proposed as shown in Figure 2. Update BCU: (Figure 2(A)). Update bay control unit as a universal bay control unit (UBCU). It integrates functions of conventional bay control unit (BCU), phasor measuring unit (PMU), and dynamic and fault recorder (DFR), and it can sample data of steady state, dynamic state and transient state. All the samples are time tagged. For traditional instrument transformers, different secondary coils are provided for different purposes, i.e. protection, measuring and tariff metering. For non-conventional instrument transformers, different communication data channels are provided for different purposes, i.e. protection, measuring / metering, as defined in IEC 60044-8. The UBCU may adopt data of different quality regarding steady state accuracy and transient performance from instrument transformers. If process bus is applied, such a new design reduces the number of secondary equipments connected to process bus for a different purpose, thus reducing the load flow on process bus. Thanks to the development of micro-controllers and DSPs, there is no doubt that a bay control unit can handle all the data efficiently. The sample rate can be high enough to process steady state, dynamic and transient data. Under normal condition, only time-tagged steady state analogue signals are sent at lower rate, for example one package per second. During a dynamic period, when a low frequency oscillation occurs, the event and oscillation frequency are reported or the low frequency oscillation can be detected in the control center, if the data sampling rate is high enough. When a fault occurs, the brief report will be sent at first, and the total recording will be sent after the recovery from the fault for further analysis. Update Gateway: Update the gateway as a universal communication interface to the control centers (see Figure 2(B)). Such a gateway can handle all the data collected by substation automation systems, and transmit them to control centers: SCADA data, protection settings and fault reports, fault recording, etc. At this stage, the communication channels are still separated between normal SCADA and other applications. As an optional solution, if it is not feasible to update BCU at once as described earlier, the steady state sample without time tag received from BCUs can be time-tagged by the Gateway before being transmitted to the control centre. The inaccuracy of time synchronization of this solution can be minimized by increasing the data refresh rate and decreasing propagation delay of communication between BCUs and the Gateway. Another idea related to Gateway is to set up a substation-level sub-database as a part of entire SCADA system. The raw data will be processed; the bad data will be checked and corrected. Furthermore, part of state estimation function can be completed at substation level, which is done in the control center today. Data and information are the basis of the operation of the electric power system. Update Tele-communication Facilities: Today, the wide area communication networks are available for all the applications described in this article, for example, Synchronous Digital Hierarchy (SDH) and Synchronous Optical Networks (SONET) links in fiber optic networks. One obstacle to improving the efficient operation of the electric power system is the number of communication protocols used. Today we have different standards for different applications; some of them are still proprietary ones, as shown in Figure 1. Measured value with time tag can be transmitted via IEC 60870-5-101 or IEC 60870-5-104, but this kind of ASDU (Application Service Data Unit) is seldom used practically today. To transmit all kinds of data between substations and control centers via a single communication channel, one solution may be IEC 61850-90-2: Using IEC 61850 for the communication between substations and control centers. Whether or not this future standard can cover all the applications is unknown yet. Update Front End Terminals: Figure 2(C) shows a front end terminal that handles all the data for a different purpose. Its function is almost the same as in the past. Update Data Collection Platform: The last step is to set up a universal data collection platform for data acquisition, data processing, and data management at control centers for various applications (Figure 2(D)). This centralized solution leads to the controlled consistency of redundant data, a high level of data integrity, standard data types and the sharing of common data. Control Center Organization and Workflow: The organizational structure of the control center is one main factor which leads to the fact that many independent systems exist in parallel. Each department in the control center only cares about the data they need. Once they feel it necessary, they will design a dedicated system with its own data acquisition devices in substation, dedicated communication channels between substations and the control center, and dedicated processing workstations table 1 Requirements on: Refreshing Cycles and Propagation delay of PMU Data Function in Control Center Refreshing cycles Propagation delay 25-50 25-50 25-50 25-50 25-50 25-100 ≤1 ≤1 Several Several ≤1 ≤ 0.1 (per sec.) Frequency monitoring Reactive power & voltage monitoring State estimation Model & parameter verification Low frequency oscillation monitoring Stability assessment and emergency control (sec.) PAC.MARCH.2010 Xi-cai ZHAO - received his master degree from Electrical Engineering Department of Shanghai Jiaotong University (SJTU) in 1997. From then on, he has been working for Nanjing NARIRELAYS Electric Co., Ltd (NR Electric). He was involved in development of digital protective relays, bay control units, protocol converters, special protection scheme, electronic instrument transformers, etc. Now, he is the deputy chief engineer of NR Electric, Secretary General of Protection Study Committee of Chinese Society of Electrical Engineering (CSEE), member of Chinese National standard Technical Committee of Measuring Relays and Protection Equipments, Member of IEC TC95 MT1, MT3 & MT4, and regular member of CIGRE SC B5. E-mail: xczhao@nari-relays. com by Xi-cai ZHAO, Shu-chao WANG, Shao-jun CHEN, NR Electric Co., China Data Acquisition SCADA 50 Shu-chao WANG received his master degree from Huazhong University of Science and Technology (HUST) in 2004. Since then he joined NR Electric as a R&D engineer. He involved in development of IEC 61850 based digital protective relays, bay control units etc. Now he is also a member of CIGRE WG B5.36. E-mail: wangsc@nari-relays. com in the control center. The seamless universal platform solution may break the boundaries between them, and change the way they work today. Redundancy and Reliability The automation industry’s rapid movement toward computer controls has increased the importance of reliability in data acquisition systems. Reliability plays a significant part in the design process of our Universal Data Acquisition Platform. According to IEC 60870-4, reliability is defined as a measure of the ability of equipment or a system to perform its intended function under specified conditions for a specified period of time. The Universal Data Acquisition Platform shall continue to be operable, according to the “graceful degradation” principle described in "Distributed Control System Reliability" by Emmanuel G. Diaz and Iliya A. Dormishev, as well as "Reliability investigations for SA communication architectures based on IEC 61850" by Lars Andersson, Christoph Brunner. This principle should be valid if any component of the data acquisition system fails, such as a failure in UBCU, gateway, FE or any of the above SCADA or other application computers. In Barry C. Ezell’s thesis on vulnerabilities of the SCADA system, it is said that a reliable system depends on its survivability. Survivability is a measure of how a system will function and recover after failure and is dependent on the system’s redundancy, security, robustness and resilience. For the Universal Data Acquisition Platform, these principles directly involve the hardware and software design of the whole system. Redundancy: Redundancy plays a crucial role in maintaining high reliability of the whole Universal Data Acquisition Platform. Redundancy in a system “…refers to the ability of certain components of a system to assume functions of failed components without adversely affecting the performance of the system itself”. Some events, such as device power off, network physical or logical disconnect, systems crash, can make the whole system out of service. Yet, these events are avoidable using redundant data acquisition methods. For example, a redundant power supply can take the UBCU unaffected when one power supply is out of service. A hot standby redundant UBCU can avoid data loss when the other UBCU crashes for an unknown reason. What’s more, the physical topology of the system network connections must support redundancy, and the network software must be intelligent enough to recover from various physical failures. On the substation side, a redundant station bus should also be adopted for redundant data acquisition of the UBCU. By combining these different redundancy methods, many hybrid systems can be created. Security: Security deals with prevention, detection and defence from internal and external attacks. If there is nothing preventing a hacker from breaking into SCADA networks, anyone with enough knowledge of computers can disrupt communications, change program parameters, and eventually cause the system to malfunction. Therefore, PAC.MARCH.2010 Tariff metering needs to be separate from the other secondary equipment . it is important to design a system that provides protection against unauthorized access to operator displays, control operations, database modification, and access to applications and critical functions. With the introduction of the international security standard IEC 62351, a role based access control solution will be established to safeguard the whole communication system. The role based access control can not only be applied to field device UBCU, gateway, but also to the whole data acquisition platform, consisting of field devices, station control and network control – the complete pyramid, in order to support end to end security. This can secure the universal data acquisition system against all kinds of cyber attacks, improving the reliability of the whole system especially in case of increasing network crimes. Robustness: The third principle of survivability involves a system’s robustness. It “…refers to the degree of insensitivity of a system design to errors”. The Universal Data Acquisition system should be robust enough to maintain all components, both software and hardware, functioning well to keep the system running after an attack or failure. And the function code implemented in the system should have the ability to function under abnormal conditions. Code in each of the network components can be wide-awake so that if there is a failure in communication, it will be detected and then a new set of rules are given to the system. The purpose of this set of rules is to keep the system stable under the presence of failure and increase failure tolerance. Resilience: “Resilience is the ability of a system to operate close to its intended design, technically and institutionally, over a short time after the attack, such that the losses are within manageable limits”. Basic examples of resilience to SCADA system are virus protection programs. If the data acquisition systems have connections to the Internet, it runs the risk of having the system infected by a computer virus. The virus protection software is constantly searching through system files for a virus, and if found, destroying it before it corrupts any data. In addition, if an attack is successful, a plan of action has to be in place so that system operators and technicians can deal with the situation to bring the system back online as soon as possible. In SCADA system, resilience is normally limited to the code used to program the gateway and UBCUs. The code has numerous checks between each unit and the supervisor, which creates a set of alarms that will be displayed through the HMI. From here, operators would attempt to correct the errors, and return the system to its normal operational mode. by Xi-cai ZHAO, Shu-chao WANG, Shao-jun CHEN, NR Electric Co., China 51 12 Steps towards the universal solution Control Center Control Center SCADA APP SCADA APP FE FE FE FE Substation Gateway Substation Gateway UBCU (BCU + PMU + DFR) A Gateway B UBCU (BCU + PMU + DFR) Control Center SCADA APP APP SCADA +APP FE FE Substation Substation Gateway Gateway UBCU (BCU + PMU + DFR) C 13 Control Center D Various applications Substation overview UBCU (BCU + PMU + DFR) UBCU 14 Universal Bay Control Unit The team work environment PAC.MARCH.2010 Shao-jun CHEN became a protective relay engineer for East China Power Grid Co. (ECPGC) in 1983. Since 1989, he has been a department manger of relaying protection in ECPGC, and had ten years’ professional experience on power system operation and protection from 1983 to 1992. From 1993 to 1996, he was a chief and professional engineer at Grid Control Center in ECPGC being responsible for power system stability control. He joined in Schweitzer Engineering Laboratories, Inc. (SEL) in 1996 and has ten years’ experienced with SEL on schematic design, project management and technical application respectively for international customers, and became a RSM for East Asia since 2002. At present, he has been working for NR Electric since 2006 as a Vice President E-mail: sjchen@narirelays.com.