RTDMS™ Continuous Phasor Data Archival Functional Specification
C onsortium for
E lectric
R eliability
T echnology
Solutions
Real-Time
Dynamics
Monitoring
System™
(RTDMS)
FUNCTIONAL SPECIFICATION
FOR
CONTINUOUS PHASOR DATA FILE ARCHIVAL
USING COMTRADE FORMAT
Prepared by
Manu Parashar
Ajay Das
Ken Martin
Date: June 1, 2009
®Built upon GRID-3P Platform, U.S. Patent 7,233,843. Electric Power Group, LLC. All rights reserved.
RTDMS™ Continuous Phasor Data Archival Functional Specification
TABLE OF CONTENTS
Page
1.
EXECUTIVE SUMMARY .................................................................................................................3
1.1
Objective....................................................................................................................... 3
1.2
Background .................................................................................................................. 3
1.3
Functional Overview ..................................................................................................... 3
2.
Introduction......................................................................................................................................4
3.
C37.118 Format...............................................................................................................................4
3.1
Data Frame................................................................................................................... 5
3.2
Configuration Frame..................................................................................................... 6
3.3
Header Frame .............................................................................................................. 7
3.4
RTDMS Server ............................................................................................................. 7
4.
COMTRADE Format........................................................................................................................8
4.1
Configuration File ......................................................................................................... 8
4.2
Data File ..................................................................................................................... 15
5.
RTDMS Server GUI .......................................................................................................................17
6.
SYSTEM ARCHITECTURE...........................................................................................................18
7.
IMPLEMENTATION SCHEDULE ..................................................................................................18
LIST OF FIGURES
Figure 1: Example of frame transmission order.....................................................................................4
Figure 2: Continuous Data File Configuration in RTDMS Server GUI.................................................17
Figure 3 (a) and (b): Current and Proposed Implementation...............................................................18
LIST OF TABLES
Table 1:
Table 2:
Table 3:
Table 4:
Table 5:
Table 6:
Table 7:
Word definitions common to all frame types............................................................................5
Data Frame..............................................................................................................................6
Configuration frame .................................................................................................................6
Header frame...........................................................................................................................7
PC37.118 Utility output............................................................................................................8
COMTRADE Configuration File.............................................................................................11
Implementation Schedule for the Continuous Data Recorder ...............................................18
2
RTDMS™ Continuous Phasor Data Archival Functional Specification
1. EXECUTIVE SUMMARY
1.1 Objective
The Real Time Dynamics Monitoring System™ (RTDMS) is the phasor platform presently in place for
viewing and extracting real time and historical data from the Eastern Interconnection (EI) phasor
network. The goal is to build continuous data recording capability which will take the existing data
stream in IEEE C37.118 format and store it in data files using COMTRADE format incorporating its
suggested profile for phasor data that was recently approved. This will allow portability of the data to
be used by engineers and analysts using any application capable of reading data in the COMTRADE
format.
1.2 Background
In the current RTDMS application setup, data is processed by RTDMS Server application and then
archived in the long term RTDMS SQL database. One way to access this data is to use the RTDMS
online report application and download the data files in CSV format. The major disadvantages of this
method of data archive/retrieval are:
−
−
−
−
Data downloaded in CSV format is not standardized and so to use these files in other applications
the data has to be manually reformatted or custom programs have to be written to use these files
CSV files are ASCII files which have higher size requirement for the same amount of data when
compared to BINARY files
Data stored in database has data filters and flags applied and so is not the actual raw data from
the PMUs that is needed for analysis
Data retrieval from the database is an on-demand manual download process
To enable automated continuous recording of synchrophasor data in binary data files, new
functionalities will be developed to work with RTDMS Server to convert real time data in C37.118
format to COMTRADE format and archive according to user configuration settings.
1.3 Functional Overview
Following modules are planned to be implemented to achieve the continuous data file recording
functionality:



Data reader to read the synchrophasor real time data in C37.118 format, buffered in RTDMS
Server 5 minute data cache
COMTRADE data file generator to convert the data from C37.118 to COMTRADE format and
generate the COMTRADE configuration file and binary data files
Configuration GUI to let user configure data file storage options like target folder, maximum
storage space etc.
Following sections will explain the requirements in detail.
3
RTDMS™ Continuous Phasor Data Archival Functional Specification
2. Introduction
Synchrophasor data is used both in real-time from direct transmission and after the fact as recorded
data. The SYNCHROPHASOR standard, C37.118, describes a real-time data format but does not
define a format for recorded data. Several formats have been used for synchrophasor data, most
notably the PhasorFile (also known as ‘dst’) data format pioneered in the WECC and supported by
Bonneville Power Administration users. While a number of users have created recording and reading
software for this format, it is not supported as a recognized standard by a standards organization. It is
preferable that data be recorded in a recognized standard format so that tools, methods, and data can
be exchanged world wide. The IEEE COMTRADE standard is a file format designed for time series
data that is established world wide and is supported by standards making bodies. It has a significant
number of recording parameters that can be adapted for phasor data. This document outlines both
C37.118 and COMTRADE formats and suggests the conversion mapping.
3. C37.118 Format
PDC collects all the data packets from the PMUs, time correlates them, and forms a message packet.
Data from various PDCs are aligned by timestamp and concentrated into a single stream that is sent
over a local Ethernet port to local applications or across a wide area network to a Super PDC and
then to applications. Phasor data is usually transmitted using the User Datagram Protocol which
provides a connectionless, reliable transport service. The User Datagram Protocol (UDP) is one of
the core protocols of the internet protocol Suite. Using UDP, programs on networked computers can
send short messages sometimes known as datagram to one another. UDP is faster and more efficient
for many lightweight or time-sensitive purposes. Also, its stateless nature is useful for servers that
answer small queries from huge numbers of clients. The data sent over a UDP may be either in
PDCStream or PC37.118 format. This document largely concentrates on PC37.118 driver utility.
PC37.118 format consists of four message types viz Data frame, Configuration frame, Header
frame, and Command frame:
o Data messages are the measurements made by a PMU
o Configuration is a machine-readable message describing the data the PMU/DC sends and
providing calibration factors
o Header information is human readable descriptive information sent from the PMU/DC but provided
by the user
o Commands are machine readable codes sent to the PMU/DC for control or configuration
The first three message types are transmitted from the PMU/DC and the last (command) is received
by the PMU/DC. Information may be stored in any convenient form in the PMU/DC itself, but when
transmitted it shall be formatted as frames. In normal operation the PMU only sends data frames,
Figure 1: Example of frame transmission order
Figure 1 gives an example of the overall message framework of PC37.118 format.
All message frames start with a 2 byte SYNC word followed by a 2 byte FRAMESIZE word, a 2 byte
IDCODE, and a time stamp consisting of a 4 byte second-of-century (SOC) and 4 byte fraction-of4
RTDMS™ Continuous Phasor Data Archival Functional Specification
second (FRACSEC) which includes a 24-bit fraction-of-second integer and an 8-bit Time Quality flag.
The SYNC word provides synchronization and frame identification. IDCODE positively identifies the
unit sending or receiving the message. Bits 4-6 in the SYNC word designate the frame type. This
word is detailed Table 4. All frames terminate in check word (CHK) which is a CRC-CCITT. All frames
are transmitted exactly as described with no delimiters. Figure 1 illustrates frame transmission order.
The SYNC word is transmitted first and check word last. Two and 4 byte words are transmitted most
significant byte first (network or ‘big endian’ order). All frame types use this same order and format.
The following tables give the details of each frame.
Field
Size
(byte)
Comment
SYNC
2
FRAMESIZE
2
IDCODE
2
SOC
4
FRACSEC
4
CHK
2
Frame synchronization word.
Leading byte: AA hex
Second byte: Frame type and Version, divided as follows:
Bit 7: Reserved for future definition
Bits 6-4: 000: Data Frame
001: Header Frame
010: Configuration Frame 1
011: Configuration Frame 2
100: Command Frame (received message)
Bits 3-0: Version number, in binary (1-15), version 1 for this initial publication
Total number of bytes in the frame, including CHK.
16 bit unsigned number. Range = maximum 65535
PMU/DC ID number, 16-bit integer, assigned by user, 1-65534 (0 and 65535
are reserved). Identifies device sending and receiving messages.
SOC timestamp, 32 bit unsigned number, Second-of-Century count starting
at midnight 01-Jan-1970. Ranges 136 years, rolls over 2106 AD. Leap
seconds are not included in count, so each year has the same number of
seconds except leap years, which have an extra day (86400 seconds).
Fraction of Second and Time Quality. Time of measurement for data frames
or time of frame transmission for non-data frames.
Bits 31-24: Time Quality
Bits 23 – 00: Fractional-of-second, 24-bit integer number.
When divided by TIME_BASE yields the actual fractional second. FRACSEC
used in all messages to and from a given PMU shall use the same
TIME_BASE that is provided in the configuration message from that PMU.
CRC-CCITT, 16 bit unsigned integer.
Table 1: Word definitions common to all frame types
3.1 Data Frame
In normal operation the PMU only sends data frames. A data frame shall contain measured data and
shall be identified by having bits 4-6 in the SYNC word set to zero as shown in Table 1. The real-time
phasor data frame shall consist of binary data ordered as shown in Table 2. All fields shall be fixed
length as described and no delimiters shall be used. The frame starts with SYNC, FRAMESIZE,
IDCODE, and SOC, and terminates with a CRC-CCITT.
No. Field
Size
(bytes)
Comments
1
2
3
4
5
6
7
2
2
2
4
4
2
4xPHNMR
Sync byte followed by frame type and version number
Number of bytes in frame
PMU/DC ID number, 16-bit integer,
SOC timestamp, for all measurements in frame.
Fraction of Second and Time Quality, for all measurements in frame.
Bit mapped flags
Phasor estimates. May be single phase or 3-phase positive, negative,
SYNC
FRAMESIZE
IDCODE
SOC
FRACSEC
STAT
PHASORS
5
RTDMS™ Continuous Phasor Data Archival Functional Specification
8xPHNMR
8
9
10
FREQ
DFREQ
ANALOG
11
DIGITAL
Repeat 6-11
12+
CHK
2/4
2/4
2xANNMR
4xANNMR
2xDGNMR
2
or zero sequence. 4 or 8 bytes each depending on the fixed 16-bit or
floating point format used, as indicated by the configuration frame.
Frequency (fixed or floating point)
Rate of change of frequency (fixed or floating point)
Analog data, 2 or 4 bytes per value depending on fixed or floating
point format used, as indicated by the configuration frame.
Digital data, usually representing 16 digital status points (channels).
Fields 6 – 11 are repeated for as many PMUs as in NUM_PMU field in
configuration frame.
CRC-CCITT
Table 2: Data Frame
3.2 Configuration Frame
A configuration frame is a machine-readable BINARY data set containing information and processing
parameters for the PMU and the current real-time data set. It is identified by bits 4-6 the SYNC word
as shown in Table 1. Two config types are identified, CFG-1 with SYNC bit 4 set to 0 and CFG-2 with
bit 4 set to 1. CFG-1 denotes the PMU capability indicating measurements that the PMU is capable of
making. CFG-2, indicates measurements currently being made and transmitted in the data frame.
This may be only a subset of available data. Both frames have all 19 fields, with 7-16 repeated as
necessary. All fields shall be fixed length as described and no delimiters shall be used. The frame
contents are shown in Table 3.
No.
Field
Size (bytes) Comments
1
2
3
4
5
6
7
8
9
10
11
12
13
14
SYNC
FRAMESIZE
IDCODE
SOC
FRACSEC
TIME_BASE
NUM_PMU
STN
IDCODE
FORMAT
PHNMR
ANNMR
DGNMR
CHNAM
15
PHUNIT
2
2
2
4
4
4
2
16
2
2
2
2
2
16x(PHNMR
+ ANNMR +
16xDGNMR)
4 x PHNMR
16
ANUNIT
4 x ANNMR
17
18
19
DIGUNIT
FNOM
CFGCNT
Repeat 8-19
4 x DGNMR
2
2
20+
21+
DATA_RATE
CHK
2
2
Sync byte followed by frame type and version number.
Number of bytes in frame.
PMU/DC ID number, 16-bit integer, 1-65534
SOC timestamp, for all measurements in frame.
Fraction of second with Time Quality
Resolution of fraction-of-second time stamp.
The number of PMUs included in the data frame.
Station Name – 16 bytes in ASCII format
PMU ID number as above, identifies source of each data block
Data format within the data frame
Number of phasors – 2 byte integer (0 to 32767)
Number of analog values – 2 byte integer
Number of digital status words – 2 byte integer
Phasor and channel names – 16 bytes for each phasor, analog,
and digital channel in ASCII format in the same order as they are
transmitted
Conversion factor for phasor channels.
PhasorMAG = PHUNIT X .00001
Voltage PHUNIT = (300,000/32768) X 10E+5
Current PHUNIT = (15,000/32768) X 10E+5
Conversion factor for analog channels.
Because the analogs are represented by floating point
conversion factor is unity (or simply not used).
Mask words for digital status words.
Nominal line frequency code and flags.
Configuration change count
Fields 8 – 18, repeated for as many PMUs as in field 7
(NUM_PMU)
Rate of phasor data transmissions.
CRC-CCITT
Table 3: Configuration frame
6
RTDMS™ Continuous Phasor Data Archival Functional Specification
3.3 Header Frame
The header frame shall be human readable information about the PMU, the data sources, scaling,
algorithms, filtering, or other related information. The frame has the same SYNC, FRAMESIZE, SOC,
and CHK as the other frames, and is identified by bits 4-6 the SYNC word as shown in Table 1. The
data section has no fixed format (Table 4).
No
1
2
3
4
5
6
K+6
K+7
Field
SYNC
FRAMESIZE
IDCODE
SOC
FRACSEC
DATA 1
DATA k
CHK
Size
2
2
2
4
4
1
1
2
Comment
Sync byte followed by frame type and version number.
Number of bytes in frame
PMU/DC ID number, 16-bit integer
SOC timestamp
Fraction of Second and Time Quality
ASCII character, 1st byte
ASCII character, Kth byte, K>0 is an integer.
CRC-CCITT check
Table 4: Header frame
3.4 RTDMS Server
RTDMS Server application receives the data in PC37.118 format at around 30 samples per second.
The input data is then checked for type of frame that has arrived i.e. a data frame, configuration frame
or command frame. At the same level, the input data is validated for any erroneous input like the
length, checksum etc. The input data is then scaled against “configuration frame” to obtain various
derived quantities such as MW and MVARs. The output data obtained is stored in 1- dimesnsional
and 2-dimensional safe arrays. Output is updated in the database archive using the server GUI
application. RTDMS client application queries the data updated in database using remote web service
for various monitoring purposes.
The output of the PC37.118 driver utility in RTDMS Server is summarized in the following table along
with example.
Output
Comment
Company Name
Contains Station ID
PMU/PDC Channel
flags
PMU Status bytes
Phasors
Contains information about
• Channel Name
• Station Name
• Signal type • Signal Voltage level
In terms of Magnitude and Phase
angle for both Voltage and
Current.
Example
CALY
CALY:Channel flag (H)
CALY:Channel Flag (L)
CALY:PMU 0 Status bytes
Example is for Magnitude:
• Mtgy-Cal-7 Line Voltage
• CALY
• VM / IM (depending on Voltage or
current)
• 345.00
Similarly for Phase angle where
signal type is replaced by VA / IA,DF
(df/dt), AN (Analog), DG(Digital), ST
(Status)
7
RTDMS™ Continuous Phasor Data Archival Functional Specification
Frequency
MWs and MVARs
Digitals
Additional
Has information about
• Station Name
• Station Name
• Signal type
• Signal Voltage level
•
•
•
•
Callaway Freq
CALY
FR
59.97
Has information about
• Station Name
• Mtgy-Cal-7 Line Current
• Station Name
• CALY
• Signal type
• PP for MW / PQ for MVAR
• Signal Voltage level
• 345.00
Has information about the digital CALY:Digital 0
signals
CALY:Digital 1
•
•
•
•
FNominal
Data rate
Configuration Count
Scaling factors
•
•
•
•
60 Hz
Sample rate
Number of Configuration changes
Conversion factors
Table 5: PC37.118 Utility output
The output summarized above are returned in six arrays:
 Analognamesarray(Signal names)
 AnalogPMUCodesArray(Signal PMU code)
 AnalogVLsArray(Signal Voltage level of Phasors, frequency, MW & MVARs)
 AnalogTypesArray(Signal type PP/PQ/VM), DigitalNamesArray(PMU/PDC channel flags)
 DigitalDataArray(status bytes, Digitals)
 List of missing PMUCodes
4. COMTRADE Format
The focus is on the configuration file portion which defines the data. Nothing in this recommendation
is outside of the COMTRADE format, so all present and future methods are applicable, including
ASCII, binary, and floating point data file types as well as single and multiple file formats. In the
following descriptions, the terms phasor value, analog value, and digital value refer to phasor, analog,
and digital values that are referenced in the synchrophasor standard, C37.118.
4.1 Configuration File
The COMTRADE standard configuration file is given below for reference and introduction.
station_name,rec_dev_id,rev_year <CR/LF>
TT,##A,##D <CR/LF>
An,ch_id,ph,ccbm,uu,a,b,skew,min,max,primary,secondary,PS <CR/LF>
An,ch_id,ph,ccbm,uu,a,b,skew,min,max,primary,secondary,PS <CR/LF>
An,ch_id,ph,ccbm,uu,a,b,skew,min,max,primary,secondary,PS <CR/LF>
An,ch_id,ph,ccbm,uu,a,b,skew,min,max,primary,secondary,PS <CR/LF>
…
Dn,ch_id,ph,ccbm,y <CR/LF>
Dn,ch_id,ph,ccbm,y <CR/LF>
…
lf <CR/LF>
nrates <CR/LF>
samp,endsamp <CR/LF>
samp,endsamp <CR/LF>
8
RTDMS™ Continuous Phasor Data Archival Functional Specification
…
dd/mm/yyyy,hh:mm:ss.ssssss <CR/LF>
dd/mm/yyyy,hh:mm:ss.ssssss <CR/LF>
ft <CR/LF>
timemult <CR/LF>
Following table lists the configuration file fields, brief description, data type and equivalent information
from C37.118 stream. The Field names highlighted in bold are critical to COMTRADE file format and
should be filled. If any of the non critical fields are not available then they will be left blank with
consecutive delimiters following each other. The program reading these file should be able to tolerate
empty fields with consecutive delimiters.
Line
Field Name
Description
Data Type
1
station_name
name of the substation
location
Identification number or
name of the recording
device
Year of the COMTRADE
standard revision
Alphanumeric,
0-64 characters
Alphanumeric,
0-64 characters
TT
Total number of
channels
##A
Number of analog
channels followed by
identifier A
Number of digital (status)
channels followed by
identifier D
Analog channel index
number
Numeric, integer,
1-7 characters
(1 to 9999999)
Alphanumeric,
2-7 characters
(0A to 999999A)
Alphanumeric,
2-7 characters
(0D to 999999D)
Numeric, integer,
1-6 characters
(1 to 999999)
Alphanumeric,
0-64 characters
Alphanumeric,
0-2 characters
Alphanumeric,
0-64 characters
Alphabetical,
1-32 characters
Real, numeric,
1-32 characters
rec_dev_id
rev_year
2
##D
3
An
ch_id
Channel identifier
ph
a
Channel phase
identification
Circuit component being
monitored
Channel units (e.g. kV,
kA)
Channel multiplier
b
Channel offset adder
skew
Channel time skew (in
µs) from start of sample
period
Range minimum data
ccbm
uu
min
Numeric,
4 characters
Real, numeric,
1-32 characters
Real, numeric,
1-32 characters
Integer, numeric
Comments
(C37.118 equivalent)
STN
Empty field means 1991
standard (1999 or higher
is useful for phasor data)
Must equal sum of ##A
and ##D
(ANNMR + DGNMR)
ANNMR
DGNMR
Sequential, without
regard to device channel
num
STN:CHNAM
User Input (Vref, Vnom,
Inom, etc.)
IEEE Standard
nomenclature
For Real, Imaginary and
Magnitude: PHUNIT x
10E-5
For floating point: 1
For Angle: User input
0 or user input for angle
offsets
0
For Integer: -32767
9
RTDMS™ Continuous Phasor Data Archival Functional Specification
value
max
Range maximum data
value
primary
Channel voltage or
current transformer ratio
primary factor
Channel voltage or
current transformer ratio
secondary factor
Primary or secondary
data scaling identifier
Status channel index
number
secondary
PS
4
Dn
ch_id
Channel name
ph
Channel phase
identification
Circuit component being
monitored
Normal state of status
channel
Nominal line frequency
(Hz)
Number of sampling
rates in the data file
ccbm
y
5
lf
6
nrates
7
samp
Sample rate (Hz)
endsamp
Last sample number at
sample rate
dd
Day of month
mm
month
yyyy
year
hh
Hour
mm
Minutes
ss.ssssss
seconds
8
1-6 characters
(-99999 to 99999)
Integer, numeric
1-6 characters
(-99999 to 99999)
Real, numeric,
1-32 characters
For floating point: -99999
For Integer: 32767
for floating point: 99999
Real, numeric,
1-32 characters
Alphabetical,
1-1 character
Numeric, integer,
1-6 characters
(1 to 999999)
Alphanumeric,
0-64 characters
Alphanumeric,
0-2 characters
Alphanumeric,
0-64 characters
Numeric, integer,
1-1 character
Real, numeric
0-32 characters
Integer, numeric
1-3 characters
(0 to 999)
Real, numeric
1-32 characters
Integer, numeric
1-10 characters
(1 - 9999999999)
Integer, numeric
1-2 characters
(1 to 31)
Integer, numeric
1-2 characters
(1 to 12)
Integer, numeric
4 characters
(1900 to 9999)
Integer, numeric
2 characters
(00 to 23)
Integer, numeric
2 characters
(00 to 59)
Decimal, numeric
9 characters
(00.000000 to
59.999999)
only p,P,s,S are valid
Sequential, without
regard to device channel
num
STN:CHNAM
User Input
Only 0, 1 are valid
DGUNIT
FNOM
DATA_RATE
Time of first data in data
file
dd/mm/yyyy grouped
field, separated by /
24h format
Resolution = 1
microsecond
10
RTDMS™ Continuous Phasor Data Archival Functional Specification
9
10
ft
11
timemult
Fields are same as line 8
Data file type
Multiplication factor for
the timestamp in data file
Alphabetical,
5-6 characters
Real, numeric
1-32 characters
Time of the trigger point
BINARY
To allow long recordings
1 or user input
Table 6: COMTRADE Configuration File
Configuration File Field details:
The first line is the station name, recording device ID, and version of the standard. For
synchrophasor use, the station name will usually be the recording location. The device ID identifies
the recording device and this is left to the user. The earliest version useable for synchrophasor data
is 1999.
The second line of the configuration file is the count of channels by type.
There will be 2 analog channels (##A) for each phasor value and 1 channel for each analog value.
The 2 channels representing phasor values will be grouped together as described under channel
information below.
##D is number of status or digital channels. In synchrophasor data, digital values come in blocks of
16 channels, the same as data is represented in COMTRADE in binary format. The channel name
and type information may be read from the synchrophasor configuration and parsed into the
COMTRADE config file as described below. Synchrophasor data also includes time and data quality
information that is not normally included in COMTRADE type data. This information needs to be
included in the recorded file, and the method is also described below.
Analog channel information:
The next block of configuration file entries is the analog channel descriptions. Analog channel
information includes both phasor data and analog data as defined in the SYNCHROPHASOR
standard. Phasors are always represented as a complex number and recorded as successive analog
channels:
1. Rectangular components with a real value first and imaginary value second
2. Polar components with a magnitude value first and angle value second
The first channel of the pair may be an even or odd numbered channel depending on what channels
have preceded it. The overall order of analog channels in the file can be set by the user, though it is
suggested following the same general order used by the synchrophasor standard for data
transmission.
An,ch_id,ph,ccbm,uu,a,b,skew,min,max,primary,secondary,PS <CR/LF>
An = Analog channel index is a sequential number starting with 1
ch_id = combination of station name and channel names as given in C37.118. Both are 16 bytes
each and will be connected with a colon. (station_name:channel_name)
ph = The channels are identified by the 2 characters XY as follows:
X = Phase where
A  A phase, single phase
B  B phase, single phase
C  C phase, single phase
R  R phase, single phase
S  S phase, single phase
T  T phase, single phase
1  1 phase, single phase
2  2 phase, single phase
11
RTDMS™ Continuous Phasor Data Archival Functional Specification
Y = Phasor component where
3  3 phase, single phase
P or +  positive sequence
N or -  negative sequence
Z or 0  zero sequence
r  real for rectangular components
i  imaginary for rectangular components
m  magnitude for polar components
a  angle for polar components
ccbm = This is a freeform field that will be left to the user except that keyword quantities may be
included. The key words defined below will be used for machine identification of the defined
parameters and should not be used in this field for any other purpose. The key word quantities should
be separated from other words by spaces and must be used exactly as shown below (case sensitive).
These key word values can easily be identified and recorded by machine methods. A key word is a 4
character mnemonic followed by the equals (=) sign followed by the identified parameter. There are
to be no spaces between characters; that is, no spaces before or after the = sign. Inclusion of key
words is not required. Note that a machine parser should scan the entire field for key words since
they may be located anywhere in the text, in any order, and more than one may be included. Add
other key words as required, keeping care that the field is limited to 32 characters which could rarely
be more than 3 key words per quantity.
Key words and their usage:
Vref this is used for a current phasor to designate the voltage channel, An, that is required
for computing power, P = EI*. Since voltage and current phasors occupy two successive channels,
this key word must accompany the first channel of the current phasor pair and refer to the first
channel of the voltage phasor pair. For example, Vref=1123 indicates the channels 1123 and 1124
are voltage phasor data and that voltage will be used with the given current to compute power
Vnom this is used for a voltage phasor to indicate the nominal voltage. This key word must
accompany the first channel of the voltage phasor pair. For example,
Vnom=345 indicates the nominal voltage is 345 kV.
Inom this is used for a current phasor to indicate the nominal current. This key word must
accompany the first channel of the current phasor pair. For example, Inom=3000 indicates the
nominal current is 3000 Amps.
uu = alphanumeric channel units as described in the COMTRADE standard.
Scaling and offset are applied ax + b where x is the data value stored in the file. b must be in the
same units as ax.
a = According to the 1999 standard, the contents of the “.dat” file must be in integer counts. For real,
imaginary, and magnitude component channels, this is the integer PHUNIT value with the 10E-05
scaling applied. That is, this is PHUNIT x 10E-5 as described in the SYNCHROPHASOR standard. If
data is stored in floating point, this number will be set to 1, also as described in the standard. For
angle component channels this number can be used at the user’s discretion
b = The offset is normally used to define the zero level for channels that are single ended or have
floating references. For angle component channels this number can be used to adjust angle offsets,
such as Y-Delta connections and inter-area differences
Scaling and offset are applied ax + b where x is the integer value stored in the “.dat” file. b must be in
the same units as ax.
skew = Phasor data is by definition synchronized within microseconds and theoretically to within less
than one microsecond, so this value should be set accordingly as described in the COMTRADE
standard.
12
RTDMS™ Continuous Phasor Data Archival Functional Specification
min = data range minimum value. For integer phasor and analog data this will be -32767. For floating
point, set this value to -99999 to indicate it can be out of the range of this factor.
max = data range maximum value. For integer phasor and analog data this will be 32767. For floating
point, set this value to 99999 to indicate it can be out of the range of this factor.
primary = PT/CT primary ratio factor as described in COMTRADE standard
secondary = PT/CT secondary ratio factor as described in COMTRADE standard
P/S = Enter P or S and use as described in COMTRADE standard
Status (digital) channel information block:
Dn, ch_id, ph, ccbm, y <CR/LF>
In the COMTRADE standard, Boolean channels represented by a single bit set to 1 or 0 are called
status channels. In the SYNCHROPHASOR standard they are called digital channels. In this latter
standard, digital channels are always grouped in blocks of 16 channels while in COMTRADE they are
represented singly in the ASCII file type but as 16 channel blocks in binary files. The data blocks will
often contain unused channels. When SYNCHROPHASOR data is stored as ASCII files it is
permissible to drop the unused channels provided the recording device has provided a means to sort
those used from those unused. With binary file types the recording will be the same for both types. It
is permissible to only list the channels that are being used in the COMTRADE configuration file.
Viewing devices or programs that convert file data back to the SYNCHROPHASOR data stream
format will be required to provide default information for the unrepresented digital channels.
Dn
digital channel index number
ch_id Channel name. Combination of station name and channel names as given in C37.118. Both
are 16 bytes each and will be connected with a colon. (station_name:channel_name)
ph
channel phase identification. Use as defined in the COMTRADE standard table.
ccbm circuit component being monitored. Use as defined in the COMTRADE standard table.
Y
is the normal state of the status channel, i.e., the state of the input when the primary apparatus
is in the steady state “in service” condition. This does not represent the physical state of the status
signal, whether there is clean contact (open or closed) or a voltage (live or dead). The purpose is to
define whether a 1 represents the normal or abnormal state.
The first 8 digital channels are reserved for bits of the time quality byte included in the fractional time
(FRACSEC) long work contained in the C37.118 time stamp. The next 8 digitals are reserved for
future use, and for the purpose of keeping the individual PMU station status indications on even 16-bit
words. The next 16 digital channels are reserved for bits of the status word for the first station
represented in the data sample. If there is more than 1 station included in the frame of data that is
represented, the status of each station is to be represented as 16 channel groups in the same order
as they appear in the data frame. The binary form of COMTRADE data files allocates digital status
information into 16-bit (or larger) blocks; the status for each station can be written directly into a binary
formatted file since digital status is allocated to 16-bit words. Each status bit is to be named by the
station name with an extension identifying the bit. This is the same station name used in identifying
the signals, so the status of each signal is identifiable. Any digital status data included in the
synchrophasor data should follow after the time quality and PMU status indications. The exact mode
of representation is shown below.
13
RTDMS™ Continuous Phasor Data Archival Functional Specification
D1, TQ_CNT0, T0,,0
first 8 bits are the Time Quality from the FRACSEC word
D2,TQ_CNT1, T1,,0
D3,TQ_CNT2, T2,,0
D4,TQ_CNT3, T3,,0
D5, TQ_LSPND, T4,,0
D6, TQ_LSOCC, T5,,0
D7, TQ_LSDIR, T6,,0
D8, TQ_reserved, T7,,0
next 8 bits are skipped to preserve 16-bit word structure for station status
D17, <station1>_TRG1, S0,,0
next 16-bits are the status for the first station in the data message
D18, <station1>_TRG2, S1,,0
D19, <station1>_TRG3, S2,,0
D20, <station1>_TRG4, S3,,0
D21, <station1>_UNLK1, S4,,0
D22, <station1>_UNLK2, S5,,0
D23, <station1>_SEC1, S6,,0
D24, <station1>_SEC2, S7,,0
D25, <station1>_SEC3, S8,,0
D26, <station1>_SEC4, S9,,0
D27, <station1>_CFGCH, SA,,0
D28, <station1>_PMUTR, SB,,0
D29, <station1>_SORT, SC,,0
D30, <station1>_SYNC, SD,,0
D31, <station1>_PMUERR, SE,,0
D32, <station1>_DTVLD, SF,,0
D33, <station2>_TRG1, S0,,0 next 16-bits are the status for the second station in the data message
D34, <station2>_TRG2, S1,,0
…
D(16(n+1)), <stationn>_DTVLD, SF,,0 end of last status of last for station in message with n stations
D(16(n+1)+1), name, etc,,0
start of all digital status indications contained in data
There are 12 additional fixed parameters specified in COMTRADE configuration.
lf <CR/LF>
lf
line frequency; This should always be 50 or 60 to indicate the system frequency.
nrates <CR/LF>
nrates A file of phasor data will normally be made using a single sampling rate, so this will usually be
1. It can be > 1 in accordance with COMTRADE variable rate recording.
samp, endsamp <CR/LF>
samp
This is the sample or frame rate of phasor data and there will normally be only one
rate. In accordance with COMTRADE variable rate recording, there can be more than one entry with
several rates in the file.
endsamp
With a single sample rate, this number is the last sample in the file. With multiple rates
in the file, several samp, endsamp pairs will indicate the relevant blocks.
dd/mm/yyyy,hh:mm:ss.ssssss <CR/LF>
timestamps two timestamps are given. The first one is the time of the first data item in the file; the
second is the point at which the file was triggered (trigger point).
ft <CR/LF>
ft
This parameter can only be ascii or ASCII for a data file using the ascii format, or binary or
BINARY for a data file using the binary format.
14
RTDMS™ Continuous Phasor Data Archival Functional Specification
timemult <CR/LF>
timemult:
This factor is used to multiply all the time stamps given in the data file to achieve a
different time scale. Time stamps in the data file are given in microseconds in a 10 character field
which limits the time span of the file to 9999999999 microseconds or about 10000 seconds. By
multiplying these time stamps by a scale factor allows storing a much longer time record. Since the
data interval between samples for phasor data is usually 16667 microseconds or larger, timemult can
be increased to allow exchange of longer data records. For example, if timemult = 1000 then data for
a time period of 10000000 seconds (over 115 days) can be represented in a single record.
Missing Data in configuration files:
The configuration file format provides for the fact that some data may be unavailable. However, it is
understood that lack of some critical data can make the file set unusable. Some data are therefore
specified as noncritical and some as critical. Loss, or lack, of critical data in the configuration file
renders the file set invalid and as not conforming to the standard. Loss, or lack, of non-critical data in
the configuration file does not render the file non-conforming and does not make the file set unusable.
When data are missing, the data separators follow each other with no intervening characters unless
otherwise specified elsewhere in this clause. Programs intended to read COMTRADE files shall be
written to tolerate data separators immediately following each other with no intervening spaces (null
fields).
4.2 Data File
The data file type (ft) field is defined in the configuration file. Data filenames shall have the .DAT
extension. The data file contains the sample number, time stamp, and data values of each channel for
each sample in the file. All data in data files are in either integer or floating point format. In binary files
there are no separators between the data for each channel within a sample or between sequential
sample periods. No other information is contained in the data file.
In Binary data file each data sample record shall consist of numbers arranged as follows:
where
n timestamp A1A2…AkS1S2…Sm
n = sample number. critical, integer, numeric, 4 bytes, 00000001 to FFFFFFFF
timestamp = time stamp. Non-critical if nrates and samp variables in .CFG file are nonzero. Critical if
nrates and samp variables in .CFG file are zero. Minimum length = 4 bytes, maximum length = 4
bytes, minimum value = 00000000, maximum value = FFFFFFFE
The format is sample number, timestamp, data value for each analog channel, and grouped status
channel data for each sample in the file. No data separators are used; the data within a binary sample
record is not separated by commas and the end of a sample record is not marked by carriage
return/line feed characters. A binary data file is a continuous stream of binary data. Data translation is
determined by sequential position within the file. If any data elements are missing or corrupt, the
sequence of variables will be lost and the file may be unusable. No provision is made for recovery
under these circumstances.
When storing a two-byte (16 bit) word, the standard DOS format is to store the least significant byte
(LSB) of the data first, then the most significant byte (MSB). The two-byte data value “1234” will be
stored in “3412” format. In storing a four-byte (32 bit) word, the least significant byte (LSB) of the
word is stored first, then the next to least significant byte, then the next to most significant byte, then
the most significant byte (MSB). The four-byte data value “12345678”, will be stored in “78563412”
format. The bits within a byte are numbered zero (least significant) to seven (most significant).
15
RTDMS™ Continuous Phasor Data Archival Functional Specification
The sequential data in a binary data file represent the following:
a) Sample number and time stamp data are stored in unsigned binary form of four bytes each.
b) Analog channel sample data are stored in two’s complement binary format of two bytes each. A
data value of zero is stored as 0000 hexadecimal, -1 is recorded as FFFF. The maximum positive
value is 7FFF, and the maximum negative value is 8001. The hexadecimal 8000 is reserved to mark
missing data.
c) Status channel sample data are stored in groups of two bytes for each 16 status channels, with the
least significant bit of a word assigned to the smallest input channel number belonging to that group of
16 channels. Thus, bit 0 of status word 1 (S1) is the status of digital input number 1, while bit 1 of
status word 2 (S2) is the status of digital input number 18. No provision is made for marking missing
status data, but a bit set to 1 or to 0 must be included to maintain the integrity of the word.
The length of the file will vary with the number of channels and the number of samples in the file. The
number of bytes required for each sample in the file will be: (Ak x 2) + (2 x INT(Dm/16)) + 4 + 4.
Where,
Ak = number of analog channels
INT(Dm/16) = number of status channels divided by 16 and rounded to nearest integer
4+4 = 4 bytes each for the sample number and time stamp
16
RTDMS™ Continuous Phasor Data Archival Functional Specification
5. RTDMS Server GUI
The RTDMS Server GUI has to be updated to include configuration options for the continuous data
recording functionality. Following figure illustrates the new configuration options:
Figure 2: Continuous Data File Configuration in RTDMS Server GUI
The new configuration option shall reside in the RTDMS Server GUI tab titled “Cache/Archive”. The
GUI has the following properties:
 Data file archival process will be triggered when Enable continuous data file archive is checked.
This will enable selection of other options which will be by default grayed out.
 Enable data file segmentation will split the files according to the length given in minutes.
 User will have option to store the archive files in a desired folder. The application should check for
write access to the folder
 Total duration of file storage will be configured by limiting archive size. This is checked by
default with default 96 hours (archive length). Minimum free space by default will be 500 Mb.
 Total disk space is the total target storage disk capacity
 Space available is the current available space in the target storage disk.
 Approx. Space needed is the space needed estimated using the archive length specified by user
 In the graphic bar, the bar length is the available space, red line is the estimated space needed for
archival and the solid red bar is the filled up space at the moment when the GUI is opened
17
RTDMS™ Continuous Phasor Data Archival Functional Specification
6. SYSTEM ARCHITECTURE
The continuous data archiving function will be integrated to the RTDMS Server application. It will read
parsed and scaled data from the available 5 min cache buffer in the RTDMS Server and convert them
to COMTRADE configuration (*.cfg) and binary data (*.dat) files for storage. Following figure illustrates
the existing system architecture and proposed system:
Figure 3 (a) and (b): Current and Proposed Implementation
In the current RTDMS Server setup, there are two data cache buffers. One stores one hour of filtered
data and alarm/event data which is used by real time monitoring and alarming client applications. The
second buffer stores 5 min of filtered data which is stored in event files when an event is detected.
In the proposed implementation, the 5 min data cache will not receive filtered data but the parsed and
scaled data. This data will be read by the continuous data file recorder function and will be stored in
data files as configured by user.
7. IMPLEMENTATION SCHEDULE
Task
Complete functional specifications
Development Completion
Factory Testing in Pasadena
Field Trial, Documentation & Training
Deployment
Schedule
March 30th 2009
August 15th 2009
August 31st 2009
September 15th 2009
September 31st 2009
Table 7: Implementation Schedule for the Continuous Data Recorder
18
RTDMS™ Continuous Phasor Data Archival Functional Specification
19