AEIC Annual Load Research Conference
September 12, 2006
-
Reno, NV
ERCOT
Residential Profile ID Assignments
– Dealing with Assignment
Accuracy and Migration
Presented By: Diana Ott
Carl Raish
1
Overview
• ERCOT Settlement highlights
• Residential Annual Validation
• Heating Fuel Type Residential Survey
• Impact of Miss-Assignment of
Residential Load Profile ID
Assignment
• New Residential Algorithm
• Q&A
2
Settlement
•
ERCOT requires a fifteen (15) minute settlement interval
•
Vast majority of Customers do not have this level of
granularity.
•
Profiles are created using adjusted static models
• Models are dependent on season, day of week, time of
day and weather
• Backcasted Profiles are generated the day following a
trade day and used for all settlements (initial, final and
true-up)
•
Load Profiling:
• Converts monthly NIDR reads to fifteen (15) minute
intervals
• Enables the accounting of energy usage in settlements
• Allows the participation of these Customers in the retail
market (reduces barrier to entry)
3
3 Load Profile Groups, 9 Segments
Residential (2)
Non-Metered (2)
Business (5)
Low-Winter Ratio
(Non-electric Heat)
High-Winter Ratio
(Electric Heat)
Low Load Factor
Medium Load Factor
High Load Factor
Non-Demand
IDR Default
Lighting (Street Lights)
Flat (Traffic Signals)
4
8 ERCOT Weather Zones
Stars represent the location for the 20 ERCOT Weather
Stations for each Weather Zone
5
Annual Validation of Profile Assignments
•
ERCOT in conjunction with Profiling Working Group
establishes the rules for Profile ID assignment and
publishes in the form of a Decision Tree on the ERCOT
website
•
Annual Validation is a process established by the Market
to annually review and update Profile ID assignments
based on the rules defined in the Decision Tree
•
Historically, May 1 thru April 31 meter reads were used to
determine the Annual Validation assignment. The
process normally began in June and completed in
January.
6
History of Residential Annual Validation
•
Oct. 2001 Initial Validation
•
•
•
2002 Annual Validation
•
•
•
Large volume of migrations (1.5 million out of 4.9 million ESIIDs)
2004 Annual Validation
•
•
•
Not performed due to 2001 Initial Validation still in progress
PWG sub team changed methodology from using billing month to usage month
2003 Annual Validation
•
•
Profile IDs were assigned by TDSPs prior to Market Open
Validation started in 2001 and was not completed until Sept. 2002
Large volumes of changes were identified (1.0 million out of 5.4 million ESIIDs)
Annual Validation suspended to allow time to improve assignment process
2005 Annual Validation
•
•
Some methodology changes were identified which still resulted in large volumes
of migrations (0.5 million out of 5.1 million ESIIDs)
Market delayed sending in transactions and ultimately decided to only send in a
subset of changes identified
7
Residential Assignment Rules
2001 - 2004


Winter Ratio >=1.5 RESHIWR
Winter Ratio < 1.5 RESLOWR
 Max ( ADUsedec , ADUse jan , ADUse feb ) 
WR  

*
Average
(
Fallbase
,
Springbase
)


*
* Round to two decimal places
Where ADUsedec = Average Daily Use in the December Usage Month,
ADUsejan = Average Daily Use in the January Usage Month,
ADUsefeb = Average Daily Use in the February Usage Month,
FallBase = minimum ADUse for the Usage Months of October and November
SpringBase = minimum ADUse for the Usage Months of March and April.
8
Preliminary Residential Assignment Rules
for Annual Validation 2005
• Do not replace a non-default
assignment with a default assignment
• Apply Dead-Bands
• RESHIWR goes to RESLOWR if WR ≤ 1.0
• RESLOWR goes to RESHIWR if WR > 1.8
• Dead-Bands do not apply if currently a
default assignment
• kWh Minimums
• WR numerator ≤ 20 then assign RESLOWR
9
Additional Profile Assignment
Improvement Ideas
• Use a statistical approach to correlate premise usage
to profile usage.
• Use a residential survey to obtain the necessary data
to relate usage patterns to heating system type.
•
•
More accurately account for weather variations
Account for periods of low/no occupancy
• Move calculation responsibility to ERCOT from
TDSPs
• Change time period for submission of assignment
change transactions
•
•
During the original October/November timeframe for submitting changes, the
RESHIWR and RESLOWR profiles are significantly different
RESHIWR and RESLOWR profiles are quite similar during the summer months
10
Residential Heating & Fuel Type Survey
•
Design:
•
•
41,000 bilingual survey forms mailed
Stratified by Weather Zone and Profile Type
•
•
•
2,563 RESHIWR per Weather Zone
2,562 RESLOWR per Weather Zone
Response
• Survey responses were identified to allow connecting the
response to usage history
• 4,669 responses as of 09/30/2005
• 11.4% response rate
11
Questions from Residential Survey
pertinent to Electric Heat Analysis
What classification best fits this address? (Check only one box)
•  Single-Family Dwelling
•  Multi-Family Dwelling (Duplex, Apartments, etc.)
•  Other (Please skip the remaining questions and disregard the survey.)
What is the primary type of home heating used at this residence? (Check only one box)
•  Electricity
•  Natural gas or bottled gas (propane/butane)
•  Other or not sure
•
•
•
•
Have you added central electric cooling or heating in the last 2 years? (Check all that apply)
 Yes, I added central air conditioning
 Yes, I added central electric heating
 No
 Not sure
•
•
•
•
•
What is the approximate age of your residence? (Check only one box)
 Less than 5 years
 5 – 15 years
 16 – 30 years
 More than 30 years
 Not sure
12
Residential Heating & Fuel Type Survey
Number of Survey Responses by WZone & Profile Type
700
600
350
333
289
500
HIWR
252
231
HIWR
400
HIWR
HIWR
176
HIWR
205
268
HIWR
HIWR
HIWR
300
200
355
262
100
LOWR
LOWR
365
309
303
LOWR
LOWR
LOWR
325
LOWR
371
275
LOWR
LOWR
0
Return rate by
WZone
COAST
EAST
FWEST
NCENT
NORTH
SCENT
SOUTH
WEST
8.5%
12.6%
10.5%
9.9%
13.6%
11.3%
10.6%
14.1%
13
Residential Heating & Fuel Type Survey
Overall - Percent of Home Heat Type
(RESLOWR & RESHIWR)
100%
1.45%
0.35%
0.80%
0.98%
1.02%
0.33%
1.19%
1.18%
1.14%
52.2%
52.0%
46.7%
46.9%
WEST
ERCOT
90%
29.2%
80%
47.5%
70%
47.7%
57.3%
62.7%
62.5%
54.8%
60%
50%
40%
69.6%
30%
52.2%
20%
51.4%
41.6%
36.5%
36.0%
44.9%
10%
0%
COAST
Electricity
EAST
FWEST
NCENT
NORTH
SCENT
Natural/Bottled Gas
SOUTH
Other/Not Sure
14
Residential Heating & Fuel Type Survey
RESLOWR Respondents - Percent of Home Heat Type
100%
1.5%
0.6%
1.0%
1.0%
1.1%
0.3%
1.5%
1.3%
RESHIWR Respondents - Percent of Home Heat Type
1.2%
90%
90%
35%
80%
68%
74%
78%
68%
0.4%
1.0%
11%
13%
0.9%
18%
0.4%
0.4%
13%
11%
86%
89%
SCENT
SOUTH
0.9%
20%
0.9%
14%
29%
69%
80%
60%
50%
50%
40%
40%
81%
88%
87%
81%
79%
85%
71%
63%
30%
30%
20%
31%
10%
18%
0.0%
70%
69%
73%
1.1%
80%
70%
60%
100%
25%
21%
31%
26%
30%
30%
19%
10%
0%
COAST
EAST
FWEST
NCENT
NORTH
20%
SCENT
SOUTH
WEST
ERCOT
0%
COAST
Overall 30%
misclassified
Electricity
Natural/Bottled Gas
EAST
FWEST
NCENT
NORTH
WEST
ERCOT
Overall 14%
misclassified
Other/Not Sure
15
Residential Heating & Fuel Type Survey
ERCOT 2005 Residential Survey Results
Added Central Electric Cooling or Heating (last 2 years)
RESLOWR
COAST
EAST
FWEST
NCENT
NORTH
SCENT
SOUTH
WEST
ERCOT
Cooling
1.9%
4.8%
5.8%
3.0%
4.4%
1.8%
8.0%
5.1%
3.3%
Heating
0.0%
2.0%
2.6%
1.0%
1.6%
0.6%
3.6%
2.2%
1.0%
Both
0.0%
1.7%
1.9%
1.0%
1.6%
0.3%
3.3%
1.6%
0.9%
No
96.2%
93.2%
91.6%
93.4%
93.2%
94.2%
88.4%
91.6%
93.9%
Not Sure
1.1%
0.6%
0.6%
1.7%
0.3%
1.8%
1.5%
0.8%
1.3%
No
93.8%
94.8%
90.5%
90.2%
94.9%
92.5%
91.0%
92.3%
91.6%
Not Sure
1.7%
1.4%
2.2%
2.4%
0.6%
2.4%
1.1%
1.1%
2.0%
Survey indicates low
rate of heating system
type change
RESHIWR
COAST
EAST
FWEST
NCENT
NORTH
SCENT
SOUTH
WEST
ERCOT
Cooling
3.4%
2.4%
6.1%
6.3%
4.2%
4.4%
6.3%
5.4%
5.4%
Heating
2.3%
1.7%
2.2%
2.9%
3.0%
2.8%
5.2%
2.9%
2.9%
Both
1.7%
1.4%
1.7%
2.9%
2.4%
2.8%
4.5%
2.6%
2.6%
16
Residential Heating & Fuel Type Survey
Primary Home Heat % for Homes Less than 5 years
100%
9.3%
90%
19.6%
23.8%
34.2%
38.9%
80%
44.2%
51.8%
70%
63.2%
60%
50%
83.8%
40%
80.4%
76.2%
65.8%
61.1%
30%
55.8%
48.2%
20%
36.8%
10%
6.9%
0%
Coast
East
Fwest
Other/Not Sure
NCent
North
Electricity
SCent
South
Natural/Bottled Gas
West
17
Residential Heating & Fuel Type Survey
•
Performed visual inspection of usage
patterns for each survey response
•
4,630 responses indicated either a “Single-Family
Dwelling” or “Multi-Family Dwelling” and a primary
home heating type of either “Electricity” or “Natural gas
or bottled gas (propane/butane)
•
673 (14.5%) responses to the home heating type were
deemed invalid by examination of their seasonal usage
pattern
•
3,957 (85%) responses were used to develop an
improved Profile Type classification algorithm
18
Survey Response Validation
- Electric Heat Example
Note:Summer usage values
omitted for clarity
19
What we found out from the Survey
•
Saturation of Electric Heat varied considerably across weather
zones
•
Saturation of Electric Heat was inconsistent with breakdown
between RESHIWR and RESLOWR
•
30% of Survey responders reporting Electric Heat were assigned
to RESLOWR
•
14% of Survey responders reporting No Electric Heat were
assigned to RESHIWR
•
There is very little year-to-year change in heating system fuel
actually occurring
•
The percent of newer homes using electric heat varies
considerably across weather zones
(37% Coast – 84 South %)
20
Why Does Assignment Accuracy Matter?
•
Profile assignment errors create two types of load
profile estimation errors
• Assignment of billing kWh to the days within the
billing period
• (RESHIWR assigns more kWh than RESLOWR to
cold days)
• Assignment of daily kWh to the intervals within the
day
• (RESHIWR assigns more kWh to morning
intervals)
21
Daily kWh as a Percent of Monthly kWh
wz o n e = COA S T g r o u p = 2 0 0 4 / 0 5
Da i l y
Pe r c e n t
Reslowr
0. 08
mo n _ p c t _ l o
Reshiwr
mo n _ p c t _ h i
0. 07
Daily kWh Pct
0. 06
0. 05
0. 04
0. 03
0. 02
0. 01
0. 00
050104
060104
070104
080104
090104
100104
110104
usedat e
Trade Day
120104
010105
020105
030105
040105
050105
22
Residential Profile Comparison
- FWEST Reshiwr vs. Reslowr
2
0%
-50%
1
-100%
0.5
-150%
0
-200%
-0.5
-250%
-1
-300%
-1.5
-350%
-2
-400%
12/10/05 12/11/05
12/4/05
12/5/05
12/6/05
12/7/05
Reslowr
12/8/05
Reshiwr
12/9/05
Pct_Diff
% Diff
kWh
1.5
23
Findings and Next Steps
•
ERCOT’s Profile ID Assignment process has resulted in
unacceptably high migration rates
•
Dead - bands would reduce migration but could do
more harm than good in terms of assignment accuracy
•
The impact of Profile ID miss-assignment is significant
at the ESIID level
•
Undertake an effort to develop a new and improved
assignment process with a goal of reducing migration
and improving accuracy
•
More improvements are needed
24
Classification Algorithm Overview
•
Use Residential Survey response data in conjunction with
responder usage data to build an algorithm to predict
heating fuel
•
Use regression between actual meter readings for a premise
and the RESHIWR and RESLOWR profile kWh for the same
time periods
•
Use reads during shoulder and winter months for several
(4.5) years
•
Omit reads during periods of very low use (no/low
occupancy)
•
Omit outlier reads and require some reads to exceed a
minimum kWh/day threshold in order to assign RESHIWR
•
Assign the better fitting profile to the ESIID
25
Classification Algorithm Development
•
For each ESI ID with a survey response usage values were
selected from Lodestar for the January 2002 – September
2005 time period
•
Each usage value was converted into units of kWh/day and
any read covering a period longer than 44 days was dropped
•
Each usage value was classified as a winter or shoulder
reading
•
•
Only shoulder and winter readings were used in the analysis
•
Winter/Shoulder:
start > September 20 and stop < May 11
•
Winter:
start > November 15 and stop < March 15
•
Shoulder:
all others
Usage values were screened for high and low outlier usage
values
26
Classification Algorithm Development

For each ESI ID compute a mean and standard deviation of
the kWh/day values for the winter and shoulder readings and
use these to “normalize” each usage value
Z

UsageValue  Mean
StDeviation
Usage value dropped if:





Z > 3 and kWh/day > 100
Z > 3.5
Outliers
Z < -2
kWh/day < 5
Low Occupancy
27
Classification Algorithm Development
Usage Screening Examples:
Start Date
Stop Date
3/4/2002
4/3/2002
4/3/2002
5/3/2002
10/30/2002 12/3/2002
3/4/2003
4/3/2003
4/3/2003
5/2/2003
10/1/2003 11/3/2003
11/3/2003 12/4/2003
3/4/2004
4/1/2004
4/1/2004
5/5/2004
9/30/2004 10/29/2004
10/29/2004 12/2/2004
3/4/2005
4/5/2005
4/5/2005
5/3/2005
1/4/2002
2/4/2002
2/4/2002
3/4/2002
12/3/2002
1/7/2003
1/7/2003
2/3/2003
2/3/2003
3/4/2003
12/4/2003
1/7/2004
1/7/2004
2/4/2004
2/4/2004
3/4/2004
12/2/2004
1/5/2005
1/5/2005
2/2/2005
2/2/2005
3/4/2005
mean
standard deviation
kWh
865
445
1,581
495
309
380
248
170
185
352
281
981
889
1,309
1,815
2,189
1,802
1,984
240
171
.
1,228
1,437
1,175
kWh/day
28.8
14.8
46.5
16.5
10.7
11.5
8.0
6.1
5.4
12.1
8.3
30.7
31.8
42.2
64.8
62.5
66.7
68.4
7.1
6.1
.
36.1
51.3
39.2
Z
0.02
-0.61
0.82
-0.53
-0.80
-0.76
-0.92
-1.00
-1.03
-0.73
-0.90
0.10
0.15
0.62
1.64
1.54
1.73
1.80
-0.96
-1.00
0.35
1.03
0.49
Dropped Dropped
kWh
kWh/day
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
132
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4.6
.
.
.
28.3428
22.2221
Usage less than 5 kWh/day dropped
28
Classification Algorithm Development
Usage Screening Examples:
Start Date Stop Date
2/25/2002
3/25/2002
9/23/2002
10/22/2002
2/24/2003
3/26/2003
9/24/2003
10/22/2003
2/26/2004
3/25/2004
9/23/2004
10/22/2004
2/24/2005
3/24/2005
1/25/2002
11/20/2002
12/23/2002
1/24/2003
11/20/2003
12/26/2003
1/26/2004
11/22/2004
12/27/2004
1/26/2005
mean
standard deviation
3/25/2002
4/25/2002
10/22/2002
11/20/2002
3/26/2003
4/25/2003
10/22/2003 .
11/20/2003
3/25/2004
4/23/2004
10/22/2004
11/22/2004
3/24/2005
4/26/2005
2/25/2002
12/23/2002
1/24/2003
2/24/2003
12/26/2003
1/26/2004
2/26/2004
12/27/2004
1/26/2005
2/24/2005
kWh
kWh/day
1931
1570
1880
1784
2284
1710
68.964
50.645
64.828
61.517
76.133
57
.
1488
1561
1527
1791
1720
1424
1296
2492
2719
3349
2877
2526
2297
2676
2592
1980
1871
51.31
55.75
52.655
61.759
55.484
50.857
39.273
80.387
82.394
104.656
92.806
70.167
74.097
86.323
74.057
66
64.517
Z
0.22
-0.81
-0.02
-0.20
0.62
-0.46
-2.47
-0.78
-0.53
-0.70
-0.19
-0.54
-0.80
-1.46
0.86
0.97
2.23
1.56
0.28
0.51
1.19
0.50
0.05
-0.03
Dropped Dropped
kWh
kWh/day
.
.
.
.
.
.
595
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
21.25
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
65.1179
17.7623
29
Usage with Z < -2.00 dropped
Classification Algorithm Development
Usage Screening Results
•
1,006 ESI IDs (21.7%) with one or more usage values
screened
•
2,414 usage values were screened out
•
1,825 usage values screened out because < 5 kWh/day
•
If an ESI ID had fewer than 3 winter readings or fewer
than 3 shoulder readings it was classified as
“RESLOWD” (Residential Low Winter Ratio Default)
and was not used for fine tuning the algorithm
30
Classification Algorithm Development
Algorithm Basics
•
If an ESI ID has (and uses) electric heating, then the winter
and shoulder usage values for that premise should be
more similar to the RESHIWR profile kWh than to the
RESLOWR profile kWh
•
The profile kWh for a day reflects the weather conditions
associated with that day in the specific weather zone as
well as the day type (day-of-week/holiday) and season of
the year
•
To perform the comparison for an ESI ID, the profile kWh is
summed across the intervals for the days in each of its
meter reading periods (shoulder and winter months only)
31
Classification Algorithm Development
Algorithm Basics
•
For each fall-winter-spring time period e.g., fall 2004 – spring 2005
the profile kWh is scaled to equal the sum of the ESI ID’s meter
kWh for that time period
•
The correlation between the actual metered kWh and the scaled
profile kWh for those readings is computed for each ESI ID
•
The R2 correlation is determined with a weighted linear regression analysis with
no intercept term
•
Each reading is weighted as follows:
Shoulder reading weight = 1
Winter reading weight =
2  RESHIWR kWh
RESLOWR kWh
Winter reading weight = 1 if RESHIWR kWh < RESLOWR kWh
•
The weighting process associates more importance with winter readings for
which the RESHIWR kWh is greater than the RESLOWR kWh
32
New Algorithm Improvement Example
ESIID
Reshiwr
Reslowr
Note: New
Algorithm
improvement
results from
using multiple
years of usage
values
33
Fe
b0
Ap 2
r- 0
Ju 2
nAu 0 2
g0
O 2
ct
-0
De 2
c0
Fe 2
b0
Ap 3
r- 0
Ju 3
nAu 0 3
g0
O 3
ct
-0
De 3
c0
Fe 3
b0
Ap 4
r- 0
Ju 4
n0
Au 4
g0
O 4
ct
-0
De 4
c0
Fe 4
b0
Ap 5
r- 0
5
kWh
Classification Algorithm Development
Example ESIID Plotted
2,500
2,000
1,500
1,000
500
-
Reading Mid-point
ACTUAL
SCALED RESHIWR
SCALED RESLOWR
34
Classification Algorithm Development
Algorithm - Classification Rules
1.
If the highest winter reading kWh/day is less than 15 kWh/day
then assign “RESLOWR”
2.
If R2RESHIWR > 0.60 and R2RESHIWR > R2RESLOWR
then assign “RESHIWR”
3.
If the number of readings available > 9
and R2RESHIWR > 0.90
and (R2RESHIWR + 0.010) > R2RESLOWR
and Winter Max kWh/day > 50
then assign “RESHIWR”
4.
If the number of readings available > 9
and R2RESHIWR > 0.95
and (R2RESHIWR + 0.015) > R2RESLOWR
and Winter Max kWh/day > 60
then assign “RESHIWR”
5.
Otherwise assign “RESLOWR”
35
Classification Algorithm Development
Algorithm – Rules Fine Tuning

Algorithm fine tuning was an iterative process to tune each
classification criterion on the previous slide individually

Each classification criterion was adjusted to minimize
misclassification error based on validated survey responses

For each iteration, misclassified ESI IDs were examined
graphically to assess the accuracy of the Profile Type
assignment and to establish new criteria

When the fine tuning was complete 184 (4.6%) validated
survey responses regarding heating system type were
different than the algorithm classification … most had usage
patterns which were ambiguous
36
Survey and Algorithm Agree on Classification
Reshiwr
Reslowr
Daily kWh
ESIID
37
Survey and algorithm both indicated electric heat, “RESHIWR”
Survey and Algorithm Disagree on Classification
Reshiwr
Reslowr
Daily kWh
ESIID
38
Survey said electric heat, algorithm said gas
Classification Algorithm Results

For the final version of the algorithm 3,773 (95.4%)
validated survey responses regarding heating system type
agreed with the algorithm classification
Definitely not electric heat!
39
Applying Algorithm to Annual Validation 2005

62% of the 578,572 AV 2005 Profile Type changes agreed with
the algorithm classification

Changes to RESHIWR were significantly more accurate (78.4%)
than changes to RESLOWR (43.5%)

Accuracy of the changes by weather zone ranged from a low of
59.8% in the SOUTH zone to a high of 68.8% in the EAST zone

The Residential population would have had somewhat more
accurate Profile Type assignments as a result of conducting AV
2005 (81.4% vs. 78.7%)

The market decided to allow only changes which were in
agreement with the algorithm (358,000 changes were
submitted)
40
Residential Changes for Annual Validation 2006
New algorithm adopted by market for AV2006, Calculation
responsibility shifted to ERCOT
Total Expected Changes
15.8%
Break Down of Changes
18.1% HIWR to LOWR (156,798)
81.9% LOWR to HIWR (711,015)
RESHIWR
RESLOWR
Current
1,789,799
3,685,490
5,475,289
Current %
32.7%
67.3%
Expected Expected %
2,344,016
42.8%
3,131,273
57.2%
5,475,289
41
Estimates of Future Load Profile ID Migrations
•Estimates below reflect migrations for 2006 if the new
algorithm had been used exclusively for 2005 Annual
Validation
•The estimated migration rates are an indicator of what can be
expected for year-to-year migration starting in 2007
•Estimates were developed from a sample of every 25th ESIID
Year 1 to Year 2 Migration Estimates
NonDefault Year to Year Migration Estimates
4.4%
3.6%
42
Conclusions

The survey successfully provided data necessary to build a
classification algorithm for electric heating and establish its accuracy.

The classification algorithm at 96% accuracy was a significant
improvement over the winter ratio method

The improved accuracy will lead to assignment stability

Profile assignments and shapes are in a feedback loop and improve
each other


The new algorithm uses load profile shapes to make profile assignments

With updated load research analysis based on the new assignments, more accurate load
profile shapes will be developed as a result of a more homogeneous population

The more accurate load profile shapes should lead to better assignments
ERCOT has completed load research analysis using the new profile
assignments and is developing new profile models based on those
latest estimates
43
44