Autotune Building
Models: Calibration,
Simulation Data,
and Data Mining
Data Science Seminar
5700, L202
Oct. 23, 2014
Joshua New, Ph.D.
865-241-8783
newjr@ornl.gov
ORNL is managed by UT-Battelle
for the US Department of Energy
A brief history of energy and life quality
2
2
Sustainability is the defining challenge
• Buildings in U.S.
– 41% of primary
energy/carbon 73%
of electricity, 34% of
gas
• Buildings in China
– 60% of urban
building floor space
in 2030 has yet to be
built
• Buildings in India
– 67% of all building
floor space in 2030
has yet to be built
3
3
Energy Consumption and Production
Commercial Site Energy
Consumption by End Use
TN 2012 Electric Bill - $1,533
4
4
Presentation summary
• Scientific Paradigms
• Roof Savings Calculator
• Visual Analytics
• Knowledge Work
• Autotune
• Publications
5
5
Presentation summary
• Scientific Paradigms (context)
• Roof Savings Calculator
• Visual Analytics
• Knowledge Work
• Autotune
• Publications
6
6
4th Paradigm –
The Science behind the Science
• Empirical – guided by experiment/
observation
– In use thousands of years ago, natural
phenomena
• Theoretical – based on coherent group of
principles and theorems
Tycho Brahe
Johannes Kepler
– In use hundreds of years ago,
generalizations
• Computational – simulating complex
phenomena
– In use for decades
• Data exploration (eScience) – unifies all 3
– Data capture, curation, storage,
analysis, and visualization
– Jim Gray, free PDF from MS Research
7
7
4th Paradigm
8
8
Presentation summary
• Scientific Paradigms
• Roof Savings Calculator
• Visual Analytics
• Knowledge Work
• Autotune
• Publications
9
9
9.00
Figure 2. Residential
energy loads attributed to
envelope and windows
8.00
7.00
Quads
6.00
5.00
4.00
Source: Building Energy Data Book, U.S.
DOE, Prepared by D&R International, Ltd.,
September 2008.
3.00
2.00
1.00
Total
Internal
Gains
Windows
(solar gain)
Windows
(conduction)
Inf iltration
Heating
Foundation
Cooling
Walls
Roof
-
5.00
1.00
Total
Internal
Gains
Windows
(solar gain)
Windows
(conduction)
Heating
Ventilation
Cooling
Inf iltration
Foundation
Source: Building Energy Data Book, U.S. DOE,
Prepared by D&R International, Ltd., September
2008.
2.00
Walls (2)
Figure 3. Commercial energy
loads attributed to envelope
and windows
3.00
Roof
Quads
4.00
Computer tools for simulating cool roofs
Roof Savings Calculator (RSC)
Marc LaFrance
Chris Scruton
CEC
INDUSTRY
COLLABORATIVE
R&D
R. Levinson,
H. Gilbert,
H. Akbari
DOE BT
A. Desjarlais,
W. Miller,
J. New
WBT
Joe Huang,
Ender Erdem
11
11
Roof Savings Calculator
• Replaces:
– EPA Roof Comparison
Calc
– DOE Cool Roof Calculator
• Minimal questions (<20)
– Only location is required
– Building America defaults
– Help links for unknown
information
12
12
RSC = AtticSim + DOE-2.1E
AtticSim - ASTM C 1340 Standard For Estimating Heat Gain or Loss
Through Ceilings Under Attics
13
13
Commercial building types
Office
14
“Big Box” Retail
Warehouse
Torcellini et al. 2008, “DOE Commercial Building Benchmark Models”,
NREL/CP-550-43291, National Renewable Energy Laboratory, Golden CO.
14
AtticSim
DOE-2
15
15
RoofCalc.com impact
100,000+ visitors, 200+ user feedback,
Average: ~81 visitors/day
16
16
Nationwide results
Cost savings for offices - 14 cities,
local utility prices, 22 roof types
Description
BUR No Coating
Mineral Mod Bit
Single Ply
Mineral Mod Bit
Metal
Aluminum Coating
Mineral Mod Bit
Coating over BUR
Metal
Reflect Emis
Houston
ance sivity SRI $ saved …13
10
90
6
42
25
88 25
103
32
90 35
230
33
92 35
197
35
82 35
60
43
45
49
49
58
79
83
83
35
55
55
55
Cooling-dominated
Southern climates
Heating-dominated
Northern climates
279
291
433
208
…14
Mellot, Joseph W., New, Joshua R., and Sanyal, Jibonananda. (2013). "Preliminary Analysis of Energy Consumption for
Cool Roofing Measures." In RCI Interface Technical Journal, volume 31, issue 9, pp. 25-36, October, 2013.
17
17
Summer operation of HVAC duct in
ASHRAE climate zone 3
18
18
Enhanced RSC Site
Input Parameter GUI
Result Output
Results
Exists?
Database
Inputs
User
Savings
Simulate
Hyperion
Simulation
RSC Engine
Savings
19
19
Quote
“We speak piously of … making small
studies that will add another brick to the
temple of science. Most such bricks just
lie around the brickyard.”
–J.R. Platt, Science 1964, 146:347-53
20
20
RSC Service Example (Python)
client = suds.client.Client('URL/TO/WEB/SERVICE/rsc.wsdl')
print(client)
sm = client.factory.create('schema:soapmodel')
load_soap_model_from_xml('../examplemodel.xml', sm)
sr = client.service.simulate(sm)
print(sr)
sm = client.factory.create('schema:soapmodel')
load_soap_model_from_xml('../examplemodel.xml', sm)
print(sm)
contents = client.service.test(sm)
with open('pytest.zip', 'wb') as outfile:
outfile.write(base64.b64decode(contents))
…download example building and batch script from rsc.ornl.gov/web-service.shtml
21
21
Update 1 line of code to change servers
22
22
Millions of simulations visualized for DOE’s Roof Savings
Calculator and deployment of roof and attic technologies
through leading industry partners
DOE: Office of Science
CEC & DOE EERE: BTO
Engine (AtticSim/DOE-2) debugged
using HPC Science assets enabling
visual analytics on 3x(10)6 simulations
Industry & Building Owners
CentiMark, the largest nation-wide
roofing contractor (installs 2500
roofs/mo), is integrating RSC into
their proposal generating system
(20+ companies now interested)
AtticSim
DOE-2
Roof Savings Calculator (RSC) web
site/service developed and validated
[estimates energy and cost savings
from roof and attic technologies]
Leveraging HPC resources to facilitate deployment of building energy efficiency technologies
23
23
Personal story behind one of DOE’s
RSC images
24
24
Presentation summary
• Scientific Paradigms
• Roof Savings Calculator
• Visual Analytics
• Knowledge Work
• Autotune
• Publications
25
25
PCP - car data set
26
26
PCP bin rendering (data)
• Transfer function coloring:
– Occupancy or leading axis
27
27
The power of “and” – linked views (info)
28
28
Knowledge
Large Data
Visualization
Outliers (wisdom)
• Selection of heating outliers
• Find all have box building
type and in Miami
30
30
Impact – RSC and Visual Analytics
12 Publications, 20+ organizations interested (licensing)
New, Joshua R., Huang, Yu (Joe), Levinson, Ronnen, Mellot, Joe, Sanyal, Jibonananda, Miller, William A.,
and Childs, Kenneth W. (2013). "Analysis of DOE's Roof Savings Calculator with Comparison to other
Simulation Engines" ORNL internal report ORNL/TM-2013/501, November 1, 2013, 63 pages.
Mellot, Joseph W., Sanyal, Jibonananda, and New, Joshua R. (2013). "Preliminary Analysis of Energy
Consumption for Cool Roofing Measures." Presented at the International Reflective Roofing
Symposium, the American Coating Association's (ACA) conference, and in Proceedings of the ACA's
Coating Regulations and Analytical Methods Conference, Pittsburgh, PA, May 14-15, 2013.
Jones, Chad, New, Joshua R., Sanyal, Jibonananda, and Ma, Kwan-Liu (2012). "Visual Analytics for Roof
Savings Calculator Ensembles." In Proceedings of the 2nd Energy Informatics Conference, Atlanta,
GA, Oct. 6, 2012.
Cheng, Mengdawn, Miller, William (Bill), New, Joshua R., and Berdahl, Paul (2011). "Understanding the
Long-Term Effects of Environmental Exposure on Roof Reflectance in California." In Journal of
Construction and Building Materials, volume 26, issue 1, pp. 516-26, August 2011.
New, Joshua R., Miller, William (Bill), Desjarlais, A., Huang, Yu Joe, and Erdem, E. (2011). "Development of a
Roof Savings Calculator." In Proceedings of the RCI 26th International Convention and Trade Show,
Reno, NV, April 2011.
Miller, William A., New, Joshua R., Desjarlais, Andre O., Huang, Yu (Joe), Erdem, Ender, and Levinson,
Ronnen (2010). "Task 2.5.4 - Development of an Energy Savings Calculator." California Energy
Commissions (CEC) PIER Project, ORNL internal report ORNL/TM-2010/111, March 2010, 32 pages.
31
31
Presentation summary
• Scientific Paradigms
• Roof Savings Calculator
• Visual Analytics
• Knowledge Work (context)
• Autotune
• Publications
32
32
McKinsey Global Institute Analysis
Source: McKinsey Global Institute analysis
33
33
$1000 machine helping meat machines
34
34
Humans and computers
• 3 lbs (2%), 20 watts (20%)
• PC – 40 lbs, 500 watts
• 120-150 billion neurons
• 4 cores
• 100 trillion synapses
• 3 billion Hz
– Firing time ~milliseconds
• 11 million bits/second input
– Consciousness - 40 bits/second
– Firing time ~nanoseconds
• 100 million bits/second
– Not yet
• Working memory – 4-9 words
• 62,500,000 words
• Long-term memory – 1-1k TB
• Disk – 3TB, perfect recall
• Complex, self-organizing
• “Dumb”, Artificial Intel.
35
35
Learning associations
Full Results
Detailed Results
36
36
Presentation summary
• Scientific Paradigms
• Roof Savings Calculator
• Autotune
37
37
Existing tools for retrofit optimization
Simulation Engine
DOE–$65M (1995–?)
API
38
38
Business limitations for M&V
3,000+ building survey, 23-97% monthly error
ASHRAE G14
Requires
Using Monthly
utility data
CV(RMSE)
15%
NMBE
5%
Using Hourly
utility data
CV(RMSE)
30%
NMBE
10%
39
39
The Autotune Idea
Automatic calibration of software to data
E+ Input
Model
.
.
.
40
40
The search problem
Problem/Opportunity:
~3000 parameters per E+ input file
2 minutes per simulation = 83 hours
41
41
ORNL High Performance Computing Resources
Titan:
299,008 CPU cores
18,688 GPU cores
710TB memory, distributed
Jaguar:
224,256 cores
360TB memory
Nautilus:
1024 cores
4TB shared-memory
Kraken:
112,896 cores
Gordon:
12,608 cores
SSD
42
42
HPC scalability for desktop software
• EnergyPlus desktop app
• Writes files during a run
• Uses RAMdisk
• Balances simulation memory
vs. result storage
• Works from directory of input
files & verifies result
• Bulk writes results to disk
Acknowledgment:
Jibo Sanyal,
ORNL R&D Staff
43
43
Computational complexity
Problems/Opportunities:
Domain experts chose to vary 156
Brute-force = 5x1052 simulations
E+ Input
Model
E+ parameters
main_Tot
1172.5
None_Tot( None_Tot( HP1_in_To HP1_out_ HP1_back HP1_in_fa HP1_compHP2_in_To HP2_out_ HP2_back HP2_in_fa
1)
2)
t
Tot
_Tot
n_Tot
_Tot
t
Tot
_Tot
n_Tot
0
0
6.75
18.75
0
0
0
6.75
18
0
0
LoKU
13.75 billion years
Need 4.1x1028 LoKU
44
44
No database technology sufficient?
Relational, Columnar, NoSQL, different compression and partitioning
MyISAM
InnoDB
• No ACID
• ACID compliant
• No foreign keys
• Foreign keys
• Speed at scale
• Slower adding data
– 0.71 s on LOAD DATA
• Better compression
– 10.27 MB
– 2.3 s on LOAD DATA
• Poorer compression
– 15.4 MB
– Read only compression:
6.003 MB
• 232 rows maximum
Comparisons are based on inserting 200 csv output files, which is 7,008,000 records.
MS Azure DB almost hosted for free with Oakwood Systems, $512,237/month!
45
What is artificial intelligence?
• Give it (lots of) data
• It maps one set of data to
another
• Paradigms
– Unsupervised (clustering)
– Reinforcement (don’t run into wall)
– Supervised (this is the real answer)
• Methods for doing that…
biologically motivated or not
46
46
MLSuite: HPC-enabled suite of
machine learning algorithms
• Linear Regression
• Feedforward Neural Network
• Support Vector Machine
Regression
• Non-Linear Regression
• Self-Organizing Map with Local
Models
• Regression Tree (using
Information Gain)
• Time Modeling with Local Models
• K-Means with Local Models
• Recurrent Neural Networks
• Gaussian Mixture Model with
Local Models
• Ensemble Learning
• Genetic Algorithms
Acknowledgment: UTK computer science
graduate graduate Richard Edwards,
Ph.D. (advisor Dr. Lynne Parker); now
Amazon
47
47
MLSuite example
• EnergyPlus – 2-10 mins for an annual simulation
!- ALL OBJECTS IN CLASS
Version,
7.0;
!- Version
!- SIMULATIONCONTROL ===
SimulationControl,
No, !-Do Zone Sizing Calc
No, !-Do System Sizing Calc
…
• ~E+ - 4 seconds AI agent as surrogate model,
90x speedup, small error, brittle
48
48
Quote
“the world is the best model
of itself.”
–Rodney Brooks, 1990, Elephants and nouvelle AI
49
49
Source of Input Data
• 3 Campbell Creek homes
(TVA, ORNL, EPRI)
• ~144 sensors/home, 15-minute data:
– Temperature (inside/outside)
– Plugs
– Lights
– Dryer
– Heat pump air flow
– Range
– Refrigerator
– Shower water flow
– Washer
– Dishwasher
– Etc.
– Radiated heat
50
50
MLSuite example
Data Preparation:
30x LS-SVM variants
(train/test and input order)
PBS
XML
Titan
MLSuite
MLS
51
MLS
Applications of machine learning
• Linear Regression predicting whole building energy use
House 1
(House 2 is
similar)
House 3
House 3
Next hour
• Accuracy Metrics for best subset of sensors
52
52
MLSuite: HPC-enabled Suite of
Machine Learning algorithms
• Linear regression
• Feedforward neural network
• Support vector machine
regression
• Non-linear regression
• K-means with local models
• Gaussian mixture model with local
models
• Self-organizing map with local
models
• Regression tree (using
information gain)
• Time modeling with local models
• Recurrent neural networks
• Genetic algorithms
• Ensemble
learning
53
53
Evolutionary computation
How are offspring produced?
Thickness
Conductivity
Density
Specific Heat
Bldg1
0.022
0.031
29.2
1647.3
Bldg2
0.027
0.025
34.3
1402.5
(1+2)1
0.0229
0.029
34.13
1494.7
(1+2)2
0.0262
0.024
26.72
1502.9
• Average each component
• Add Gaussian noise
• … “AI inside of AI”
54
54
Getting more for less
• EnergyPlus is slow
– Full-year schedule
– 2 minutes per simulation
• Use abbreviated 4-day schedule instead
– Jan 1, Apr 1, Aug 1, Nov 1
– 10 – 20 seconds per simulation
r = 0.94
Monthly Electrical Usage
r = 0.96
Hourly Electrical Usage
55
55
Evolutionary combination
Island Hopping
1.
2.
3.
4.
4 of 19 experiments
Surrogate Modeling
Sensor-based Energy
Modeling (sBEM)
Abbreviated Schedule
Island-model evolution
56
56
Automated M&V process Autotune
calibration of simulation to measurements
XSEDE and DOE Office of Science
DOE-EERE BTO
Industry and building owners
Commercial Buildings
Monthly
utility data
Hourly
utility data
Features:
Works with “any” software
Tunes 100s of variables
Customizable distributions
Matches 1+ million points
CVR
NMBE
CVR
NMBE
ASHRAE
G14
Requires
Autotune
Results
15%
5%
30%
10%
0.32%
0.06%
0.48%
0.07%
Residential Tuned input
home
avg. error
Within
Hourly – 8%
30¢/day
Monthly – 15%
(actual use
$4.97/day)
10+ companies interested
Leveraging HPC resources to calibrate models for optimized building efficiency decisions
57
57
HPC-informed algorithmic reduction…
to commodity hardware
1 hour
LoKU
13.75 billion years
Need 4.1x1028 LoKU
58
Science of how Autotune works
Communications (33+ related publications):
1) Autotune Overview (2)
2) Large Ensemble Simulations and Big Data Challenges (2)
3) Machine Learning Techniques – on sensor data (2),
MLSuite scalability on HPC (5)
4) Machine Learning applied to Calibration – Surrogate modeling (1), inverse
modeling (1), PhD dissertation (1)
5) Fixing time-series (sensor) data (6), weather data (1)
6) Provenance for tracking sensor data usage (3)
7) Autotune System Results – residential ZEBRAlliance (3), comparison to
manual M&V (2), commercial buildings (2), high-resolution calibration (2), and
“trinity testing” (2)
59
59
Website
60
60
Website
jsonString = tune(userdata, basemodel, schedule,
parameters, weather, email)
Returns JSON containing tracking number and
queue position
Service
Get Status
Database
Get Next Job
Response
Insert Tuned Models
Tuned Models
Insert Job
Request
Scripted System
Parameters
Parameters
User
Tuned Models
Frontend
Backend
10 companies interested, 3 have provided data
61
61
FEMP/BTO resource collaboration
Rapid Model
Assembly Tool
Input Data File
(IDF)
Generator
Public
Users
ORNL Rapid Model
Assembly Tool compiles
IDF file with limited
inputs
FEMP IGA Tool
presently utilizes
ORNL IDF Assembly
Tool
ECM’s/
savings
Investment Grade Audit
Tool
(ESPC/UESC)
Uncertainty
Module
Representativ
eE+ IDF
Open Studio
Autotune
Calibrated
E+ IDF
Actual
Meteorological
Data
EnergyPlus (E+)
Utility/
Measured Data
SensorDVC
62
62
Building 3147 Autotune results
Autotune
ENABLE IGA Tool
Uses defaults based
on building type,
location, and age.
Climate Zone 4a
weather file
(Baltimore)
CV(RMSE) = 119.9%
Calibrated model
Local AMY weather file
ENABLE IGA Tool
Quick audit (10 min
walkthrough, 3 inputs
altered)
Climate Zone 4a
weather file
(Baltimore)
CV(RMSE) = 33.4%
Local Weather (TMY3)
Quick audit
Knoxville airport
weather file (TMY3)
Local Weather (AMY)
Quick audit
Oak Ridge weather file
(AMY) – Warm winter
CV(RMSE) = 23.3%
CV(RMSE) = 9.0%
4096 simulations
12 hours on 4 cores
(48 core-hours)
Typically:
1024 simulations
~12 core-hours
ASHRAE Guide14 < 15%
CV(RMSE) = 3.5%
Defaulted (climate zone TMY3)
Quick audit (climate zone TMY3)
Quick audit (local TMY3)
Quick audit (local AMY)
Tuned (local AMY)
15 Minute
155.55%
72.48%
63.89%
43.30%
38.12%
Hourly
154.14%
71.33%
62.60%
41.64%
35.99%
Daily
129.11%
52.25%
41.08%
22.18%
17.75%
Monthly
119.87%
33.43%
23.33%
9.03%
3.50%
63
63
Presentation summary
• Scientific Paradigms
• Roof Savings Calculator
• Visual Analytics
• Knowledge Work
• Autotune
• Publications
64
64
Publication venues
• Tier 1 (10)
–
–
–
–
–
–
–
–
–
–
ASHRAE Transactions
ASHRAE Journal
International Journal of Refrigeration
Proceedings of the International Compressor, Refrigeration
and Air Conditioning Conferences at Purdue
ASHRAE HVAC&R Research Journal
Energy and Buildings
Proceedings of ACEEE Summer Study on Energy Efficiency
in Buildings (bi-annual)
Proceeding of the International Energy Agency Heat Pump
Conference (tri-annual)
Proceedings of National SimBuild Conference (International
Building Performance Simulation Association: IBPSA-USA)
(annual)
Journal of Heat Transfer
• Tier 2 (30)
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Applied Thermal Engineering
Applied Energy
Building and Environment
Energy - The International Journal
Energy Conversion and Management
Experimental Heat Transfer
Heat Transfer Engineering
Home Energy Magazine
International Journal of Thermal Sciences
International Journal of Heat & Mass Transfer
International Journal of Energy Research
International Journal of Air-Conditioning and Refrigeration
International Journal of Sustainability and Built Environment
Journal of Power & Energy
…
Journal/Conference
5-Year Impact
Factors
Nature
36.280 (2011)
Science
31.201 (2011)
Acceptance rate – 7%
80% rejected in 10 days
Proceedings of National
Academy of Sciences (PNAS)
10.6
Renewable and Sustainable
Energy Reviews
6.577
Applied Energy
4.783
Energy
4.107
Energy and Buildings
2.809
Buildings and Environment
2.281
HVAC&R Research
1.034
ASHRAE Transactions
0.420
ASHRAE Journal
0.277
65
65
Publication Impact
(publication, venue)
• Edwards, Richard E., New, Joshua R., and Parker, Lynne E. (2012). "Predicting Future Hourly
Residential Electrical Consumption: A Machine Learning Case Study." In Journal of Energy and
Buildings, volume 49, issue 0, pp. 591-603, June 2012. (27, 2.809)
• Bhandari, Mahabir S., Shrestha, Som S., and New, Joshua R. (2012). "Evaluation of Weather Data
for Building Energy Simulations." In Journal of Energy and Buildings, volume 49, issue 0, pp. 109118, June 2012. (20, 2.809)
• Boudreaux, Philip R., Gehl, Anthony C., Dockery, R., New, Joshua R., and Christian, Jeffrey E.
(2010). "Tennessee Valley Authority's Campbell Creek Energy Efficient Homes Project: 2010 First
Year Performance Report July 1, 2009 - August 31, 2010." ORNL internal report ORNL/TM2010/206, November 2010, 165 pages. (10)
• Sanyal, Jibonananda, Al-Wadei, Yusof H., Bhandari, Mahabir S., Shrestha, Som S., Karpay, Buzz,
Garret, Aaron L., Edwards, Richard E., Parker, Lynne E., and New, Joshua R. (2012). "Autotune:
Building Energy Model Calibration using EnergyPlus, Machine Learning, and Supercomputing."
In Proceedings of the 5th National SimBuild of IBPSA-USA, International Building Performance
Simulation Association (IBPSA), August 1-3, 2012. (9)
• New, Joshua R. (2013). "Autotune Building Energy Models." DOE Building Technology Office (BTO)
Review, Washington DC, April 2, 2013. (9)
…45 publications since Mar. 2010, 115 citations since 2009, h-index 7, i10-index 5
13 acknowledge OLCF resources
66
66
Discussion
Oak Ridge National Laboratory
EESD – Martin Keller
ETSD – Johney Green
BTRIC – Patrick Hughes &
Ed Vineyard
WBCI – Melissa Lapsa
Joshua New, Ph.D.
newjr@ornl.gov
67
“Big Ideas”
Model US – SimCity for all US Buildings
Data integration and computer vision for low-fidelity,
wide-area software models of all existing buildings
with eventual calibration and simulation
Technology/Approach Summary
• Consolidate building imagery for box-plot models
• Classify imagery via machine learning for layering boxplot models with material and equipment properties
• Create building database for year constructed, square
footage, no. of beds and baths, heating type in public
database, etc. for use by industry partners
• Create models with methods similar to LDRD
CoNNECT used on 220,000 Knox-county buildings
• HPC for annual energy simulations of all US buildings
(requires avg. INCITE allocation of 127M core hours)
• Data sources have identified databases covering 100+
million buildings, surpassing the largest DOE’s Building
Technology Office database of ~30,000.
Technology/Approach Impact
• Light commercial support $5-9 billion industry
• ESCO revenue in 2008 was $4.1 billion, with the market
served being 9% residential (public utility programs and
public housing)
• At 96% of the building stock, ESCO uptake could be a
$12 billion dollar industry with 75% of revenue from EE
Data Integration, Mining, and Modeling
68
Model US – 125.1 million buildings
BEM – modeling and analysis tools
Differentiating
Capabilities
Supercomputing,
simulation engines, webbased tools, Autotune
calibration, uncertainty
analysis, domain driven
modeling development
G14 Autotun
Mont
hly
data
Hourl
y
data
CV(RMS
E)
NMBE
CV(RMS
E)
NMBE
15
%
5%
30
%
10
%
e
0.318%
0.059%
0.483%
0.067%
40,000
38,000
36,000
34,000
32,000
30,000
28,000
26,000
24,000
22,000
20,000
18,000
16,000
14,000
12,000
Frequencies
PostPreRetrofit Retrofi
t
Collaboration
Impacts
EnergyPlus and
OpenStudio development
teams,
Several universities for
simulation and AI,
Several companies for
tools
• Web services for
integration
• Autotune – trial FEMP,
OWIP, 10+
organizations
• Roof Savings Calculator
– 100,000 visits, 20+
organizations interested
– CentiMark integrating
RSC into their proposal
generating system RSC
and –
will provide market
data
71 visits/day
to DOE
AtticSim
Annual Energy Consumption (kWh)
• OpenStudio
Refrigeration
– 1st E+ refrigeration GUI
69
• Uncertainty analysis
Measurement, data, and analysis
Differentiating
Capabilities
Whole Building, Emulated
Occupancy Research
Buildings,
Big Data, Machinelearning, Data
Provenance, Data
Validation and Correction
Collaboration
Impacts
Several Companies and
Universities for Data and
Tools
• Building science is
measurement and data
driven
• High resolution sensor
data
Genetic
Algorithm
Optimization
SensorDVC Quality
Ctrl
MLSuite
Artificial
Intelligenc
e
on HPC
– FRPs: 1000+ sensors, 30
sec
– ZEBRAlliance:1300+
sensors, 15 min
– Campbell Ck: 500 sensors,
15 min
– Yarnell: ~300 sensors, 30
sec
• Better data
management
– GB to TB datasets
– Productivity tools
– Analysis capabilities
• Building envelope and
HVAC system
performance evaluation
in real buildings
• DOE Simulation
70,000
60,000
Cooling (Btu/hr)
50,000
40,000
30,000
20,000
10,000
0
0
10
20
30
40
50
60
70
Hourly OA Temperature (F)
Sensible Cooling
70
Total (Sensible + Latent) Cooling
Cooling Elec.
80
90
100
The Autotune Team
Jibo
Mahabir
Sanyal Bhandari
Som
Shrestha
Joshua New
Aaron
Garrett
Buzz
Karpay
Richard
Edwards
http://autotune.roofcalc.com
Science of how Autotune works
Communications (33 related publications):
1) Autotune Simulation Overview [4][5][25][26]
2) Large Ensemble Simulations and Big Data Challenges [1][9]
3) Machine Learning Techniques – ML on sensor data [7][19], MLSuite algorithm
scalability on supercomputers [10][16][17][29], GPU-accelerated statistics[27]
4) Machine Learning applied to Calibration - EnergyPlus approximation in 3 seconds
[10], inverse EnergyPlus approximation of tuned inputs [11], PhD dissertation on
machine learning for calibration [2]
5) Fixing time-series (sensor) data – statistical methods[3], filtering methods[13][31],
machine learning methods[14][28][30]
6) Provenance for tracking sensor data usage – provDMS sensor data download,
experiments, visualization of historical usage, and software interoperability[32][33][34]
7) Weather data – Actual Meteorological Year variability vendor survey [6]
8) Autotune System Results – evolutionary approach initial Autotune testing[18] applied
to residential field demonstration ZEBRAlliance building WC1[21][22] compared to
manual effort for monthly data[8], for 15-minute data and multiple channels[12], and
“trinity testing” of 3 commercial buildings at monthly whole-building energy and
hourly data for 20 channels involving 3-79 tunable variables with error metrics on
output and input[20]
72
Discontinued
TBDiscontinued – May
31st
HESPro.lbl.gov
www.energyright.com
You pay $150 for energy audit
Auditor provides list of
recommendations
You pay up to $1000 for subset of
recommendations
TVA give you $650 (reimburses $150
fee and pays 50% of cost up to $500)
ID Actions Pathway
Research
Data Integration
Physical
Procurement
E+
Data
Audi
t Kit
77
Energy
Decisions
ID Actions Pathway, continued
Multimodal image processing for characterization of buildings
Features extracted from aerial and ground imaging databases can be used to characterize buildings’ shape and
exterior materials, and identify high-energy loss areas, without a priori knowledge.
Ground imagery
Combined information
Images captured by imagers mounted on road vehicles:
- Google Street View and Bing are well known examples
- GPS information for each image allowing images cross
referencing and geo-localization;
- High definition imaging: fine details i.e. color data, textures,
regions of interest, and structure corners can be extracted and
used for pattern recognition and classification;
- Will complement 3D reconstruction of buildings provided by
complementary technologies.
Aerial imagery
Images provides by satellites, drones, UAVs:
- Geo-referenced building localization and orientation;
- High definition imaging (up to .4m per pixel) allowing to identify
numerous parts of buildings: skylight, solar panels, etc;
- Multispectral signatures to identify roof type, composition and
parts;
- Combining images of the building under different orientations
allows to reconstruct a full 3D model of the roof and a partial 3D
model of the entire building.
Main image processing development will focus on segmentation and classification
Image data:
Gather standard color
images, multispectral,
hyperspectral and/or
thermal images from
available
databases
ORNL
is managed
by UT-Battelle
for the US Department of Energy
Segmentation:
Identify main structures
of buildings : windows,
walls, doors, vents, etc.
Will use color, texture,
climatic specificities, etc.
Feature extraction:
Refine specific
characteristics for each
identified structures:
spectral signatures,
textures, etc.
Classification:
Machine learning techniques
will be used to characterize
building structures based on
high dimensional feature
vectors
Scientific Impact – RSC and Visual Analytics
11 Publications:
Mellot, Joe, New, Joshua R., and Sanyal, Jibonananda. (2014). "Cool Roofing:
Analysis of Energy Consumption for Cool Roofing." In Western Roofing Insulation and Siding, pp. 50-56, issue January/February, 2014.
New, Joshua R. (2014). Invited Speaker. "DOE's Roof Savings Calculator." Metal
Construction Association's (MCA) MetalCon Roofing Council, Clearwater
Beach, FL, January 27, 2014.
New, Joshua R., Huang, Yu (Joe), Levinson, Ronnen, Mellot, Joe, Sanyal,
Jibonananda, Miller, William A., and Childs, Kenneth W. (2013). "Analysis of
DOE's Roof Savings Calculator with Comparison to other Simulation Engines"
ORNL internal report ORNL/TM-2013/501, November 1, 2013, 63 pages.
Mellot, Joseph W., Sanyal, Jibonananda, and New, Joshua R. (2013). "Preliminary
Analysis of Energy Consumption for Cool Roofing Measures." Presented at
the International Reflective Roofing Symposium, the American Coating
Association's (ACA) conference, and in Proceedings of the ACA's Coating
Regulations and Analytical Methods Conference, Pittsburgh, PA, May 14-15,
2013.
Mellot, Joseph W., New, Joshua R., and Sanyal, Jibonananda. (2013). "Preliminary
Analysis of Energy Consumption for Cool Roofing Measures." In RCI Interface
Technical Journal, volume 31, issue 9, pp. 25-36, October, 2013.
Scientific Impact – RSC and Visual Analytics
Jones, Chad, New, Joshua R., Sanyal, Jibonananda, and Ma, Kwan-Liu (2012).
"Visual Analytics for Roof Savings Calculator Ensembles." In Proceedings of
the 2nd Energy Informatics Conference, Atlanta, GA, Oct. 6, 2012.
New, Joshua R., Jones, Chad, Miller, William A., Desjarlais, Andre, Huang, Yu Joe,
and Erdem, Ender (2011). "Poster: Roof Savings Calculator." In Proceedings
of the International Conference on Advances in Cool Roof Research,
Berkeley, CA, July 2011.
Cheng, Mengdawn, Miller, William (Bill), New, Joshua R., and Berdahl, Paul (2011).
"Understanding the Long-Term Effects of Environmental Exposure on Roof
Reflectance in California." In Journal of Construction and Building Materials,
volume 26, issue 1, pp. 516-26, August 2011.
New, Joshua R., Miller, William (Bill), Desjarlais, A., Huang, Yu Joe, and Erdem, E.
(2011). "Development of a Roof Savings Calculator." In Proceedings of the
RCI 26th International Convention and Trade Show, Reno, NV, April 2011.
Miller, William A., New, Joshua R., Desjarlais, Andre O., Huang, Yu (Joe), Erdem,
Ender, and Levinson, Ronnen (2010). "Task 2.5.4 - Development of an Energy
Savings Calculator." California Energy Commissions (CEC) PIER Project,
ORNL internal report ORNL/TM-2010/111, March 2010, 32 pages.
Miller, William A., Cheng, Mengdawn, New, Joshua R., Levinson, Ronnen, Akbari,
Hashem, and Berdahl, Paul (2010). "Task 2.5.5 - Natural Exposure Testing in
California." California Energy Commissions (CEC) PIER Project, ORNL
internal report ORNL/TM-2010/112, March 2010, 56 pages.
Scientific Impact – HPC
4 Publications:
• Sanyal, Jibonananda and New, Joshua R. (2013). "Oak Ridge Institutional Cluster
Autotune Test Drive Report." ORNL internal report, February 17, 2014, 6 pages.
• Ranjan, Niloo, Sanyal, Jibonananda, and New, Joshua R. (2013). "In-Situ Statistical
Analysis of Autotune Simulation Data using Graphical Processing Units." Presented as
part of the Science Undergraduate Laboratory Internships (SULI) program, August 8,
2013.
• Sanyal, Jibonananda, New, Joshua R., and Edwards, Richard E. (2013).
"Supercomputer Assisted Generation of Machine Learning Agents for the Calibration of
Building Energy Models." In Proceedings of the Extreme Science and Engineering
Discovery Environment (XSEDE13) Conference and selected to be featured
in Lightning Talks, San Diego, CA, July 22-25, 2013.
• Sanyal, Jibonananda and New, Joshua R. (2013). "Simulation and Big Data
Challenges in Tuning Building Energy Models." In Proceedings of the IEEE Workshop
on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES), Berkeley,
CA, May 20, 2013. Republished in IEEE Xplore October, 2013.
• 13 acknowledge OLCF resources
81
Scientific Impact – MLSuite
10 Publications
• Sanyal, Jibonananda, New, Joshua R., Edwards, Richard E., and Parker, Lynne E.
(2014). "Calibrating Building Energy Models Using Supercomputer Trained Machine
Learning Agents." In Proceedings of the Journal on Concurrency and Computation:
Practice and Experience, March, 2014.
• Edwards, Richard E., New, Joshua R., and Parker, Lynne E. (2012). “Predicting Future
Hourly Residential Electrical Consumption: A Machine Learning Case Study.” In
Journal of Energy and Buildings, pp.591-603, volume 49, issue 0, June 2012.
• Edwards, Richard E., New, Joshua R., and Parker, Lynne E. (2011). "Sensor-based
Building Energy Modeling." ORNL internal report ORNL/TM-2011/328, September
2011, 79 pages.
• Edwards, Richard E., New, Joshua R., and Parker, Lynne E. (2013). "Approximate lfold Cross-Validation with Least Squares SVM and Kernel Ridge Regression."
In Proceedings of the IEEE 12th International Conference on Machine Learning and
Applications (ICMLA13), Miami, FL, December 4-7, 2013.
82
23
citations
June 2012
Scientific Impact – MLSuite
• Edwards, Richard E., New, Joshua R., and Parker, Lynne E. (2013). "Estimating
Building Simulation Parameters via Bayesian Structure Learning." In Proceedings of
the 2nd International Workshop on Big Data, Streams and Heterogeneous Source
Mining: Algorithms, Systems, Programming Models and Applications (BigMine13), part
of the 19th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
(KDD2013), Chicago, IL, August 11, 2013.
• Edwards, Richard E. (2013). "Automating Large-Scale Simulation Calibration to RealWorld Sensor Data." Doctoral Committee: Lynne E. Parker (advisor), Joshua R. New,
Michael Berry, Hamparsum Bozdogan, and Husheng Li. A Dissertation presented for
the Doctor of Philosophy Degree in Archives of The University of Tennessee,
Knoxville, TN, March 13, 2013.
• Edwards, Richard E., Parker, Lynne E., and New, Joshua R. (2013). "MLSuite
Software Transfer Document." ORNL internal report ORNL/TM-2013/131, May 8,
2013, 36 pages.
• Edwards, Richard E., New, Joshua R., and Parker, Lynne E. (2012). "MLSuite FY2012 Final Report." ORNL internal report ORNL/TM-2013/385, October 2012, 68
pages.
• Edwards, Richard E., Zhang, Hao, Parker, Lynne E. and New, Joshua R. (2012).
"Approximate l-fold Cross-Validation with Least Squares SVM and Kernel Ridge
Regression." ORNL internal report ORNL/TM-2012/419, August 2012, 9 pages.
• Edwards, Richard E., New, Joshua R., and Parker, Lynne E. (2011). "Sensor-based
Building Energy Modeling." ORNL internal report ORNL/TM-2011/328, September
83
Scientific Impact – Sensor Q/A
7 Publications
• Castello, Charles C., Sanyal, Jibonananda, Rossiter, Jeffrey S., Hensley, Zachary P.,
and New, Joshua R. (2014). "Sensor Data Management, Validation, Correction, and
Provenance for Building Technologies." Technical paper TRNS-00223-2013.R1. To
appear in Proceedings of the ASHRAE Annual Conference and ASHRAE Transactions
2014, Seattle, WA, June 28-July 2, 2014.
• Hensley, Zachary P., Sanyal, Jibonananda, and New, Joshua R. (2014). "Provenance
in Sensor Data Management: A Cohesive, Independent Solution for Bringing
Provenance to Scientific Research." In Communications of the ACM, volume 57, issue
2, pp. 55-62, December 2013 and ACM Queue, volume 11, issue 12, pp. 1-14,
December 2013.
• Hensley, Zachary, Sanyal, Jibonananda, and New, Joshua R. (2013). "ProvDMS -- A
Provenance Data Management System for Sensor Data." Presented as part of the
Science Undergraduate Laboratory Internships (SULI) program, August 8, 2013.
84
Scientific Impact – Sensor Q/A
• Smith, Matt K., Castello, Charles C., and New, Joshua R. (2013). "Machine Learning
Techniques Applied to Sensor Data Correction in Building Technologies."
In Proceedings of the IEEE 12th International Conference on Machine Learning and
Applications (ICMLA13), Miami, FL, December 4-7, 2013.
• Castello, Charles C., New, Joshua R., and Smith, Matt K. (2013). "Autonomous
Correction of Sensor Data Applied to Building Technologies Using Filtering Methods."
In Proceedings of the IEEE Global Conference on Signal and Information Processing
(GlobalSIP), Austin, TX, December 3-5, 2013.
• Smith, Matt K., Castello, Charles C., and New, Joshua R. (2013). "Sensor Validation
with Machine Learning." Presented as part of the Science Undergraduate Laboratory
Internships (SULI) program, July 29, 2013.
• Castello, Charles C. and New, Joshua R. (2012). "Autonomous Correction of Sensor
Data Applied to Building Technologies Utilizing Statistical Processing Methods."
In Proceedings of the 2nd Energy Informatics Conference, Atlanta, GA, Oct. 6, 2012.
85
Scientific Impact – Autotune
9 Publications
• Sanyal, Jibonananda, New, Joshua R., and Edwards, Richard E. (2013). "Calibration
of Building Energy Models: Supercomputing, Big-Data and Machine-Learning."
In Proceedings of the ORNL Postdoc Symposium. Oak Ridge, TN, July 19, 2013.
• Garrett, Aaron, New, Joshua R., and Chandler, Theodore (2013). "Evolutionary Tuning
of Building Models to Monthly Electrical Consumption." Technical paper DE-13-008.
In Proceedings of the ASHRAE Annual Conference andASHRAE Transactions 2013,
volume 119, part 2, pp. 89-100, Denver, CO, June 22-26, 2013.
• New, Joshua R. (2013). "Autotune Building Energy Models." DOE Building Technology
Office (BTO) Peer Review, Washington DC, April 2, 2013.
• Al-Wadei, Yusof H., New, Joshua R., and Sanyal, Jibonananda (2012). "Interactive
Web Design through Survey and Adoption of Modern Web-Technologies." Presented
as part of the Science Undergraduate Laboratory Internships (SULI) program, August
2012.
• New, Joshua R., Sanyal, Jibonananda, Bhandari, Mahabir S., Shrestha, Som S.
(2012). "Autotune E+ Building Energy Models." In Proceedings of the 5th National
SimBuild of IBPSA-USA, International Building Performance Simulation Association
(IBPSA), Aug. 1-3, 2012.
86
Scientific Impact – Autotune
• Sanyal, Jibonananda, Al-Wadei, Yusof H., Bhandari, Mahabir S., Shrestha, Som S.,
Karpay, Buzz, Garret, Aaron L., Edwards, Richard E., Parker, Lynne E., and New,
Joshua R. (2012). "Autotune: Building Energy Model Calibration using EnergyPlus,
Machine Learning, and Supercomputing." In Proceedings of the 5th National SimBuild
of IBPSA-USA, International Building Performance Simulation Association (IBPSA),
Aug. 1-3, 2012.
• Boudreaux, Philip R., Gehl, Anthony C., New, Joshua R., and Christian, Jeffrey E.
(2012). "Campbell Creek Energy Efficient Homes Project: Summer 2011 Performance
Report." ORNL internal report ORNL/TM-2012/51, July 2012, 49 pages.
• Bhandari, Mahabir S., Shrestha, Som S., and New, Joshua R. (2012). "Survey and
Analysis of Weather Data for Building Energy Simulations." In Journal of Energy and
Buildings, volume 49, issue 0, pp. 109-118, June 2012.
• Garrett, Aaron and New, Joshua R. (2012). "An Evolutionary Approach to Parameter
Tuning of Building Models." ORNL internal report ORNL/TM-2012/418, April 2012, 68
pages.
87
9 citations
August
15
citations
June 2012
Provenance – sensor lineage
88
Sensor Correction
89
Trinity Test
• Automated testing system – match measured
data, but how close to the real building?
90
Outputs used for Autotune fitness
• Monthly, 1 channel, EnergyPlus output variable (12 data
points/yr)
• Hourly, 20 channels, EnergyPlus output variables (175,200
pts/yr)
and 15-minute EnergyPlus output variables (700,800 pts/yr)
91
Input parameters Autotuned
92
Input parameters Autotuned
93
Input parameters Autotuned
94
Trinity Results
Monthly Energy Data
95
Hourly Energy Data
CV(RMSE)
0.318%
CV(RMSE)
0.483%
NMBE
0.059%
NMBE
0.067%
Input Error
15%
Input Error
8%