Autotune
For:
Consortium for Building
Energy Innovation’s
2nd Annual Building Energy
Informatics Summit
Dec. 19, 2014
Joshua New, Ph.D.
865-241-8783
newjr@ornl.gov
ORNL is managed by UT-Battelle
for the US Department of Energy
The Autotune Idea
Automatic calibration of software to data
E+ Input
Model
.
.
.
2
2
The search problem
Problem/Opportunity:
~3000 parameters per E+ input file
2 minutes per simulation = 83 hours
3
3
ORNL High Performance Computing Resources
Titan:
299,008 CPU cores
18,688 GPU cores
710TB memory, distributed
Jaguar:
224,256 cores
360TB memory
Nautilus:
1024 cores
4TB shared-memory
Kraken:
112,896 cores
Gordon:
12,608 cores
SSD
4
4
HPC scalability for desktop software
• EnergyPlus desktop app
• Writes files during a run
• Uses RAMdisk
• Balances simulation memory
vs. result storage
• Works from directory of input
files & verifies result
• Bulk writes results to disk
Acknowledgment: Jibo Sanyal, ORNL
R&D Staff
5
5
Computational complexity
Problems/Opportunities:
Domain experts chose to vary 156
Brute-force = 5x1052 simulations
E+ Input
Model
E+ parameters
main_Tot
1172.5
None_Tot( None_Tot( HP1_in_To HP1_out_ HP1_back HP1_in_fa HP1_compHP2_in_To HP2_out_ HP2_back HP2_in_fa
1)
2)
t
Tot
_Tot
n_Tot
_Tot
t
Tot
_Tot
n_Tot
0
0
6.75
18.75
0
0
0
6.75
18
0
0
LoKU
13.75 billion years
Need 4.1x1028 LoKU
6
6
MLSuite: HPC-enabled suite of
machine learning algorithms
• Linear Regression
• Feedforward Neural Network
• Support Vector Machine
Regression
• Non-Linear Regression
• Self-Organizing Map with Local
Models
• Regression Tree (using
Information Gain)
• Time Modeling with Local Models
• K-Means with Local Models
• Recurrent Neural Networks
• Gaussian Mixture Model with
Local Models
• Ensemble Learning
• Genetic Algorithms
Acknowledgment: UTK computer science
graduate graduate Richard Edwards,
Ph.D. (advisor Dr. Lynne Parker); now
Amazon
7
7
MLSuite example
Data Preparation:
30x LS-SVM variants
(train/test and input order)
PBS
XML
Titan
MLSuite
MLS
8
MLS
Applications of machine learning
• EnergyPlus – 2-10 mins for an annual simulation
!- ALL OBJECTS IN CLASS
Version,
7.0;
!- Version
!- SIMULATIONCONTROL ===
SimulationControl,
No, !-Do Zone Sizing Calc
No, !-Do System Sizing Calc
…
• ~E+ - 4 seconds AI agent as surrogate model,
90x speedup, small error, brittle
9
9
Evolutionary computation
How are offspring produced?
Thickness
Conductivity
Density
Specific Heat
Bldg1
0.022
0.031
29.2
1647.3
Bldg2
0.027
0.025
34.3
1402.5
(1+2)1
0.0229
0.029
34.13
1494.7
(1+2)2
0.0262
0.024
26.72
1502.9
• Average each component
• Add Gaussian noise
• … “AI inside of AI”
10
10
Getting more for less
• EnergyPlus is slow
– Full-year schedule
– 2 minutes per simulation
• Use abbreviated 4-day schedule instead
– Jan 1, Apr 1, Aug 1, Nov 1
– 10 – 20 seconds per simulation
r = 0.94
Monthly Electrical Usage
r = 0.96
Hourly Electrical Usage
11
11
Evolutionary combination
Island Hopping
1.
2.
3.
4.
4 of 19 experiments
Surrogate Modeling
Sensor-based Energy
Modeling (sBEM)
Abbreviated Schedule
Island-model evolution
12
12
Automated M&V process Autotune
calibration of simulation to measurements
XSEDE and DOE Office of Science
DOE-EERE BTO
Industry and building owners
Commercial Buildings
Monthly
utility data
Hourly
utility data
Features:
Works with “any” software
Tunes 100s of variables
Customizable distributions
Matches 1+ million points
CVR
NMBE
CVR
NMBE
ASHRAE
G14
Requires
Autotune
Results
15%
5%
30%
10%
0.32%
0.06%
0.48%
0.07%
Residential Tuned input
home
avg. error
Within
Hourly – 8%
30¢/day
Monthly – 15%
(actual use
$4.97/day)
10+ companies interested
Leveraging HPC resources to calibrate models for optimized building efficiency decisions
13
13
HPC-informed algorithmic reduction…
to commodity hardware
1 hour
LoKU
13.75 billion years
Need 4.1x1028 LoKU
14
Autotune
Implementation
Joshua New
Oak Ridge National Laboratory
Website
Service
Parameters
Tuned Models
Frontend
Database
Get Next Job
Response
User
Insert Job
Get Status
Insert Tuned Models
Parameters
Tuned Models
Request
Scripted System
Backend
Website
Website
Website
Website
Website
Website
Website
Website
Website
Parameter Markup
• IDD <-> XSD, IDF<->XML conversion for EnergyPlus 7.0
http://evenstar.ornl.gov/trinity/
Class
Sizing:Parameters
Lights
ZoneInfiltration:DesignFlowRate
Object
Sizing:Parameters
Dining_Lights
Kitchen_Infiltration
Field
Heating Sizing Factor
Watts per Zone Floor Area
Flow per Exterior Surface Area
Schedule:Compact
Schedule:Compact
CLGSETP_KITCHEN_SCH
HTGSETP_KITCHEN_SCH
Field 4
Field 7
Default
Minimum Maximum Distribution
1.2
0.84
1.56 uniform
22.6
15.82
29.38 uniform
0.000302 0.000211 0.000393 uniform
30
19
21
13.3
39 uniform
24.7 uniform
Type Group
float
float
float
Constraint
float CA1
float HA2
HA2 - CA1 < - 1
Web Service
• jsonString = tune(userdata, basemodel, schedule,
parameters, weather, email)
• Returns JSON containing tracking number and queue position
• jsonString = retune(trackingNumber)
• Returns JSON containing tracking number and queue position
• jsonString = terminate(trackingNumber)
• Returns JSON containing tracking number, queue position, total
runtime, and an array of model IDs and fitness values
Web Service
• jsonString = getOutput(trackingNumber)
• Returns JSON containing tracking number, queue position, total
runtime, and an array of model IDs and fitness values
• jsonString = getFullOutput(trackingNumber)
• Returns JSON containing tracking number, queue position, total
runtime, and an array of model IDs and fitness values, along with
all the inputs (base model, weather, etc.)
• jsonString = getModel(trackingNumber, modelNumber)
• Returns JSON containing tracking number, model ID, model file,
and model fitness
Discussion
Oak Ridge National Laboratory
EESD – Martin Keller
ETSD – Johney Green
BTRIC – Patrick Hughes &
Ed Vineyard
WBCI – Melissa Lapsa
Joshua New, Ph.D.
newjr@ornl.gov