Design of Experiment and
Assessing Interactions within
Atmospheric Processes
Dev Niyogi
North Carolina State University
Email: dev_niyogi@ncsu.edu
Human thinking is
logical , sequential, and linear
Real world is
Convulated, Non-linear, and
Interactive
Some References
• Mesoscale Meteorological Modeling, Roger Pielke Sr.,
Second Edition. (blue.atmos.colostate.edu)
• Box, Hunter and Hunter, Design of Experiment, 1987
• Stein and Alpert, Factor Separation Analysis, J.
Atmos. Sci. 1993
• Alpert et al., How good are sensitivity studies?, J.
Atmos. Sci. 1995
• Henderson- Sellers, A fractional – factorial approach,
J. Climate, 1993
• Niyogi et al. 1995, Env. Mod. Assess, Statistical –
Dynamical Experiments
• Niyogi et al. 1999 Uncertainty in initial specificationhierarchy; Boun. Layer Meteorol.
• Niyogi et al. 2002 Land surface response- midlatitudes
and tropics, J. Hydromet
(www4.ncsu.edu/~dsniyogi)
Sensitivity Analysis
Sensitivity Analysis
• Inherent component of model studies
• Both observational as well as
numerical modeling studies rely on
sensitivity analysis
• Approach – Change a variable see the
effect on the outcome
Sensitivity Analysis
• Objective
– Understand the cause – effect relationship
– Understand the relative importance of the
different processes affecting the outcome
– Develop focused efforts on improving input
for critical variables (GIGO)
– Develop if – then scenarios for policy
makers; socioeconomic analyses, …
– Evaluate models
– …
OAT Analysis
• Inherent component of model studies
• Both observational as well as
numerical modeling studies rely on
sensitivity analysis
• Approach – Change a variable see the
effect on the outcome
One at A Time Analysis
One at A Time Analysis
Another Example OAT Analysis
Summary of OAT Analysis
- Linear results
- Interactions need to
be extracted in a adhoc manner /
subjectively
- Results state ‘what
is happening’ and
not ‘how it is
happening’ in the
analysis
Sensitivity Analysis using
Observations
Sensitivity Analysis using
Observations
- Needs careful planning (several known and
unknown feedbacks possible)
- Effects cannot be “switched off” reliably (unlike
in a model)
- Modeling OAT sensitivities could be used for
developing trends and extrapolations
- Observational OAT can be largely used for
hypothesis tests (too many factors; too much
noise)
- KISS (Keep it Simple Stupid) syndrome can be
boon and a bane (too much confounding and
original results may be lost)
Sensitivity Analysis using
Observations
- Either Absent / Present scenarios tested
- Cloud cover and no clouds;
- Irrigation and no irrigation
- Fertilizer and no fertilizer
- Or High / Low scenarios tested
- High soil moisture and low soil moisture
- Ambient CO2 and Doubled CO2
High soil moisture
LESS DIFFUSE
Low Soil Moisture
Measure the environment below the two experimental
domains and evaluate the outcome (temperature, crop
yield, photosynthesis, …)
Ambient versus doubled
CO2 levels
Measure the environment below the two experimental
domains and evaluate the outcome (temperature, crop
yield, photosynthesis, …)
Example of a field sensitivity study
Field Measurements at NCSU to
assess diffuse radiation feedback
Does increase in diffuse radiation
fraction help crop yield?
MORE DIFFUSE
LESS DIFFUSE
Measure the environment below the two experimental
domains and evaluate the outcome (temperature, crop
yield, photosynthesis, …)
Comparison of OAT in models and in field
• Models – High Diffuse - > more photosynthesis
• Observations – High diffuse -> less
temperatures -> more shade on crops -> leaf
geometry changes -> large fluctuations
Clustering and Cleaning eventually gets the right results from
observations ;always an element of uncertainty that results
could have gone ‘other way’ too in some scenarios.
Q: Should the observations be relied on for testing models?
(of course yes;
but
Walker
Branchdon’t
CO2 Flux use observations as the truth!)
June - August 1997
Walker Branch CO2 Flux
June - August 1998
Rd/Rs > 0.5
Rd/Rs < 0.3
10
0
0
CO2 flux Mean Values
(umol/m^2s)
CO2 Flux umol/m^2s
20
-10
-20
-30
-40
0
200
400
600
800
Global Radiation W/m^2
1000
1200
-2
-4
-6
-8
-10
-12
-14
-16
-18
-20
0.00
0.20
0.40
0.60
Rd/Rs Mean Values
0.80
1.00
Analysis 1
If > 0.6
High Diffuse
If < 0.4
Low Diffuse
Diffuse Fraction
If Smooth
Anaysis 2
Clear
Satellite
Verified
Radiation Flux
If Reduced
If > 0.6
Analysis 3
Cloudy
High Aerosol Loading
Aerosol Loading
If < 0.4
Low Aerosol Loading
Diffuse
radiation
effect
under
cloudy
conditions
Diffuse
radiation
effect
under noncloudy
conditions
Clustering and Synthesis of Sensitivity
Experimentation Data Interpretation
June - Aug 1998
avg LHF
High LAI case
500
450
400
350
300
250
200
150
100
50
0
0
0.2
0.4
0.6
0.8
AOD 500nm
May 2001
400.00
350.00
avg LHF
300.00
250.00
200.00
150.00
100.00
50.00
0.00
0.00
0.10
0.20
0.30
0.40
0.50
0.60
AOD 500nm
Low LAI case
0.70
Observations and
Models need to go hand
in hand to help develop
the understand (don’t
treat observations alone
as the truth)
1
Too many model evaluation studies; particularly for synthesizing
processes rely overtly on observations;
Observations are essential for model testing and evaluation
But observations are also chaotic
QUESTION OBSERVATIONS
Know the uncertainty associated with the measurements
Models need not agree with all observations to be good
Models give time dependent ensemble output;
observations at a given time are just that- observation at
that point and/or time
Even observations have feedbacks embedded which have
not been traditionally extracted
No I am not a modeler!
Models do not represent the reality but neither do observations
unless they are clearly synthesized.
Need to synthesize results (observations and model output) in a
nonlinear / feedback and interaction perspective
Modeling Analysis
Feedbacks and Interactions
Feedbacks and Interactions
• Feedback processes are a result of
cause and effect.
• That is, one follows the other in a time
sequential manner
• The processes could be coupled as
well as uncoupled
• A -> B -> C
• A-> B -> C ->a -> b -> c …
Feedbacks and Interactions
• Interactions, on the other hand, implies
concurrence.
• There is no cause and effect associated with the
interactions and a simultaneous effect is
associated.
• A -> B -> C and D
Interactions
• Examples
– Medicine and Prescription Drug Use
• Drug ‘A’ will lead to helping relieve headache
• Drug ‘A’ taken while taking Drug ‘B’ will cause nausea
• Drug ‘A’ taken with coffee can cause marked improvement
– Nutrition and Health
• Results show wine is good for health; add to your diet;
–
–
–
–
red wine is better;
true for people exercising
Same effect as grape juice
Wine is not necessary, take out of your diet
– All are examples of real-life interactions occurring which
need to be resolved
Feedbacks and Interactions
• Surface Energy Balance
& Evapotranspiration
• Rn = Etr + Shf + storage;
Etr = Eg+Tr
• Gradients in Surface
Fluxes
• Non-classical
Circulation
• Convection and
Cumulus Formation
• Precipitation and Land
Use Change
• Regional Climate
Change
Factor Separation (FacSep) Analysis (Stein and Alpert,
1993; Alpert et al. 1995; J. Atmos. Sci.)
Interaction Explicit Analysis of effect of simultaneous
soil moisture and CO2 changes on terrestrial
feedback
Eo = Fo = f [ CO2 - , Moist -]
F1 = f [ CO2 + , Moist - ]
F2 = f [CO2 - , Moist + ]
F12 = f [ CO2 + , Moist + ]
E(CO2) = F1 - Eo
E(SM) = F2 - Eo
E(CO2:SM) = F12 - (F1+F2) - Fo
FacSep results can be interpreted and analyzed using either time
series tools or other traditional descriptive statistics routinely
used in One at A Time Sensitivity Analysis
Vegetation Type 6
0.0001
8 10
6 10
4 10
2 10
-5
-5
-5
An-sm-co2-6
An-sm+co2-6
An-sm-co2+6
An-sm+co2+6
-5
0
0
3 10
2 10
1 10
2
4
6
LT since 0600 (h)
8
10
12
Vegetation Type 6
-5
-5
-5
0
-1 10
-2 10
An-CO2-6
An-SM-6
An-SMCO2-6
-5
-5
0
2
4
6
LT since 0600 (h)
8
10
12
3 10
2.5 10
2 10
1.5 10
1 10
5 10
Vegetation Type 7
-5
-5
-5
An-sm-co2-7
An-sm+co2-7
An-sm-co2+7
An-sm+co2+7
-5
-5
-6
0
2
2.5 10
2 10
1.5 10
1 10
5 10
4
6
8
LT since 0600 (h)
10
12
Vegetation Type 7
-5
-5
-5
An-CO2-7
An-SM-7
An-SMCO2-7
-5
-6
0
-5 10
-1 10
-6
-5
2
4
6
8
LT since 0600 (h)
10
12
Feedbacks and Interactions
• Full Factorial (2^n) I.e. 8 combns for 3 factors; 16 for 4; 32
for 5 etc.
• At three settings (low, medium, and high) this will be 3^n
I.e. 27 for 3 factors, 64 for 4 factors etc.
• Solution?
– Fractional Factorial Approach (statistical design)
– Some confounding (all interactions / combinations not resolved)
– Several design matrices routinely available (statistics texts,
software packages, internet, …)
Fractional Factorial Designs
• “Resolution 5” all main effects and two-factor interactions
resolved (FF0516)
• “Resolution 4” some two factor Xns retained (FF0616)
• “Resolution 3” Screening type; interactions may not be
resolved (FF0508)
• Nonlinear response surface (fc0318)
• Effect = Main Effect + Interaction
• Main effect plots -> Pareto plot -> Interaction plots->
normal plots / active contrast / gambler plots -> diagnosis
of feedbacks and interactions
Summary of OAT Analysis
- Linear results
- Interactions need to
be extracted in a adhoc manner /
subjectively
- Results state ‘what
is happening’ and
not ‘how it is
happening’ in the
analysis
Analysis of Variance
Why land surface changes in tropics matter?
The answer could be in the soil moisture availability
Relevance of the results to Biosphere
Atmosphere Interaction studies
• Process - based analysis of the physical
parameterizations for every vegetation - type
• Extracted direct as well as interactive feedbacks
• Interaction effects can be equated to the indirect
effects of CO2 doubling (though not causally, often
as empirical corrections)
• Previous studies suggested, CO2 doubling will affect
C3 vegetation and may not affect C4. This may be
true only for the direct effects but considering
interactions, both C3 and C4 vegetation appears to
be significantly affected by CO2 changes
• CO2 doubling effects should not be discussed
without considering soil moisture status
• Carbon Assimilation Rates are intrinsically linked with
soil moisture availability
• Used coupled GEM based outcome over all the
nine SiB2 vegetation types to prove the hypothesis
• landscape can be a source / sink depending on the
soil moisture status
• Need to consider interactions explicitly while
analyzing Biosphere Atmosphere Interactions
Hydrological – Carbon Feedbacks
CO2 issues need implicit hydrological considerations
e.g. Ball Berry carbon assimilation / transpiration model
Gs = (m . An / Cs . RHs ) + b
m, b - specie specific ‘constants’
An
- Net Assimilation
Cs
- CO2 at leaf surface
RHs - humidity at leaf surface
 Carbon Assimilation is linked with transpiration
(which is linked with surface energy balance, and so
on…)
 Possible to scale carbon effects via hydrological
considerations
Differential Vegetation Characteristics
based SGS heterogeneity consideration
• For the example considered (C3 and C4 grassland)
– Air temperature and SHF related impacts were minimal
– Transpiration and LHF effects were significantly affected
– Largest errors could be in carbon budget or environmental
(air pollution, hydrometeorological) studies
• Results are from a One - At - Time (OAT) approach
(without interactions)
C3 - C4 Interactions
Use Factor Separation approach (Stein and Alpert, 1993) for CO2
(present day, doubled), soil moisture (wet, dry), soil texture (clay,
loam), and vegetation type (C3, C4) changes
f 0  F0
F0  ( C 3 ,CO2 , Soil  , Moist  )
f 1  F 1  F0
F 1  ( C 4 , CO 2 , Soil  , M oist  )
f 2  F 2  F0
F 2  ( C 3 , C O 2 , S o il  , M o is t  )
f 3  F 3  F0
F 3  ( C 3 , CO 2 , Soil  , M oist  )
f 4  F 4  F0
F 4  ( C 3 , CO 2 , Soil  , M oist  )
f 1 , 2  F 1 , 2  ( F 1  F 2 )  F0
F 1 , 2  ( C 4 , C O 2 , Soil  , M oist  )
f 1 , 3  F1 , 3  ( F1  F 3 )  F0
F1 , 3  ( C 4 , CO 2 , Soil  , M oist  )
f 1 , 4  F1 , 4  ( F1  F 4 )  F0
F1 , 3  ( C 4 , CO 2 , Soil  , M oist  )
f 2 ,3  F2 ,3  ( F2  F3 )  F0
F2 ,3  ( C 3 ,CO2 , Soil  , Moist  )
f 2 ,4  F2 ,4  ( F2  F4 )  F0
F2 ,3  ( C 3 ,CO2 , Soil  , Moist  )
f 3 ,4  F3 ,4  ( F3  F4 )  F0
F3 ,4  ( C 3 ,CO2 , Soil  , Moist  )
f 1 ,2 ,3  F1 ,2 ,3  ( F1 ,2  F1 ,3  F2 ,3 )  ( F1  F2  F3 )  F0
F1 ,2 ,3  ( C 4 , CO 2 , Soil  , Moist  )
f 2 ,3 ,4  F2 ,3 ,4  ( F2 ,3  F3 ,4  F2 ,4 )  ( F2  F3  F4 )  F0
F2 ,3 ,4  ( C 3 , CO 2 , Soil  , Moist  )
f 1 ,3 ,4  F1 ,3 ,4  ( F1 ,3  F1 ,4  F3 ,4 )  ( F1  F3  F4 )  F0
F1 ,3 ,4  ( C 4 , CO 2 , Soil  , Moist  )
f 1 ,2 ,4  F1 ,2 ,4  ( F1 ,2  F1 ,4  F2 ,4 )  ( F1  F2  F4 )  F0
F1 ,2 ,4  ( C 4 , CO 2 , Soil  , Moist  )
f 1 ,2 ,3 ,4  F1 ,2 ,3 ,4  ( F1 ,2 ,3  F2 ,3 ,4  F1 ,2 ,4  F1 ,3 ,4 )
 ( F1 ,2  F1 ,3  F1 ,4  F1 ,4  F2 ,3  F2 ,4  F3 ,4 )
 ( F1  F2  F3  F4 )  F0
F1 ,2 ,4  ( C 4 , CO 2 , Soil  , Moist  )
Differential Vegetation Characteristics
based SGS heterogeneity consideration
• FacSep study identified two as well as higher order
interactions are significantly active with C3 - C4
vegetation based DVC
• Interaction term do not show “expected”
compensation (SHF and LHF main effects could be
inversely linked but the interactions could be directly
related)
400
800
(b)
Rs-C4
Rs-CO2
Rs-Styp
Rs-SM
200
400
Stomatal Resistance (s/m)
500
(c)
600
0
-500
-1000
200
0
-200
Rs-C4-CO2
Rs-C4-Styp
Rs-C4-SM
Rs-CO2-Styp
Rs-CO2-SM
ETR-Styp-SM
-400
-1500
Stomatal Resistance (s/m)
(a)
Stomatal Resistance (s/m)
Higher Interactions: Stomatal Resistance
Two Factor Analysis; Stomatal Resistance
Main Effect: Stomatal Resistance
1000
0
-200
Rs-C4-CO2-Styp
Rs-C4-CO2-SM
Rs-C4-Styp-SM
Rs-CO2-Styp-SM
Rs-C4-CO2-Styp-SM
-400
-600
-600
-800
-2000
2
4
6
8
10
LT since 0600 (h)
12
14
16
18
2
4
6
8
10
LT since 0600 (h)
12
14
16
18
2
4
6
8
10
LT since 0600 (h)
12
14
16
18
5 10
-6
1 10
(c)
-5
-5
-5 10
-1 10
-6
An-C4-CO2
An-C4-Styp
An-C4-SM
An-CO2-Styp
An-CO2-SM
An-Styp-SM
-5
0
-1 10
-1.5 10
-5
2
4
6
8
10
LT since 0600 (h)
12
14
16
18
1 10
-5
5 10
-6
An-C4-CO2-Styp
An-C4-CO2-SM
An-C4-Styp-SM
An-CO2-Styp-SM
An-C4-CO2-Styp-SM
2
0
2
Net Carbon Assimilation (mol/m /s)
An-C4
An-CO2
An-Styp
An-SM
-5
2
Net Carbon Assimilation (mol/m /s)
2 10
-5
(b)
(a)
3 10
1.5 10
Net Carbon Assimilation (mol/m /s)
4 10
Higher Interactions: Photosynthesis
Two Factor Analysis: Photosynthesis
Main Effect: Photosynhtesis
-5
-5
2
4
6
8
10
LT since 0600 (h)
12
14
16
18
0
-5 10
-6
-1 10
-5
2
4
6
8
10
LT since 0600 (h)
12
14
16
18
Differential Vegetation Characteristics
based SGS heterogeneity consideration
• Are all the interactions similarly important?
– Need to identify statistically significant interactions
– Fractional Factorial Analysis performed for 12-h averaged
(day time) coupled GEM outcome
• What is the effect of CO2 doubling on such a DVC
based SGS heterogeneity?
Differential Vegetation Characteristics
based SGS heterogeneity consideration
• Numerous conditions of C3 - C4 like DVC interaction
analyzed under varying soil moisture, CO2, and soil
texture conditions
• Analysis confirms interactions are an important
component of the carbon budget (not simply addition
as often perceived, but also need to consider higher
order terms to identify ‘missing’ components)
• DVC errors were reduced under doubling of CO2
conditions (and when resources are not limiting), and
significantly persist otherwise.
• Anomaly results (CO2 doubling exercises need to be
re-evaluated
• Simple area - averaging is not adequate and may lead
to incorrect delineation of carbon source - sinks as
well as moisture budget.
Statistical design used in developing the C3 – C4 effects for changes in CO2
concentrations, soil moisture, and soil texture.
Run
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
C3 Effect
C4 Effect
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Ambient CO2
34 Pa
Soil Texture
Loam
34 Pa
Clay
68 Pa
Loam
68 Pa
Clay
34 Pa
Loam
34 Pa
Clay
68 Pa
Loam
68 Pa
Clay
34 Pa
Loam
34 Pa
Clay
68 Pa
Loam
68 Pa
Clay
34 Pa
Loam
34 Pa
Clay
68 Pa
Loam
68 Pa
Clay
Soil Moisture
+
+
+
+
+
+
+
+
Variation of the ‘effective’ variables based
on the C3 – C4 area averaging and explicit
interaction consideration.
400
EtrE
LhfE
ShfE
10*AnE
0.1*RsE
Effective Value
300
200
100
0
0
0.2
0.4
0.6
0.8
Fractional Area for C3 Grass
1
(a) Area - Averaged, and (b) Interaction
effect (~ 20 % of the direct effect)
400
40
AEtr
ALhf
AShf
10*AAn
0.1*ARs
300
(b)
20
Interaction Effect
Area Averaged Values
(a)
200
100
0
-20
IEtr
ILhf
IShf
10*IAn
0.1*IRs
-40
0
0
0.2
0.4
0.6
0.8
Fractional Area for C3 Grass
1
-60
0
0.2
0.4
0.6
0.8
Fractional Area for C3 Grass
1
“Effective” Parameter / relations for C3 C4 like DVC based SGS heterogeneity
Rseff = a3.C3 + a4.C4 + max {0.35(a3.C3), 1.5(a4.C4)}
Aneff = a3C3 + a4C4 – max{0.5(a3.C3), 0.25(a4.C4)}
Etreff = a3.C3 + a4.C4 – max{0.33(a3.C3), 0.2(a4.C4)}
LHFeff = a3.C3 + a4.C4 – max{0.25(a3.C3), 0.15(a4.C4)}
SHFeff = a3.C3 + a4.C4 + max{0.2(a3.C3), 0.3(a4.C4)}
Future Directions
• Interactions are dominant in atmospheric processes
• Methods are still evolving to extract and analyze
them
• Two of the ‘popular’ methods Fractional Factorial and
Factor Separation appear promising
• Fractional Factor Separation also evolving
• Results function of sampling?
• Need for using these observations in field
experiments and then for parameterization testing
• Question Observations..
Brain Storming Exercise
• Develop an interaction explicit scenario
which you think is not well understood?
Describe how interaction explicit approaches
may help explain the feedbacks and
interactions