Nutrition, information, and household
behaviour: experimental evidence from
Malawi
Emla Fitzsimons (IoE and IFS)
Bansi Malde (IFS and UCL)
Alice Mesnard (City U and IFS)
Marcos Vera-Hernández (IFS and UCL)
Towards a common language in statistical methodology:
examples from current applications in economics health
research – Joint Pathways-PEPA event – April 23
Motivation
• Very poor nutritional outcomes in developing countries
• One possible reason (of several) is that households have
incorrect knowledge on how best improve nutrition
• Challenging to establish it using observational data
– Those with better knowledge on nutrition also have
more resources and it might be difficult to fully control
for them
• Exploit a randomised trial which provided mothers
with information on child nutrition only
– MaiMwana Infant Feeding Intervention
Objectives
• Did MaiMwana Infant Feeding Intervention improve mother’s
knowledge on child nutrition?
• Did the improved knowledge led to better nutritional outcomes
for children?
• How did households adjust to this improved knowledge to make
the improvement in nutrition happen?
– Use a simple theoretical model to show how the household adjust to
improved knowledge
Some methodological challenges
• High attrition (= loss to follow-up) of around 33%
• More than 30 outcome variables
• Only 24 clusters, special methods might be required to obtain valid
p-values to test hypotheses of interest
Setting: Mchinji (Malawi)
• Child health is very poor in Malawi
– 48% of kids aged < 5yrs stunted
• Misperceptions on child nutrition are widespread:
– Common to give porridge diluted with unsterilized water to infants as
young as 1 week
– Widespread belief that eggs are harmful to 9-month old infants and
that the broth of a soup is more nutritious than vegetables and meat
within
– Common belief that children should be given broth in you cook the
vegetables/meat, instead of the vegetables/meat themselves
The Intervention
• Set up in 2005 by Mai Mwana, a research and development project,
working to improve maternal and child health in collaboration with
the Institute of Child Health at UCL
• Trained local women provide information and advice on infant
feeding to mothers of babies aged < 6 months
– 5 visits, once before birth, 4 times after birth
– Counsellors cover a population of around1,000 individuals
• All pregnant women are eligible for the intervention, but in practice
around 60% are visited by counsellors
• Intervention began in July 2005, and is still on-going
The Intervention
• Visits:
– Exclusive breastfeeding and post-breastfeeding nutrition
– Locally available foods (i.e. groundnuts)
– How to cook so that child digests better
– Clarify some local beliefs (whether to give broth or vegetables/meat)
– Some information also provided on hygiene, HIV testing, vaccination and
family planning
Experimental Design
• Mchinji District divided into 48 clusters, each with a population of
roughly 8,000 individuals
• The inner part of each cluster was chosen as study area, leaving a
buffer area to limit contamination between neighbouring clusters
• 12 clusters randomly chosen to receive intervention, 12 clusters
serve as controls
Source: Lewycka et al 2010
Sample
• Sampling frame: census of all women of child bearing age in the
study areas conducted by Mai Mwana in 2004, pre-intervention
– Also our source of baseline information
• Intervention started in 2005
• Follow-up data collected in 2008-09 and 2009-10
– Random sample of 104 women from each cluster drawn the baseline
census, regardless of their fertility
– Target sample of 2496 women
Evaluation Sample
• Succeeded in interviewing two-thirds of the drawn sample in the
first follow-up
– Final sample of 1660 households
Sample balance:
Table 1: Baseline Sample Balance
Full Sample
Woman's Characteristics
Married (dv = 1)
Some Primary Schooling or Higher
Some Secondary Schooling or Higher
Age (years)
Chewa
Christian
Farmer
Student
Small Business/Rural Artisan
Interviewed Sample
Difference:
Control Treatment Group Control p-value
Control
Group
0.615
0.707
0.066
24.571
0.948
0.977
0.661
0.236
0.036
0.661
0.682
0.060
25.492
0.957
0.979
0.688
0.204
0.037
-0.021
0.033
0.010
-0.180
-0.044
0.006
-0.075
0.015
0.030
0.386
0.402
0.535
0.637
0.330
0.476
0.108
0.438
0.129
Difference:
Treatment Control p-value
-0.034
0.040
-0.007
-0.429
-0.050
0.008
-0.060
0.022
0.024
0.184
0.340
0.545
0.376
0.246
0.336
0.128
0.274
0.220
Some Primary Schooling or Higher
Some Secondary Schooling or Higher
Age (years)
Chewa
Table 1: Baseline Sample Balance
Christian
Farmer
Student
Small Business/Rural Artisan
Sample Balance:
Woman's Characteristics
Household
Married
(dv Characteristics
= 1)
Some
Primaryhousehold
Schooling or Higher
Agricultural
Some
Schooling
or sand
Higher
Main Secondary
Flooring Material:
Dirt,
or dung
Age (years)
Main roofing Material: Natural Material
Chewa
HH Members Work on Own Agricultural Land
Christian
Piped water
Farmer
Student
Traditional pit toilet (dv = 1)
Small
Artisan
# of hhBusiness/Rural
members
# of sleeping rooms
Household Characteristics
HH has electricity
Agricultural
household
HH has
radio Material: Dirt, sand or dung
Main
Flooring
Main
roofing
Material: Natural Material
HH has
bicycle
HH Members Work on Own Agricultural Land
HH has motorcycle
Piped water
HH has car
Traditional pit toilet (dv = 1)
hasmembers
paraffin lamp
#HH
of hh
has oxcartrooms
#HH
of sleeping
HH
N has electricity
HH has radio
HH has bicycle
0.707
0.066
24.571
0.948
0.977
0.661
0.236
Control
0.036
Group
0.033
0.010
-0.180
-0.044
0.006
Full Sample
-0.075
Difference:
0.015
Treatment 0.030
Control
0.402
0.535
0.637
0.330
0.476
0.108
0.438
0.129
p-value
0.615
0.707
0.995
0.066
0.913
24.571
0.853
0.948
0.942
0.977
0.011
0.661
0.236
0.772
0.036
5.771
-0.021
0.033
-0.005
0.010
-0.041
-0.180
-0.018
-0.044
-0.057
0.006
0.040
-0.075
0.015
0.054
0.030
0.066
0.386
0.402
0.471
0.535
0.232
0.637
0.697
0.330
0.124
0.476
0.314
0.108
0.438
0.218
0.129
0.817
0.661
0.682
0.995
0.060
0.916
25.492
0.857
0.957
0.950
0.979
0.009
0.688
0.204
0.791
0.037
5.848
-0.034
0.040
0.002
-0.007
-0.027
-0.429
-0.004
-0.050
-0.056
0.008
0.032
-0.060
0.022
0.054
0.024
0.132
0.184
0.340
0.591
0.545
0.474
0.376
0.891
0.246
0.120
0.336
0.340
0.128
0.274
0.182
0.220
0.863
2.116
0.002
0.995
0.630
0.913
0.853
0.509
0.942
0.008
0.011
0.006
0.772
0.925
5.771
0.058
2.116
0.002
1248
0.199
0.007
-0.005
0.030
-0.041
-0.018
0.015
-0.057
0.001
0.040
-0.002
0.054
0.032
0.066
-0.015
0.199
0.007
1248
0.038*
0.166
0.471
0.408
0.232
0.697
0.643
0.124
0.925
0.314
0.612
0.218
0.262
0.817
0.204
0.038*
2.152
0.002
0.995
0.641
0.916
0.857
0.512
0.950
0.007
0.009
0.007
0.791
0.926
5.848
0.059
2.152
0.002
846
0.166
0.004
0.002
0.015
-0.027
-0.004
0.008
-0.056
0.002
0.032
-0.003
0.054
0.036
0.132
-0.022
0.166
0.004
814
0.128
0.338
0.591
0.709
0.474
0.891
0.843
0.120
0.779
0.340
0.298
0.182
0.178
0.863
0.090+
0.128
0.630
0.509
0.030
0.015
0.641
0.512
0.015
0.008
0.166
0.408
0.643
0.682
0.040
0.340
0.060
-0.007
0.545
25.492
-0.429
0.376
0.957
-0.050
0.246
0.979 Interviewed
0.008Sample 0.336
0.688
-0.060
0.128
Difference:
0.204
0.022
0.274
Control
Treatment 0.037
0.024
0.220
Group
Control
p-value
0.338
0.709
0.843
Notes to Table: + indicates significant at the 10% level, * indicates significant at the 5% level. p-values reported here are computed using the wild
Theoretical model
• Simple model to obtain predictions on how improved knowledge
will affect household choices: child nutrition, consumption, labour
supply…
• Households use a “production function” to transform money spent
on food for children ( ) into child nutrition ( )
• This transformation depends on a efficiency parameter ( )
• With more knowledge, the household will be able to get better
nutritional outcomes with the same money spent on food
• ( ) will increase with the intervention
© Institute for Fiscal Studies
Theoretical model
• Still, assuming the existence of a production function
does not solve our problem: How does the change in ( ) affect
the change in ( ) and ( )?
• We assume that households “mix” adult consumption ( ), leisure ( ),
and child nutrition outcome ( ) so as to maximize a certain type of
utility (≈ “satisfaction”) function
• Subject to a resource constraint that earnings plus non-labour income
cannot be more than child and adult consumption
© Institute for Fiscal Studies
Model predictions:
Providing information on child nutrition to parent ( θ) will (under
most common assumptions):
1) Increase child consumption
2) Increase adult labour supply
3) Reduce adult consumption
4) Increase household consumption
A priori, one could have thought that households would spend the
same amount of resources on child nutrition but just doing it better.
Why is that not the answer? With improved knowledge, each
additional Kwacha spent on child food has a higher return than before
so they want to do more of it: they work more !
Regression Model
i for child, h for household, c for cluster, t for time periods
Y = outcome variable (child nutrition, food intake…)
T = 1 if cluster is an intervention one, 0 if control
X = Age and gender of the child at time t
Z = Education levels and Chewa ethnicity proportion in the cluster in
2004
Regression coefficients estimated using Ordinary Least
Squares
Inference
Obtaining valid confidence-intervals for
as well as pvalues for H0:
= 0 must take into account the error
terms are not independent for children/households living in
the same cluster.
One approach is to model this dependency as is done in
multilevel modeling (a parametric model for the error term).
More common in economics is to use Cluster Robust
Standard Errors, which does not require parametric
assumptions on the dependence relationship of the error
term (cluster option in STATA, GENMOD in SAS)
C
VˆLZ  ( X ' X ) ( X cucuc' X c' )( X ' X ) 1
1
c 1
Inference
However, the Cluster Robust Standard Errors provides
standard errors which are too small if the number of
clusters is not sufficiently large (24 is sort of small)
We implement both:
• Wild-bootstrap-t procedure recommended by Cameron, Gelbach,
Miller (2008)
• Randomisation Inference (Fisher 1935; Rosenbaum 2002; Small et
al 2008)
And compare the test size of both approaches using a Montecarlo
exercise sharing important features of our data
Interestingly, both approaches provide quite similar results
Multiple Outcomes
Interested in testing the effects of the intervention on 6
domains: health knowledge, child consumption, household
consumption, labor supply, child growth and child morbidity
For each of these domains, we have multiple measures !
almost 30 outcomes in total
Concerns about multiple inference: The probability of
rejecting a test is increasing in the number of tests
carried out
Reduce the number of tests… Aggregate multiple outcome
measures in a domain into a summary index following
Anderson (2008)
Multiple Outcomes
To build the index… for the outcomes of a particular domain
1. Re-define outcomes so that a higher value implies a
better outcome
2. Standardize outcomes to have a 0 mean and standard
deviation of 1
3. Summary index is calculated as a weighted mean of the
standardized outcome values within each domain
1. Weights are obtained from the VCM of the outcomes
2. Less weight is given to highly correlated outcomes
3. Boosts efficiency
Multiple Outcomes: examples of some of the
components of each subdomain
Results – Indices main subdomains
Results – summary
• Positive and statistically significant effects on
– Knowledge on nutrition
– Child food consumption
– Household food consumption
– Male labour supply (but not female)
– Child physical growth (older than 6 months only)
• Magnitude of effects is much harder to assess using these
summary index, better to go to the raw outcomes
– Some of them follow…
Results – Household Consumption (MK 140=$1)
• Substantial increases in consumption, particularly food
• Concentrated among nutritious, but more expensive foods –
proteins and fruit and vegetables
Child nutrition
Attrition
The baseline was conducted in 2004, the first follow-up in
2008
Around 1/3 of the planned sample could not be found
This could be a big threat to the validity of the results (but it
gives us the benefit of evaluating the medium term effects of
the program rather than the very short term ones)
Obviously, attrition was not random: those who attrited had
more education in 2004 and were less likely to be married in
2004
Reassuringly, the attrition rate was very similar in Treatment
and Control clusters (34.7% vs. 32.2%)
Attrition:
Also reassuringly, attrition did not alter the balance in
observable characteristics (collected at baseline) between
treatment and control clusters
Table 1: Baseline Sample Balance
Full Sample
Control
Group
Difference:
Treatment Control
Woman's Characteristics
Married (dv = 1)
Some Primary Schooling or Higher
Some Secondary Schooling or Higher
Age (years)
Chewa
Christian
Farmer
Student
Small Business/Rural Artisan
0.615
0.707
0.066
24.571
0.948
0.977
0.661
0.236
0.036
Household Characteristics
Agricultural household
Main Flooring Material: Dirt, sand or dung
0.995
0.913
Interviewed Sample
p-value
Control
Group
Difference:
Treatment Control
p-value
-0.021
0.033
0.010
-0.180
-0.044
0.006
-0.075
0.015
0.030
0.386
0.402
0.535
0.637
0.330
0.476
0.108
0.438
0.129
0.661
0.682
0.060
25.492
0.957
0.979
0.688
0.204
0.037
-0.034
0.040
-0.007
-0.429
-0.050
0.008
-0.060
0.022
0.024
0.184
0.340
0.545
0.376
0.246
0.336
0.128
0.274
0.220
-0.005
-0.041
0.471
0.232
0.995
0.916
0.002
-0.027
0.591
0.474
Some Primary Schooling or Higher
Some Secondary Schooling or Higher
Age (years)
Chewa1: Baseline Sample Balance
Table
Christian
Farmer
Student
Small Business/Rural Artisan
0.707
0.066
24.571
0.948
0.977
0.661
0.236
Control
0.036
Group
0.033
0.010
-0.180
-0.044
Full0.006
Sample
-0.075
Difference:
0.015 Treatment
0.030
Control
0.402
0.535
0.637
0.330
0.476
0.108
0.438
0.129
p-value
Woman's Characteristics
Household
Married
(dv Characteristics
= 1)
Agricultural
Some
Primaryhousehold
Schooling or Higher
Some
Schooling
or sand
Higher
Main Secondary
Flooring Material:
Dirt,
or dung
Age
(years)
Main roofing Material: Natural Material
Chewa
HH Members Work on Own Agricultural Land
Christian
Piped water
Farmer
Traditional pit toilet (dv = 1)
Student
# of hhBusiness/Rural
members
Small
Artisan
0.615
0.995
0.707
0.066
0.913
24.571
0.853
0.948
0.942
0.977
0.011
0.661
0.772
0.236
5.771
0.036
-0.021
-0.005
0.033
0.010
-0.041
-0.180
-0.018
-0.044
-0.057
0.006
0.040
-0.075
0.054
0.015
0.066
0.030
0.386
0.471
0.402
0.535
0.232
0.637
0.697
0.330
0.124
0.476
0.314
0.108
0.218
0.438
0.817
0.129
0.661
0.995
0.682
0.060
0.916
25.492
0.857
0.957
0.950
0.979
0.009
0.688
0.791
0.204
5.848
0.037
-0.034
0.002
0.040
-0.007
-0.027
-0.429
-0.004
-0.050
-0.056
0.008
0.032
-0.060
0.054
0.022
0.132
0.024
0.184
0.591
0.340
0.545
0.474
0.376
0.891
0.246
0.120
0.336
0.340
0.128
0.182
0.274
0.863
0.220
# of sleeping rooms
Household
Characteristics
HH has electricity
Agricultural
HH has radiohousehold
Main
Flooring
HH has
bicycleMaterial: Dirt, sand or dung
Main roofing Material: Natural Material
HH has motorcycle
HH Members Work on Own Agricultural Land
HH has car
Piped water
HH has paraffin
lamp
Traditional
pit toilet
(dv = 1)
HH
has
oxcart
# of hh members
#Nof sleeping rooms
2.116
0.002
0.995
0.630
0.913
0.509
0.853
0.008
0.942
0.006
0.011
0.925
0.772
0.058
5.771
1248
2.116
0.199
0.007
-0.005
0.030
-0.041
0.015
-0.018
0.001
-0.057
-0.002
0.040
0.032
0.054
-0.015
0.066
1248
0.199
0.038*
0.166
0.471
0.408
0.232
0.643
0.697
0.925
0.124
0.612
0.314
0.262
0.218
0.204
0.817
2.152
0.002
0.995
0.641
0.916
0.512
0.857
0.007
0.950
0.007
0.009
0.926
0.791
0.059
5.848
846
2.152
0.166
0.004
0.002
0.015
-0.027
0.008
-0.004
0.002
-0.056
-0.003
0.032
0.036
0.054
-0.022
0.132
814
0.166
0.128
0.338
0.591
0.709
0.474
0.843
0.891
0.779
0.120
0.298
0.340
0.178
0.182
0.090+
0.863
Attrition:
0.682
0.040
0.340
0.060
-0.007
0.545
25.492
-0.429
0.376
0.957
-0.050
0.246
0.979 Interviewed
0.008
Sample 0.336
0.688
-0.060
0.128
Difference:
0.204
0.022 0.274
Control
Treatment
0.037
0.024
0.220
Group
Control
p-value
0.038*
0.128
HH has electricity
0.002
0.007
0.166
0.002
0.004
0.338
HH
hastoradio
0.630 significant
0.030
0.408p-values reported
0.641 here are computed
0.015 using0.709
Notes
Table: + indicates significant at the 10% level, * indicates
at the 5% level.
the wild
HH
has
bicycle
0.509
0.015
0.643
0.512
0.008
0.843
cluster bootstrap-t procedure as in Cameron et al. 2008, explained in section 4.1. Full Sample includes all women (and their households) originally
Attrition:
But it could still be that attrition caused the treatment
group to be different to the control group in some
variable that we have not measured.
Possible approaches:
-Compute bounds
-Heckman selection model
Attrition: Heckman selection model
Attrition: Heckman selection model
After assuming bivariate normality of the error terms, an
expression can be found for E(Yihct|Dihct=1) which forms the basis
of a modified regression
It is considered good practice to exclude from the Yihct model one
or more variables that are good predictors of whether the
individual is found or not (W)
-But results will be misleading if it turns out that the excluded
variables should not have been excluded
We exclude the size of the interviewers plot of land and the
number of children 0-3 living in the interviewer’s household
-They affect the opportunity cost of searching for interviewees as
well as the likelihood that they knew where the interviewees
should be living
Attrition: Heckman selection model
Results are quite similar to the basic OLS ones, so attrition does
not seem a problem
Conclusions
• Investigate how household choices change due to the
provision of information on child nutrition
• Exploit a randomized experiment in rural Malawi that
provided mothers of young children with information
on child nutrition for identification
• Deal with several challenges such as (1) multiple
outcomes, (2) attrition, (3) relatively low number of
clusters
• To deal with attrition, we estimated a Heckman
selection model
• Positive results found on knowledge on nutrition, child
food intake, household consumption and children
anthropometrics
Extra slides
Condition to ensure child consumption increases