Hold your tongue but do your homework?
Michael McMahon
@mcmahonecon
ESRC Festival: Nov 2014
Transparency and Deliberation within the FOMC:
A computational linguistics approach
by Stephen Hansen, Michael McMahon and Andrea Prat
FOMC
ESRC Festival: Nov 2014
1
How would your behaviour change?
Imagine...
FOMC
ESRC Festival: Nov 2014
2
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
FOMC
ESRC Festival: Nov 2014
2
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
• The decisions your committee makes affect the whole school / club.
FOMC
ESRC Festival: Nov 2014
2
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
• The decisions your committee makes affect the whole school / club.
• Now consider the following scenario:
FOMC
ESRC Festival: Nov 2014
2
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
• The decisions your committee makes affect the whole school / club.
• Now consider the following scenario:
1. Meetings are private and only the decision gets revealed
FOMC
ESRC Festival: Nov 2014
2
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
• The decisions your committee makes affect the whole school / club.
• Now consider the following scenario:
1. Meetings are private and only the decision gets revealed
2. Everything said in the meeting is recorded & put on youtube
FOMC
ESRC Festival: Nov 2014
2
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
• The decisions your committee makes affect the whole school / club.
• Now consider the following scenario:
1. Meetings are private and only the decision gets revealed
2. Everything said in the meeting is recorded & put on youtube
Dilemma
Does it matter which is selected?
FOMC
ESRC Festival: Nov 2014
2
Transparency and Deliberation
Mario Draghi (2013): “It would be wise to have a richer communication
about the rationale behind the decisions that the governing council takes.”
Minutes?
Transcripts?
Fed
X
X
BoE
X
X
ECB
X
X
FOMC
ESRC Festival: Nov 2014
3
Transparency and Deliberation
Mario Draghi (2013): “It would be wise to have a richer communication
about the rationale behind the decisions that the governing council takes.”
Minutes?
Transcripts?
Fed
X
X
BoE
X
X
ECB
X
X
April 30, 2014: BoE to review of non-release of transcripts
July 3, 2014: ECB to release account of meetings
FOMC
ESRC Festival: Nov 2014
3
Transparency and Deliberation
Mario Draghi (2013): “It would be wise to have a richer communication
about the rationale behind the decisions that the governing council takes.”
Minutes?
Transcripts?
Fed
X
X
BoE
X
X
ECB
X
X
April 30, 2014: BoE to review of non-release of transcripts
July 3, 2014: ECB to release account of meetings
Specific goal of my paper
We want to study how transparency affects FOMC deliberation.
⇒ how is internal deliberation affected by greater external communication?
FOMC
ESRC Festival: Nov 2014
3
The World is Watching
FOMC
ESRC Festival: Nov 2014
4
The outline of our paper
FOMC
ESRC Festival: Nov 2014
5
The outline of our paper
Theory
Discipline vs Conformity/Non-conformity
FOMC
ESRC Festival: Nov 2014
5
The outline of our paper
Theory
Discipline vs Conformity/Non-conformity
Empirical 1
FOMC
ESRC Festival: Nov 2014
5
The outline of our paper
Theory
Discipline vs Conformity/Non-conformity
Empirical 1
Natural Experiment
+
Diff-in-Diff
FOMC
ESRC Festival: Nov 2014
5
The outline of our paper
Theory
Discipline vs Conformity/Non-conformity
Empirical 1
Natural Experiment
+
Diff-in-Diff
FOMC Transcripts
+
Computational Linguistics
FOMC
ESRC Festival: Nov 2014
5
The outline of our paper
Theory
Discipline vs Conformity/Non-conformity
Empirical 1
Natural Experiment
+
FOMC Transcripts
+
Diff-in-Diff
Computational Linguistics
Evidence of Discipline and Conformity
FOMC
ESRC Festival: Nov 2014
5
The outline of our paper
Theory
Discipline vs Conformity/Non-conformity
Empirical 1
Natural Experiment
+
FOMC Transcripts
+
Diff-in-Diff
Computational Linguistics
Evidence of Discipline and Conformity
Empirical 2
Influence: indirect test of informativeness
FOMC
ESRC Festival: Nov 2014
5
The outline of our paper
Theory
Discipline vs Conformity/Non-conformity
Empirical 1
Natural Experiment
+
FOMC Transcripts
+
Diff-in-Diff
Computational Linguistics
Evidence of Discipline and Conformity
Empirical 2
Influence: indirect test of informativeness
Empirical 3
Macro changes consistent with info ↑
FOMC
ESRC Festival: Nov 2014
5
Take-away for Central Bank Design
Policymakers & future research should explore ways to structure the
deliberation process in order to maximize the discipline effect and
minimize the conformity effect.
FOMC
ESRC Festival: Nov 2014
6
Transcript Data
• Nearly verbatim records of FOMC meetings available from Fed
website with five year lag
• Includes staff statements from appendices
• Use the entire Greenspan era (8/87-1/06): 149 meetings:
• 46,502 statements
• Almost 6m words
• Lots of other text data available from Fed
• We have also collected biographical information on FOMC members
FOMC
ESRC Festival: Nov 2014
7
The Natural Experiment
• FOMC meetings were recorded and transcribed from at least the
mid-1970’s in order to assist with the preparation of the minutes.
• Committee members were unaware that transcripts were stored
• October 1993: Alan Greenspan acknowledged the transcripts’
existence to the Senate Banking Committee.
• The Fed then quickly agreed:
• To begin publishing them with a five-year lag.
• To publish the back data
• A natural experiment to assess the effect of transparency:
• Prior to Nov 1993: Discussion took place under the assumption that
individual statements would not be on the public record
• After Nov 1993: Each policy maker knew that every spoken word
would be public within five years.
FOMC
ESRC Festival: Nov 2014
8
Natural Experiment - FOMC members reaction (Oct 1993)
President Keehn: “Until 10 minutes ago I had no awareness that we did
have these detailed transcripts.”
President Boehne: “...to the very best of my recollection I don’t believe
that Chairman Burns or his successors ever indicated to the Committee as
a group that these written transcripts were being kept. What Chairman
Burns did indicate at the time when the Memorandum was discontinued
was that the meeting was being recorded and the recording was done for
the purpose of preparing what we now call the minutes but that it would
be recorded over at subsequent meetings. So there was never any
indication that there would be a permanent, written record of a transcript
nature. .... So I think most people in the subsequent years proceeded
on that notion that there was not a written transcript in existence.
And I suspect that many people on this conference call may have acquired
this knowledge at about the same time that Si Keehn did.”
FOMC
ESRC Festival: Nov 2014
8
The Natural Experiment
• FOMC meetings were recorded and transcribed from at least the
mid-1970’s in order to assist with the preparation of the minutes.
• Committee members were unaware that transcripts were stored
• October 1993: Alan Greenspan acknowledged the transcripts’
existence to the Senate Banking Committee.
• The Fed then quickly agreed:
• To begin publishing them with a five-year lag.
• To publish the back data
FOMC
ESRC Festival: Nov 2014
8
The Natural Experiment
• FOMC meetings were recorded and transcribed from at least the
mid-1970’s in order to assist with the preparation of the minutes.
• Committee members were unaware that transcripts were stored
• October 1993: Alan Greenspan acknowledged the transcripts’
existence to the Senate Banking Committee.
• The Fed then quickly agreed:
• To begin publishing them with a five-year lag.
• To publish the back data
• A natural experiment to assess the effect of transparency:
• Prior to Nov 1993: Discussion took place under the assumption that
individual statements would not be on the public record
• After Nov 1993: Each policy maker knew that every spoken word
would be public within five years.
FOMC
ESRC Festival: Nov 2014
8
The Latent Dirichlet Allocation (LDA) model
• Blei, Ng and Jordan (2003) cited 8,800+ times: new to economics.
• LDA (and its extensions) estimates what fraction of each document
in a collection is devoted to each of several “topics.”
• JSTOR example
• Great promise for economics more broadly.
• LDA is an unsupervised learning approach - we don’t set probabilities
FOMC
ESRC Festival: Nov 2014
9
The Latent Dirichlet Allocation (LDA) model
• Blei, Ng and Jordan (2003) cited 8,800+ times: new to economics.
• LDA (and its extensions) estimates what fraction of each document
in a collection is devoted to each of several “topics.”
• JSTOR example
• Great promise for economics more broadly.
• LDA is an unsupervised learning approach - we don’t set probabilities
1. Start with words in statements
FOMC
ESRC Festival: Nov 2014
9
The Latent Dirichlet Allocation (LDA) model
• Blei, Ng and Jordan (2003) cited 8,800+ times: new to economics.
• LDA (and its extensions) estimates what fraction of each document
in a collection is devoted to each of several “topics.”
• JSTOR example
• Great promise for economics more broadly.
• LDA is an unsupervised learning approach - we don’t set probabilities
1. Start with words in statements
2. Tell the model how many topics there should be
• Perplexity scores
FOMC
ESRC Festival: Nov 2014
9
The Latent Dirichlet Allocation (LDA) model
• Blei, Ng and Jordan (2003) cited 8,800+ times: new to economics.
• LDA (and its extensions) estimates what fraction of each document
in a collection is devoted to each of several “topics.”
• JSTOR example
• Great promise for economics more broadly.
• LDA is an unsupervised learning approach - we don’t set probabilities
1. Start with words in statements
2. Tell the model how many topics there should be
• Perplexity scores
3. Model will generate βK topic distributions
• the distribution over words for each topic
FOMC
ESRC Festival: Nov 2014
9
The Latent Dirichlet Allocation (LDA) model
• Blei, Ng and Jordan (2003) cited 8,800+ times: new to economics.
• LDA (and its extensions) estimates what fraction of each document
in a collection is devoted to each of several “topics.”
• JSTOR example
• Great promise for economics more broadly.
• LDA is an unsupervised learning approach - we don’t set probabilities
1. Start with words in statements
2. Tell the model how many topics there should be
• Perplexity scores
3. Model will generate βK topic distributions
• the distribution over words for each topic
4. Model also generates θd document distributions
FOMC
ESRC Festival: Nov 2014
9
Example statement: Yellen, March 2006, #51
Raw Data → Remove Stop Words → Stemming → Multi-word tokens = Bag of Words
We have noticed a change in the relationship between the core CPI and the
chained core CPI, which suggested to us that maybe something is going on
relating to substitution bias at the upper level of the index. You focused
on the nonmarket component of the PCE, and I wondered if something
unusual might be happening with the core CPI relative to other measures.
Federal Funds Rate → fed fund rate → ffr
monetary policy → monetari polici → monpol
FOMC ESRC Festival: Nov 2014
10
Example statement: Yellen, March 2006, #51
Raw Data → Remove Stop Words → Stemming → Multi-word tokens = Bag of Words
We have noticed a change in the relationship between the core CPI and the
chained core CPI, which suggested to us that maybe something is going on
relating to substitution bias at the upper level of the index. You focused
on the nonmarket component of the PCE, and I wondered if something
unusual might be happening with the core CPI relative to other measures.
Federal Funds Rate → fed fund rate → ffr
monetary policy → monetari polici → monpol
FOMC ESRC Festival: Nov 2014
10
Example statement: Yellen, March 2006, #51
Raw Data → Remove Stop Words → Stemming → Multi-word tokens = Bag of Words
We have noticed a change in the relationship between the core CPI and the
chained core CPI, which suggested to us that maybe something is going on
relating to substitution bias at the upper level of the index. You focused
on the nonmarket component of the PCE, and I wondered if something
unusual might be happening with the core CPI relative to other measures.
Federal Funds Rate → fed fund rate → ffr
monetary policy → monetari polici → monpol
FOMC ESRC Festival: Nov 2014
10
Example statement: Yellen, March 2006, #51
Raw Data → Remove Stop Words → Stemming → Multi-word tokens = Bag of Words
We have noticed a change in the relationship between the core CPI and the
chained core CPI, which suggested to us that maybe something is going on
relating to substitution bias at the upper level of the index. You focused
on the nonmarket component of the PCE, and I wondered if something
unusual might be happening with the core CPI relative to other measures.
Federal Funds Rate → fed fund rate → ffr
monetary policy → monetari polici → monpol
FOMC ESRC Festival: Nov 2014
10
Example statement: Yellen, March 2006, #51
Raw Data → Remove Stop Words → Stemming → Multi-word tokens = Bag of Words
We have noticed a change in the relationship between the core CPI and the
chained core CPI, which suggested to us that maybe something is going on
relating to substitution bias at the upper level of the index. You focused
on the nonmarket component of the PCE, and I wondered if something
unusual might be happening with the core CPI relative to other measures.
Federal Funds Rate → fed fund rate → ffr
monetary policy → monetari polici → monpol
FOMC ESRC Festival: Nov 2014
10
Example statement: Yellen, March 2006, #51
Allocation
We have 17ticed a 39ange in the 39lationship 1etween the 25re 25I and the
41ained 25re 25I, which 25ggested to us that 36ybe 36mething is 38ing on
43lating to 25bstitution 20as at the 25per 39vel of the 16dex. You 23cused
on the 25nmarket 25mponent of the 25E, and I 32ndered if 38mething
16usual might be 4appening with the 25re 25I 16lative to other 25asures.
Federal Funds Rate → fed fund rate → ffr
monetary policy → monetari polici → monpol
FOMC ESRC Festival: Nov 2014
10
0
# of Word Assignments
5
10
Example statement: Yellen, March 2006, #51
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50
Topic Number
Discussion topics
Economics topics
FOMC ESRC Festival: Nov 2014
11
Topic 25 - Inflation
FOMC ESRC Festival: Nov 2014
12
Topic 38 - Banking
FOMC ESRC Festival: Nov 2014
13
Topic 22 - Referring to figures (staff materials)
FOMC ESRC Festival: Nov 2014
14
Topic 40 - Risk
FOMC ESRC Festival: Nov 2014
15
1985m1
0
0
100
200
300
Uncertainty (MA)
Fraction of time in meeting
.02
.04
.06
.08
400
BBD Uncertainty and Discussion of “Risks Topic”
1990m1
1995m1
2000m1
FOMC Meeting Date
Risk topic
2005m1
2010m1
BBD Uncertainty
FOMC ESRC Festival: Nov 2014
16
Topic 6 - CB communication
FOMC ESRC Festival: Nov 2014
17
Fraction of time spent on communication topic
0
.05
.1
.15
.2
Topic 16 Discussion: Surprised by transparency revelation?
1985m1
1990m1
1995m1
2000m1
FOMC Meeting Date
2005m1
2010m1
FOMC ESRC Festival: Nov 2014
18
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
• The decisions your committee makes affect the whole school / club.
• Now consider the following scenario:
1. Meetings are private and only the decision gets revealed
2. Everything said in the meeting is recorded & put on youtube
FOMC ESRC Festival: Nov 2014
19
How would your behaviour change?
Imagine...
• You are sitting on an important school or club committee.
• The decisions your committee makes affect the whole school / club.
• Now consider the following scenario:
1. Meetings are private and only the decision gets revealed
2. Everything said in the meeting is recorded & put on youtube
If scenario 2 is selected...
Would you hold your tongue but do your homework?
FOMC ESRC Festival: Nov 2014
19