From Data to Policy: Scientific Excellence
Is Our Future
Sallie K ELLER -M C N ULTY
Science, engineering, technology, and people—these are the ingredients that must come together to support the growing complexity of
today’s global challenges, ranging from international security to space exploration. As scientists and engineers, it is essential that we develop
the means to put our work into a decision context for policy makers; otherwise, our efforts will only inform the writers of textbooks and not
the leaders who shape the world within which we live. Statisticians must step up to that challenge! Scientific and technical progress requires
interdisciplinary teams, because it is impossible for a single individual to have enough knowledge to solve many of today’s problems,
for example, mapping the genome, modeling the spread of a pandemic, and developing diagnostic and treatment devices for developing
countries. A principal role of the statistician is to bring the cutting edge of statistical sciences to these problems. By the nature of our training,
statisticians are well poised to assume the role of science and technology integrator. To be successful, this must place statisticians closer to
policy pressures and politics. This address will focus on the growing expectations facing statistical sciences and how we, as statisticians,
must take responsibility for separating the scientific method from the politics of the scientific process to guarantee that scientific excellence
and impact is communicated to decision makers.
KEY WORDS: Information integration; Policy; Science integration process; Scientific method.
Thank you very much for coming tonight.
I think this is a really exciting time for our profession. It is
my job tonight to see if I can convince you of this. In preparation for this talk, I made the mistake of reading the last nearly
100 years of presidential addresses to the American Statistical
Association. This is a mistake because once you take that wonderful walk through history through the eyes of our leadership,
you realize you have nearly nothing new to say. Nevertheless,
I am going to attempt to inspire you to at least think about some
new things.
This is an incredible time to be a scientist or engineer. I use
these words in the absolutely broadest sense, including areas
such as mathematics, computer science, statistics, chemistry,
biology, civil engineering, social sciences, art, and even humanities. We are all scientists and engineers. All of those areas can
contribute to scientific discovery and technology innovation in
ways that propel our society forward. I really think that is what
it is all about.
The global challenges we face today are immense—everything from national and international security, global climate
change, the success or collapse of international economics,
space exploration, and biomedicine to air quality. I chose the
theme of this meeting, “Statistics in an Uncertain World: Meeting Global Challenges,” to get us to think beyond some of the
boundaries our profession is comfortable with and try to understand how we can contribute to the resolution of society’s challenges. To contribute, we have to figure out how we are going
to leverage our deepest theory, the most complex computation,
our roots in experimentation and experimental design, multiple
disciplines, information of all sorts, and all of the technology
innovations we have access to. How do we blend all of this together to try to meet these challenges?
It is a pretty daunting task. Perhaps the real question is, “How
do we fit in?” How do statisticians and statistical sciences fit
into all this?
I decided I was not going to debate or discuss whether we
have a discipline tonight. I am standing up here to tell you that
we do, and I am going to tell you what it is. I know you all know
this, but I am just going to remind you, because understanding
our discipline helps to answer the question of how we fit in.
Statistics is the discipline designed to unravel the mystery of
decision making under uncertainty. “Uncertainty R Us.” Statistics requires data and information to come together to support
decision making. Ours is the field that tries to understand, quantify, and explain what this means. Our discipline is unique in the
sense that there are no fundamental physical laws underlying it.
We are not about rocks; we are not about particles; we are not
about “E = mc2 .” We are about trying to bring to bear for society the mysteries of information and what it tells us. Statistics
exists because the world is not deterministic. As a discipline, we
are inherently interdisciplinary. There is simply nothing to debate regarding the existence and importance of our discipline!
The real questions today are “Do we have a profession?, Do
we need a profession?, Do we have a future?, Do we actually
have jobs, or do we have careers?” And if we believe we have
a profession and have careers, how do we best use them for the
benefit of society?
In Hunter’s (1994) 1993 address, he gave a wonderful definition of what a profession is. He said:
Sallie Keller-McNulty is William and Stephanie Sick Dean of Engineering,
George R. Brown School of Engineering, Rice University, Houston, TX 77005
(E-mail: sallie@rice.edu). This article is the text of her Presidential Address
delivered to the American Statistical Association in Seattle on August 8, 2006.
© 2007 American Statistical Association
Journal of the American Statistical Association
June 2007, Vol. 102, No. 478, Presidential Address
DOI 10.1198/016214507000000275
• “A profession advances its art through research.” We do
that.
• “It communicates its art through journals and meetings.”
These are critically important to the profession—things we
worry about vehemently: How do we continue to support
and sustain the field?
• “It educates.” We do that.
• “It defines its art to society.” We do that every day.
• “It advocates its art to society.” I would argue we need to
do more of that.
• “It serves society.” If we do not do this, we should all go
home and find new jobs.
• And, finally, “it serves its members.”
I believe we clearly have a profession. In fact, the early objectives of our society, the American Statistical Association
395
396
Journal of the American Statistical Association, June 2007
(ASA), as stated in the 1839 ASA Constitution, were to collect, preserve, and diffuse statistical information in the different
departments of human knowledge.
In 1909, North (1911) had a wonderful phrase in his talk. He
said, “. . . statistics is the guiding thread to lead us into and out
of the labyrinth mazes of social progress.” That is powerful, and
it is interesting to think about how we have walked that historical path and what the future holds for us. I am going to take
you down the path of what I have learned through my reading
of the past presidential addresses. For me, it has been a fascinating journey. As we begin, I want to make a disclaimer: I am not
a historian. I will also remind you that few, if any, of our previous presidents were historians. This will be a historical view
through the eyes of the leadership of our profession.
First of all, we are taught from the earliest days of our education in statistics that we are the stewards of data analysis and
data methodology development. By “data” I mean all sorts of
information, not just the little numbers and tables we use in our
early classes. We like to tell our students—and those students
in the audience will confirm this—we are guardians of the scientific method:
•
•
•
•
•
•
Observation
Hypothesis development
Prediction
Experimentation
Analysis
Decision.
You have heard this in many of the talks already at this conference. Sometimes we tend to forget our contributions to all
the parts of the method, particularly the decision piece. But
nonetheless, we view our field as the one that helps pull it all
together.
What did statisticians do 100 years ago, or at least from 1908
on? Interestingly enough, statisticians played critical roles in
the enumeration, recording, and reporting about society and the
population. The statisticians—and the American Statistical Association, in particular—drove policy concerning the world’s
population. Our field played a major role in the development
of economics and social sciences. In fact, our 1914 president,
Koren (1915), looked out at his audience and complained it was
filled with economists and social scientists. He wanted to know
where the statisticians were. Where were the people who were
going to be able to aggressively develop the methods the rest of
the people in the room needed? In this period, statisticians were
employed primarily in government and some in academia. In
fact, there was frustration stemming from the belief that American universities were not doing an adequate job of educating
statisticians.
There were some really significant changepoints that had an
impact on our profession in these early years. These were the
two world wars. I had not fully appreciated the importance of
these events on our profession until I took this historical walk.
World War I brought a tremendous number of statisticians into
all areas of government. A huge change occurred, because the
need was no longer just enumeration and reporting what had
happened; the interest was in developing ways to forecast the
new situations and address resource needs. There was a call
for formalization of inference and of our training in the universities. In World War II, again, there were a large number of
statisticians who helped with the war effort.
After both wars, there was no problem with the job market.
We are a field where demand has always exceeded supply. In
fact, one of my colleagues at Rice suggested I get a sandwich
board and wear it at freshman orientation to increase our number of majors.
Interestingly enough, even before World Wars I and II,
H. G. Wells (1903) commented, “[S]tatistical thinking will one
day be as necessary for efficient citizenship as the ability to read
and write.” We are seeing this necessity being realized today,
after more than two decades of the Quantitative Literacy Initiative. Following the wars, many statisticians emerged from positions in government. This workforce understood policy and they
understood the government. There was a cry for statisticians, as
citizens, to engage in policy in all aspects of their work. This
cry was largely ignored.
In the fifties and early sixties, there was great concern that we
were becoming too specialized. In fact, Cochran (1954) had a
wonderful comment in his presentation. He said, “The increasing specialization within statistics has set up forces which tend
to decrease the amount of common interest among members
and to split them into separate groups. The task of serving all
areas of application in this rapidly changing environment will
require us to be wide awake, adaptable, and receptive to new
ideas and new ventures.” As I tour our academic departments
and our nonacademic statistical groups around the country today, I suggest we begin to heed his advice and open our eyes.
Let me speed up a bit and ask the question, “How did we
train statisticians 45 years ago?” In the mid-sixties, statistics departments across the country were emerging from mathematics
departments and becoming separate units. Statistics, as a discipline distinct from mathematics, was coming into being in
our universities. This same phenomenon was occurring within
industrial research laboratories. For example, the Los Alamos
National Laboratory Statistical Sciences Group was formed in
1965.
Mathematics was then, and always will be, at the core of what
statisticians do. It is amusing to consider the amount of headbutting between the mathematical statisticians and the applied
statisticians documented in our history. Yet when you talk to
people, you are not always sure who’s who. In the early years,
even before the creation of many of our departments, there was
a concern that mathematics was becoming an art within statistics; the cart was in front of the horse and the horse was running
away (Ayres 1927). To be honest, when I saw that comment, I
was not sure who was the cart and who was the horse.
In the sixties, computation was very limited. We were doing a lot of manual calculation and our ability to address major
problems was limited by the available technology. Our research
and our journals reflected our training. There was a tremendous
amount of material going into the Annals of Statistics and going into the very theoretical work of JASA. We were working,
in academia and in government again. Statisticians were starting to find their way into industrial settings. Consulting, however, was still a bit of a mystery. Agriculture was a new application driver. Many of our departments grew out of the needs of
schools of agriculture. Genetics, epidemiology, and federal statistics continued to have a strong influence on the profession.
Keller-McNulty: From Data to Policy: Scientific Excellence
Once again there were some major developments that helped
change the face of our field. Computers emerged, and eventually high speed PCs. We were finally able to look seriously at
“real” data sets. We were able to begin to move into the realm
of computational experiments, not just physical experiments.
Ron Snee’s Deming Address earlier today discussed the quality movement, which had a major impact on our profession, not
the least of which was to reinforce the concepts of the scientific
method. The Internet emerged, leading to the start of globalization. It is interesting to consider the impact Tom Friedman’s
book, The World is Flat (2006), has had. Apparently we were
not listening carefully to Hogg (1989) in 1988. Perhaps we did
not understand the visionary he is. He told us, “. . . really, we
have one marketplace.” He said we needed to pay attention to
this because it calls for a different type of statistics and suggested it might be something automated. Are we listening yet?
We did make some very significant progress during that time.
We saw model building come on the horizon, computational
statistics, graphical statistics, and data mining. There was a
Bayesian revolution that helped us begin to envision data combination and the promise of information integration methods.
We saw a proliferation of publications with a lot of diversity. We
also saw statistical applications being developed in all phases of
industry and society.
Okay, that is the good news. What is the bad news? The bad
news is what we didn’t do. As a profession we did not aspire
to leadership roles. Some individuals did, but as a profession,
we did not. Janet Norwood (1990) in her presentation in 1989
very specifically said, “We must place at all heads of each of the
country’s major statistical agencies a person with professional
qualification and unquestioned integrity.” She talked about how
statistics has clearly come to play a role in critical policy decisions, but questioned whether statisticians played a role.
The problem then was the same as now: we were fat and
happy. We had great jobs. We had strong engagement across
science and engineering, and today, even social sciences. But
I wonder, are we actually stepping up and recognizing what
society needs today? The twenty-first century has brought us
new challenges and has brought us new crises. It has actually
brought us a lot of sadness.
What comes to mind when you think of 2001? My guess is
you remember September 11th. I attended a talk recently by a
respected colleague, Hector Ruiz, the CEO of Advanced Micro
Devices. He said we should remember 2001 as the year we finally sequenced the human genome. I am going to encourage
you to take that thought home with you tonight.
There are hard and important problems out there. To contribute, we need to step up and understand what our role is as
a profession. I am going to share with you the hardest lesson
I have had to learn, and I argue it is a new changepoint. The
lesson is, at the end of the day, it is not about modeling or data
analysis, it is about decision making.
I started out defining our field as the discipline responsible
for unraveling the mysteries of decision making in the presence
of uncertainty. If you really try to absorb that message and internalize it, you have to recognize that it means there is diversity
of information and a diversity of players that come together in
this context. Our failure to understand this nuance means we
won’t be able to contribute to the significant decisions, strategic decisions, and science decisions that confront society. Our
397
profession will miss the opportunity to make significant contributions.
I really do believe science is about pushing society forward.
The other thing I strongly believe is that science and policy continually collide. One of my favorite examples of this is a 1939
letter Einstein wrote to President Roosevelt. In this letter he told
the President about a new energy source: uranium. That piece of
information—science—affected policy and changed the world
forever.
Based on that example and many more I could provide, we
must realize we have to expand our thinking beyond the scientific method into what I call the scientific process. Otherwise,
we are not going to be able to effectively contribute to the solution of problems we are well poised to help solve, supporting
decision making in the presence of uncertainty. We have to be
cautious because decisions will be political and will be politicized, and they are in the hands of the policy makers as strategic
decision makers. That is not necessarily us.
Our job as scientists, as statisticians, as engineers, is to be
sure we are presenting the information these policy makers and
strategic thinkers need, the information that can impact and can
help them in their deliberations. Likert (1960) in 1959 told us,
“We are coming to recognize with increasing clarity that the
capacity of a nation to function will depend upon the quality
of its decision-making processes and upon the adequacy and
accuracy of the information used.” This is our problem.
In 1990, Barabba (1991), from General Motors, gave a talk titled “From Data to Wisdom.” He made a wonderful statement I
wish I had remembered, because I know it would have impacted
me greatly. I have had to come to understand this on my own.
I am going to share his wisdom with you and ask you to heed
his remarks. Barabba said, “The successful statisticians of the
future must not only view their roles as bringing information to
bear which reduces uncertainty relative to a particular decision
at hand, but to also be sensitive to the manner in which they surface new areas of uncertainty which could materially affect the
outcome of a particular decision.” He does not say we should
not surface this information. He says we need to be cognizant
of the effect it will have, which means we have to understand
how to communicate it and how to do this in a productive way.
The problems I have been engaged with over the last
decade include natural disasters, terrorism or man-made crises,
weapons of mass destruction, and technology development in
developing countries. These problems are big, difficult, and frequently scary. The progress made has had everything to do with
people—people collaborating toward a common goal. I have
learned that the scientific progress we can make today comes
through assembling multiple talents and getting them to work
together.
Problems today will be solved by interdisciplinary teams.
Why? Because the depth and breadth in any single field has
grown to the point that you cannot know everything in your
own field, let alone know the depth needed in the partner fields
to make rapid progress. Therefore, we must learn to communicate with our colleagues and pull teams together to make this
progress.
I have recently heard leaders in different industrial sectors
talking about the need for “T-people.” T-people? Is this a new
fitness craze? What are they talking about? They are talking
398
about people who have deep knowledge in their fields, yet great
curiosity across many other fields, hence the top of the T, and
terrific communication skills to bring it together. Some of these
same leaders say this is the time of the Renaissance team, not
the Renaissance man or woman. Most importantly, they say,
innovation will emerge in the gray spaces between disciplines.
In statistics, our job is to know the cutting edge of statistical
sciences and to bring this knowledge to our scientific colleagues
in such a way that we can help move forward the problem–
solution spaces.
I agree, this is daunting. T-people, Renaissance teams, innovations: What does it mean for us? Statistics is the quintessential interdisciplinary science. I like to tell my junior colleagues
sometimes, “Don’t worry. When we collaborate, no one expects
us to know anything about chemistry, biology, or physics, even
though we probably know quite a bit.” We are viewed as the
neutral party. We are the ones who get to ask all the questions
everyone else at the table wants to know the answers to. We are
the ones who have a systematic way of thinking, are able to take
a systems view. That is how we are trained. We are the ones who
are willing to take a risk, willing to jump in and try to figure out
how to characterize the problem space and move toward solutions, sometimes swimming out from the middle. Yes, we will
make mistakes, but tomorrow we are confident someone will
have an even better solution. All along the way we will be the
ones able to quantify and describe the uncertainty. The moral of
this message is this: We are the ones who can actually step up
and take a leadership role in moving forward the integration of
statistics and science into decisions.
In addition to various disciplines required to tackle today’s
problems, there are also a large number of skills needed. We
need deep theory; we need computation; we need people who
can visualize and synthesize the big picture. We need people
who are meticulously detail oriented. We need terrific communication skills from the 50,000-foot view all the way down to
the very detailed interactions with our scientific colleagues. We
need people who are willing to take risks and jump into the
middle of problems, and we need those who are going to yank
us back at the ankles and force us to think about first principles.
We need people who have great organizational skills and can
manage the complexity of the problem setting.
It is unlikely anyone in this room is good at all those things,
and it is even more unlikely anyone in this room likes to do all
those things. But by recognizing what skills need to come to the
table, you will know how to assemble your team to be sure those
skills and areas are covered. Hopefully, you will also recognize
that everyone’s contributions are important to the success of the
team and aggressively share the credit.
I have talked a lot about complex problems, global challenges, and the need to bring interdisciplinary teams together.
When it comes to defense and national security, we have many
pressing problems. I think we really need to pay more attention
to what Savage (1985) told us in 1984. He was not personally
interested in the military secrets; however, he did want to know
that the defense community could connect to the best of our
field. He said something else that is not only true for defense
and security. He said, “. . . [T]he first lesson the statistician has
to offer to the strategist is the concept of uncertainty and the
measurement of variability. Getting these ideas into strategic
Journal of the American Statistical Association, June 2007
decisions will take skill and hard work, but they will add much
realism.” I don’t think you can point to a single walk of life, a
single area for which that is not true. We have strategic decision
makers in every part of society.
I want to say just a couple of words about technology. Yesterday, the presidential speaker, William Pulleyblank, talked to
us about high performance computing, supercomputing, giant
computers, all this wonderful technology. For me, the question is, “What role do we have in all of this as statisticians
and as a profession?” This might appear hard to answer at first.
Technology, by design, comes in search of questions. Technology frequently leads to expectations for the uninformed. Imagine the supercomputers Pulleyblank was talking about: 60,000
processors, massively parallel, operating at incredible speeds
of 300 teraflops. To policymakers, such technology holds the
promise of solving significant problems. For example, maybe
those computers would allow us to assess the reliability of the
nuclear stockpile by actually cutting back on tests because we
can develop incredibly sophisticated physics codes, run them,
and perhaps get answers out we actually believe. Those of us
who guard the scientific method may have a few things to say
about this. Forecasting, prediction, and uncertainty quantification have got to be conditioned on experimentation and data.
Our job is to be able to be articulate, and push back and try to
bring realistic expectations to the role of technology. However,
we crave the technology because it is what helps to drive us forward. Consequently, we have to be responsible and manage the
expectations of technology innovation.
A current area I worry about a lot is nanotechnology. Many
people say if we just develop the nanowire, it will be the mechanism by which we can have efficient energy and can solve some
of the energy demands of the world. I believe this is actually
likely. But when? In 10 years or 40? It is the data; it is the uncertainty about the information that needs to come together and
be evaluated to understand what can be expected from these
technologies.
David Moore (1998) told us all about this in his address. As
I said when I started, there is nothing new in what I am telling
you. It has been fun to learn (or to remember) my colleagues
were thinking really deeply about these issues. David told us
“. . . technology empowers, but thinking enables.”
Where does that leave us today? Statistics today is all about
interdisciplinary collaboration. We need to bring statistical reasoning and rigor into a decision context. Ron Snee talked to
us today about statistical thinking, something he has been promoting for more than 20 years. Let’s pay attention to it. Let’s
bring it to the table. To bring it to the table means we have to be
at the table. The context we are working in is interdisciplinary
and complex. We have a role in handling the consequences of
the technology explosion that empowers us: it brings to us new
data types we have never had before and challenges us with
those problems. It causes us to worry about how we are going
to put it together. But we do have to play a role in managing
the expectations. We have to worry about how to integrate theoretical models, computational models, physical experiments,
observational data, historical data, and expert judgment. How
do we put these things together?
We are in the information millennium. We are in the decade
of data revolution. This is the heyday for our field. Who will
Keller-McNulty: From Data to Policy: Scientific Excellence
actually solve these problems? We will. We can lead this revolution and help propel science and technology forward.
Do we have a future? Does our profession have a future?
I think what I have just described to you is clearly not just a
job. I think our future is to really think about what leadership is
and try to become leaders in the science integration process. We
have to understand how policy and science can collide if we are
going to be productive. Most importantly, we have to recognize
communication is key: not only is it part of the process, it is a
process in its own right.
We must step out and take leadership roles, and encourage
our colleagues to do the same. We don’t need everyone to do
this, but we need some of our community to step up. We need
representation at the levels of president, vice president, and
chief executive officers in our major industries. We need people,
as Janet Norwood (1990) said, in the leadership of not just the
statistical agencies, but our federal agencies too. We need people in leadership roles in our academic institutions: presidents,
provosts, and deans. We need to realize this is important for
our profession and it is important for society to have statistical
thinking and rigor at those tables.
I say if we want to achieve scientific excellence, it is our time,
it is our decade, it is our century to step up and do it. We need
to become the stewards of the science integration process. We
have already mastered being stewards of the data analysis and
data methodology development processes. We have completely
succeeded in guarding the scientific method. Let’s step out and
see if we can be the guardians of the scientific process.
We are not going to succeed alone. We are only going to
succeed if we recognize our profession plays a critical role.
399
The American Statistical Association, which is our society, is
the largest and most powerful statistical society in the world.
This means the leadership of this society and the leaders among
our membership are exactly who will propel our profession forward. This is your society. Help shape it.
Thank you for your attention and support. Good night.
[Received February 2007.]
REFERENCES
Ayres, L. P. (1927), “The Dilemma of the New Statistics,” Journal of the American Statistical Association, 22, 1–8.
Barabba, V. P. (1991), “Through a Glass Less Darkly,” Journal of the American
Statistical Association, 86, 1–8.
Cochran, W. G. (1954), “The Present Structure of the Association,” Journal of
the American Statistical Association, 49, 1–12.
Friedman, T. L. (2006), The World Is Flat, New York: St. Martins Press.
Hogg, R. V. (1989), “How to Hope With Statistics,” Journal of the American
Statistical Association, 84, 1–5.
Hunter, S. J. (1994), “Statistics as a Profession,” Journal of the American Statistical Association, 89, 1–6.
Koren, J. (1915), “Some Statistical Ideals,” Journal of the American Statistical
Association, 14, 351–357.
Likert, R. (1960), “The Dual Function of Statistics,” Journal of the American
Statistical Association, 55, 1–7.
Moore, D. S. (1998), “Statistics Among the Liberal Arts,” Journal of the American Statistical Association, 93, 1253–1259.
North, S. N. D. (1911), “The Relation of Statistics to Economics,” Journal of
the American Statistical Association, 12, 431–443.
Norwood, J. L. (1990), “Statistics for Public Policy: Reflections of a Changing
World,” Journal of the American Statistical Association, 85, 1–5.
Savage, R. I. (1985), “Hard/Soft Problems,” Journal of the American Statistical
Association, 80, 1–7.
Wells, H. G. (1903), Mankind in the Making, London: Chapman and Hall.