Reconsidering the Fundamental Contributions of Fisher and Neyman on Experimentation and
Sampling
Author(s): Stephen E. Fienberg and Judith M. Tanur
Source: International Statistical Review / Revue Internationale de Statistique, Vol. 64, No. 3
(Dec., 1996), pp. 237-253
Published by: International Statistical Institute (ISI)
Stable URL: http://www.jstor.org/stable/1403784
Accessed: 19/10/2008 12:47
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=isi.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the
scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that
promotes the discovery and use of these resources. For more information about JSTOR, please contact support@jstor.org.
International Statistical Institute (ISI) is collaborating with JSTOR to digitize, preserve and extend access to
International Statistical Review / Revue Internationale de Statistique.
http://www.jstor.org
InternationalStatistical Review (3), 1966, 64, 237-253, Printedin Mexico
? InternationalStatisticalInstitute
the
Fundamental Contributions
Reconsidering
on
and
of Fisher
Experimentation
Neyman
and
Sampling
Stephen E. Fienberg1 and Judith M. Tanur2
1Departmentof Statistics, CarnegieMellon University,Pittsburgh,PA 15213-3890, USA
2Departmentof Sociology, State University of New Yorkat StonyBrook,Stony Brook,NY
11794-4356, USA
Summary
R.A. Fisher and Jerzy Neyman are commonly acknowledged as the statisticians who provided the
basic ideas that underpin the design of experiments and the design of sample surveys, respectively. In
this paper, we reconsider the key contributions of these great men to the two areas of research. We also
explain how the controversy surrounding Neyman's 1935 paper on agricultural experimentation in effect
led to a split in research on experiments and on sample surveys.
Stratification.
Designof samplesurveys;Randomization;
Designof experiments;
Key words:Clustering;
1 Introduction
The 1920s and 1930s marka criticalperiodin the developmentof statistics.Much of the modern
theory of estimationand statisticaltesting emergedfrom paperspublishedduringthis period by two
of the most importantstatisticiansof the century,RonaldAylmerFisherandJerzyNeyman. In many
ways, the theoreticalwork of these two greatmen was stimulatedby practicalproblemsthey encountered in their day-to-day statisticalwork of agriculturalexperimentationand, later for Neyman, the
sampling of humanpopulations.Moreovermany believe that their greatestaccomplishmentswere
not in the realmof theoriesof inferencebutratherin theirarticulationof theoryand principlesunderpinning the two key modernmethods for the scientific collection of data-randomized experiments
and randomsampling.
In connection with a largerprojecton experimentsand sample surveys, and in partstimulatedby
the recent occasion of the 100th anniversaryof Neyman's birth,we have rereadmuch of the work
of both Fisher and Neyman in the areasof samplingand experimentation.Ourstory for both Fisher
and Neyman begins with the work thatled to their 1923 paperson agriculturalexperimentation.For
Neyman, this workcontinuedlargelyin the directionof samplingand culminatedin his now famous
1934 paper on the topic. Despite the many contributionsof others to the technical developmentof
sampling in the period from 1900 to 1925, Neyman's 1934 paper played a pivotal role in turning
the dry mathematicsof expectations into real sampling plans for actual randomly selected large
scale surveys. For Fisher,his 1923 paperled to furtherpioneeringwork in experimentaldesign that
culminatedin the publicationof his 1935 book, The Design of Experiments.Fisher's book played
a catalytic role in the actual use of randomizationin controlled experimentsthat is similar to the
role Neyman's 1934 paperplayed in the use of methodsfor randomsampling.The year 1935 was a
238
and J.M. TANUR
S.E. FIENBERG
criticalone in the developmentsin the fields of experimentationand sampling,not simply because of
the publicationof Fisher'sbook but also becauseof a majorcontroversybetweenFisherandNeyman
engenderedby Neyman's (1935) paper on agriculturalexperimentation.Because of the bitterness
that grew out of this dispute (and a related one between Fisher on the one hand and Neyman and
Pearson on the other,over tests of hypotheses and then later over confidenceintervals),Fisher and
Neyman were never able to bring their ideas togetherand benefit from the fruitfulinteractionthat
would likely have occurredhad they done so. And in the aftermath,Neyman stakedout intellectual
responsibilityfor samplingwhile Fisherdid the same for experimentation.It was in partbecause of
this rift between Fisher and Neyman that the fields of sample surveys and experimentationdrifted
apart.
In the next section, we begin with Neyman'simportant1923 paperon agriculturalexperimentation,
and we remind the reader of the interrelatednature of fundamentalideas on experiments and
surveys and the pivotal role that randomizationplayed in both areas. Then our narrativeproceeds
by interweaving accounts of the work by Fisher and Neyman on experimentationand sampling,
and controversiesthat surroundedthem. Our accountculminateswith the clash between Fisher and
Neyman over ideas in Neyman's 1935 paper.Throughoutwe include relevantbiographicalmaterial
assembledfrom a varietyof sources includingBox (1978, 1980), Lehmann& Reid (1982) and Reid
(1982).
2
Parallels between Surveys and Experiments: Innovations in Neyman's 1923 Work on Agricultural Experimentation
JerzyNeyman was bornof Polish parentsin 1894, in Bendery,which has been variouslylabeledas
Rumania,Ukraine,and Moldavia because of the vicissitudes of border-drawingin EasternEurope.
In 1912 he enteredthe Universityof Kharkov(which laterbecameMaximGorkiUniversity)to study
mathematics.Finishing his undergraduatestudies in 1917, Neyman remainedat the University of
Kharkovto begin to preparefor an academiccareer;he also received an appointmentas a lecturer
at the KharkovInstituteof Technology.In the fall of 1920 he passed the examinationfor a Masters
degree and became a lecturerat the University.Thusuntilafterhis 27th birthdayNeymanwas doubly
isolated-both by living in a provincial city and by being part of an ethnic minority in that city.
In the spring of 1921, learning that he was to be arrested,Neyman fled to the country home of a
relative,where he supportedhimself by teachingthe childrenof peasants.In the summerhe returned
to Kharkovand thence to Bydgoszcz in northernPoland,as partof an exchangeof nationalsbetween
Russia and Poland agreed to at the end of the Russian-Polish War.In Bydgoszcz, Neyman went
to work as "senior statistical assistant"at the National AgriculturalInstituteand it was there that
he wrote two long papers on agriculturalexperimentationthat were published in 1923 in Polish
(Splawa-Neyman, 1990 [1923a], 1925 [1923b]). In this section we focus primarilyon the first of
these papers,and we returnto the second in Section 4, discussing its 1925 republicationin English.
An excerpt of the 1923 paper(Splawa-Neyman, 1990 [1923a]) on experimentationwas recently
translatedfrom the Polish originaland publishedin StatisticalScience, and in it we were especially
struckby the importancethatrepeatedrandomsamplingplayed in Neyman's thinking.This seemed
to us to foreshadow,at the very least, the use of randomizationin experimentation.Reid (1982, p.
44) quotes Neyman considerablylateras denying his priorityhere:
". .. I treatedtheoreticallyan unrestrictedlyrandomizedagriculturalexperimentandthe
randomizationwas consideredas a prerequisiteto probabilistictreatmentof the results.
This is not the same as the recognitionthat without randomizationan experimenthas
little value irrespectiveof the subsequenttreatment.The latterpoint is due to Fisherand
I consider it as one of the most valuableof Fisher'sachievements."
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 239
Since we see one of the majorpurposesof experimentalrandomizationas the necessary precondition for probabilisticinferencefrom the results, we wouldjoin Rubin (1990, p. 477) in saying that
had Neyman later claimed priorityratherthan denying it, we would have had no reason to quarrel
with that claim. Rubin (1990) also remindsus that the use of randomizationwas "in the air"in the
early 1920s, citing Student(1923, pp. 281-282) and Fisher & MacKenzie(1923, p. 473).
There is, however, anotherimportantfeatureof Neyman's first paperon the topic of agricultural
experimentationwe think especially worthy of note. For a numberof years, we have pursuedthe
parallels and linkages between surveys and experiments(e.g., see Fienberg & Tanur,1987, 1988,
1989) and, in particular,Fisher's and Neyman's roles as progenitorsof ideas in both areas.Although
in fact surveys and experimentshad developed very long and independenttraditionsby the startof
the 20th century, it was only with the rise of ideas associated with mathematicalstatistics in the
1920s thatthe tools for majorprogressin these areasbecame available.The key intellectualidea was
the role of randomizationor randomselection, both in experimentationand in sampling, and both
Neyman and Fisherutilized this idea, althoughin differentways. But then, as now, it is clear thatone
can adaptthe ideas from one field to the otherand that one can profitablylink them, as in sampling
embedded within experimentsor experimentsembeddedwithin surveys.
Neyman's first 1923 paperon experimentaldesign embodies this sort of adaptation.He conceptualizes the assignmentof treatmentsto units in an experimentas the drawingwithoutreplacementof
balls from urns, one urnfor each treatment.These urnshave the special propertythatthe removalof
a ball (representingthe outcome of an experimentalunit) from one urn causes it to disappearfrom
the otherurnsas well. Thus Neyman shows thatwhen thereis a finite pool of experimentalunits that
need to be assigned to treatments,the randomassignmentof units to treatmentsis exactly parallel
to the randomselection of a sample from a finite population.Hence, when the numberof units used
in an experimentis a large fractionof the units in the population,a finite populationcorrectionmust
be used in an experiment,just as it is in a sample survey. The parallel between experimentsand
sampling is particularlyclose in this case. Nevertheless,all but a few moderninvestigatorshave lost
sight of the paralleland fail to take advantageof insights offered in the parallelliterature.
3
Fisher's Initial Contributions to the Design of Experiments in the 1920s
While it is only recently that statisticiansand othershave rediscoveredNeyman's early contributions to the design of experiments,Fisher has long been recognized for the innovationhe broughtto
the field.
Fisherwas bornin 1890 in Londonandit was duringhis undergraduate
yearsat Cambridgethathis
interestsin eugenics, genetics, and biometrystimulatedhis interestin probabilityand statistics.After
graduationin 1913, he taughtmathematicsand physics at a series of schools but pursuedresearchin
both genetics and statistics and publishedhis first majorpaperson these topics. This work led, six
years later,to a temporarystatisticalposition in 1919 at RothamstedExperimentStation,where the
director,Sir John Russell, wantedsomeone who would be preparedto examine the accumulateddata
on wheat yields from BroadbalkFields in orderto elicit furtherinformationthat might have been
missed. Russell quickly recognized Fisher's genius and set aboutconvertingthe temporaryposition
to a more permanentone (see Box, 1978, pp. 96-97). In 1921, Fisherpublishedthe firstof a series of
papersentitled"Studiesin crop variation"intendedfor the JournalofAgriculturalScience', in which
he presentedsome of his reanalysesof the Broadbalkdata. But it was in the second in this series of
papers,writtenwith W.A. Mackenzie(1923), thatFisher'snew ideas on agriculturalexperimentation
emerged.
Fisher & Mackenzie (1923) contains two key innovations for the design of experiments: the
'Actually the third paper in the series appearednot in the Journal of AgriculturalScience but in the Philosophical
Transactionsof the Royal Society, and without the title, "Studiesin crop variation".Two later paperswith Thomas Eden in
1927 and 1929 are labelled as V and VI.
240
S.E. FIENBERGand J.M. TANUR
introductionof the analysisof variance,adaptedfromFisher's 1918 paperon Mendelianinheritance,
and randomization.In fact, in introducingthe analysis of variance, the authors argue that it is
conditionalon randomization:
"Further,if all the plots are undifferentiated,as if the numbershad been mixed up and
writtendown in randomorder,the averagevalue of each of the two partsis proportional
to the numberof degrees of freedomin the variationof which it is compared."(p. 315)
The spirit of this statementis not unlike that in Splawa-Neyman (1990 [1923a]): "The goal of a
field experimentwhich consists of the comparisonof n varietieswill be regardedas equivalentto the
problemof comparingthe numbersal, a2, . . ., an or theirestimatesby way of drawingseveralballs
from an urn."(p. 467) Both speak of randomization,but in hypotheticalterms.There are dramatic
differences between Fisher's and Neyman's treatmentsas well, with Fisher focusing on the actual
dataanalysisandNeymanputtingmuchemphasison the mathematicalformulationof finitesampling
and expectations.
Neither Fisher nor Neyman actually wrote about the implementationof randomizationin 1923,
however.In fact, as Cochran(1980, p. 18) notes in connectionwith the 2 x 12 x 3 factorialexperiment
describedby Fisher & Mackenzie(1923):
"No randomizationwas used in this layout. Following the procedurerecommendedat
thattime, the layoutapparentlyattemptsto minimizethe errorsof thedifferencesbetween
treatmentmeans by using a chessboardarrangementthat places different treatments
near one another as far as is feasible. This arrangementutilizes the discovery from
uniformitytrialsthatplots nearone anotherin a field tend to give closely similaryields.
A consequenceis, of course, thatthe analysis of varianceestimateof the errorvariance
perplot, which is derivedfromdifferencesin yield perplots receivingthe sametreatment,
will tend to overestimate,since plots treatedalike are fartherapartthanplots receiving
differenttreatments.Fisherdoes not commenton the absenceof randomizationor on the
chessboarddesign. Apparentlyin 1923 he had not begun to think about the conditions
necessaryfor an experimentto supply an unbiasedestimateof error."
A furtherproblem with Fisher's 1923 analysis was his failure to recognize the split-plot structure
of the actual experimentwith respect to a thirdfactor,potassium.This he correctedin Statistical
Methodsfor Research Workers(1925).
Thus, in 1923, we see both Fisherand Neyman as having arrivedat similarpositions with respect
to randomization,althoughFisher alreadyhad moved aheadin his thinkingabout otherfeaturesof
design. Box (1980) argues, that the statementin Fisher & Mackenzie on the analysis of variance
rested heavily on Fisher's appreciationof the requirementsof underlying theory for the normal
distributionupon which hereans,
reld in his analysis,as well as upon Fisher's"geometricrepresentation
that by then was second natureto him. He could picture the distributionof n results as a pattern
in n-dimensional space, and he could see that randomizationwould produce a symmetry in that
patternratherlike thatproducedby a kaleidoscope,and which approximatedthe requiredspherical
symmetryavailable,in particular,from standardnormaltheoryassumptions"(p. 2). While we agree
thatFisher's geometricalintuitionwas highly refined,we see little supportin the 1923 paperfor this
conclusion.
But by the publicationof StatisticalMethodsin 1925, Fisherclearly had a deeperappreciationof
the function of randomizationin experimentation:
"Thefirstrequirementwhich governsall well-plannedexperimentsis thattheexperiment
should yield not only a comparisonof differentmanures,treatments,varieties,etc., but
also a means of testing the significance of such differences as are observed .... For
our test of significance to be valid the differences in fertility between plots chosen as
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 241
parallels must be truly representativeof the differences between plots with different
treatment[s];and we cannotassume thatthis is the case if our plots have been chosen in
any way accordingto a prearrangedsystem."(p. 248)
According to Fisher,valid tests resultif treatmentsare assigned to plots wholly at random,and here
we have a clear departurefrom the 1923 papers.
The treatmentof experimentationin StatisticalMethods,which follows this discussion of randomization, consists of only a few pages. Fisherintroducedblocking as a device to increasethe accuracy
of treatmentcomparisons and he illustratedits use in both randomizedblock experiments and in
Latin squares. He described the split-plot structureof the experimentanalyzed in the 1923 paper,
which should have requiredtwo levels of randomizationfor its validity, but Fisher did not discuss
this point in the original or subsequenteditions of the book.
Later embellishmentsto the theory of experimentaldesign came in Fisher (1926) where he laid
out in a more systematicfashion the principlesunderlyingfield experimentation,including the need
for replication,and in which he expoundedupon the notion of factorialdesigns with new insights,
including the role of confounding.The example used in this paperwas from a real experimentbeing
runby a colleague, ThomasEden, and Eden & Fisher(1927), presentsa more detaileddescriptionof
the experimentand an analysis of its results.Factorialstructures,Fisher(1952) would later observe,
had their roots in the "simultaneousinheritanceof Mendelianfactors".
Throughoutthis period, Fisher continuedto work with colleagues at Rothamstedand elsewhere
in the design and analysis of experimentsthatimplementedhis ideas on randomizationand factorial
structures.But not all of his Rothamstedcolleagues had fully accepted Fisher's insistence on randomization as the foundationof experimentatiion
(Box, 1980). In fact, Fisher's 1926 article was at
least in part a response to an articleby Sir John Russell, directorof Rothamsted,on the state of the
artin experimentaldesign. In his paper,Russell had noted Fisher's argumentfor randomizationbut
said that it was impossible to use in practice!Despite this disagreement,Russell allowed Fisher and
Eden to runtheircomplex randomizeddesigns on fields at Rothamstedand this ultimatelyconvinced
many others of the value of Fisher's innovativeideas (Box, 1980).
4
Neyman's Initial Contributions to the Theory of Sampling in the 1920s
In 1924, Neyman obtainedhis doctorsdegree from the Universityof Warsaw,using as a thesis the
work done in Bydgoszcz, and in 1925, he obtaineda governmentgrantto study for a year in London
with KarlPearson.Before his arrivalin London,Neyman had shippedto KarlPearsonseveral of his
statisticalpublications,includingthe two 1923 paperson agriculturalexperimentation,and Pearson
suggestedthatNeymanrepublishpartof the secondpaperin Englishin Biometrika(Splawa-Neyman,
1925 [1923b]). But Pearsonbelieved thatNeyman'sstatementat the end of the paper,thatit is only in
samplingfrom a normalpopulationthatthe samplemean and samplevarianceare independent,to be
mistaken.When Neyman tried to explain to Pearsonhis confusion between independenceand lack
of correlation(in haltingEnglish and in frontof severalotherPearsonstudents),Pearsoninterrupted:
"Thatmay be true in Poland, Mr. Neyman, but it is not true here."(Reid, 1982, p. 57) Dismayed
at having offended Pearson and at perhapshaving lost a chance to publish in English-his Polish
mentorshad sent Neyman to Englandas a kind of test to see if his ideas were worth anything,and
a publicationin English would go a long way towardssettling that matter-Neyman searchedfor a
way to communicatehis explanationto Pearson.He finallyofferedhis explanationto J.O. Irwin,who
communicatedit to Egon Pearson,who finally convinced his fatherthatNeyman was not mistaken.
Thus the Biometrikaversion does contain this observationfrom the Polish version.
In the resulting 1925 Biometrikapaper, Neyman gave the higher moments of the means and
variancesof samples from finite populations.The paperis succinct and to the point. He begins by
deriving formulas for moments of the sampling distributionof the mean of samples of size n from
242
and J.M. TANUR
S.E. FIENBERG
a finite population of size m, where the sample members are drawn without replacement,and he
relates these to the moments of the underlyingfinite population.Then he focuses on the variance,
skewness, and kurtosis, of the sampling distribution,i.e., M2, B1 = M/M23, and B2 = M 2/M2,
where M2, M3, and M4 are the centralmomentsof the samplingdistribution,and he expresses these
in terms of /,2, P1 = ,LlJ/,t3, and P2 = 4/ 1A2 where u2>,/3, and 14 are the centralmoments of the
finite populationof m elements. He explains how the momentsof the samplingdistributiontend to
the usual ones for sampling with replacementwhen we let m - 0oo.He then turnsto the first and
second moments of the sample variance,expressingthem in terms kL2 and B2, and to the correlation
between the squareof the samplemean andthe samplevarianceas well as to the correlationbetween
the sample mean and the sample variance,expressingthe firstin termsof P2 and the second in terms
of P31and B2. In the final paragraphof the paper, by letting the finite population size m -+ 0o,
Neyman uses the correlationformulasto argue that the independenceof the sample mean and the
sample varianceholds only for normaldistributions,the point originallyin dispute with Pearson.
The 1925 papermight have passed unnoticed,butfor the fact that,in 1927, MajorGreenwoodand
Leon Isserlis publisheda complaintaboutit in the Journalof the Royal StatisticalSociety, pointing
out that Neyman had failed to acknowledgethe publishedpapersof the recently deceased Russian
statistician,AlexanderAlexandrovitchTchouproff2.
Greenwood & Isserlis (1927, p. 348) quote Neyman as writing of the formula for the second
moment of the variance"[t]hisresult, being a generalizationof formulaegiven by otherauthors,is,
I believe, novel and of considerableimportance".Indeed, after that statementon page 477 of his
1925 paper,Neyman did actuallycite one of Tchouproff's1918 Biometrikapapersas well as a 1925
paper by Church.Greenwoodand Isserlis neverthelesstake him to task for failing to cite works by
Karl Pearson,Isserlis, and Edgeworthbesides those of Tchouproff.In their view, Neyman's felony
was compoundedby the work of A.E.R. Church,especially a 1926 paperpublishedin Biometrika,
which cited Neyman for some of the formulasfor momentsratherthan citing Tchouproff.There is
little question but that Neyman's results can be found at least in some form in the sea of formulas
in Tchouproff's 1923 papers. What appearsto have been happeningin these and other papers by
Tchouproffis the alternativealgebraicderivationandexpressionof a varietyof momentexpressions,
most of which were quitecomplex. One of the nice featuresof Neyman(1925) is the relativelyclean
derivationsand succinct formulas.
By the time the Greenwood-Isserliscritiquewas published,Neyman had returnedto Poland. In
obvious distress at the accusation,he wrote to Egon Pearsonsoliciting advice. The reply, it turned
out, was handledby KarlPearson(1927), who editorializedin Biometrikain Neyman's defense (as
well as Church's). He arguedthat Neyman's original publicationin Polish in 1923 was certainly
contemporaneouswith Tchouproff'spair of 1923 articlesin Metron,and perhapsactuallypredated
those Tchouproffpublicationsbecause delays in the publicationof Metroncaused it to appearlater
than its cover date. But it seems to us that this argumentfrom Pearson,while it may well be true, is
irrelevantto the controversy-Greenwood and Isserlis were accusingNeyman of ignoringnot these
specific papers of Tchouproffbut a whole body of literaturethat originatedsome 20 years earlier
with work by Pearson himself. PerhapsPearsonwas in some sense defending himself as editor of
Biometrikafor his laxness in failing to urgeNeyman, the young foreigner,to updatehis work for an
English-readingaudiencewith citationsto the literaturein English.ThatNeymancould haveprofited
by such urgingseems clear from anotherfacet of Pearson'seditorialdefense-he cites a letterfrom
Dr. K. Bessalik, Professor of the University of Warsaw,who in 1922 was Director of the State
Instituteof AgriculturalResearchin Bydgozcz. In that letter,Bessalik certifies that Neyman wrote
his paperin Bydgozcz in 1922 and that "no English Journals"were accessible to him. Had Pearson
urgedNeyman to place the republicationof his results in the historicalcontext about which he had
2This was the transliterationof Tchouproff'sname, as it appearedin the 1918 Biometrikaarticles. We use the spelling
Tchouproffthroughout,even though when he publishedin Metron,the editor used Tschuprow,and elsewhere he is referred
to as Chuprovand Chouprow,but we referto specific papersusing the spelling used in connectionwith them.
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 243
presumablybeen ignorantduringthe time he was writing the original paper in Poland, we believe
thatNeyman would undoubtedlyhave agreed.He had, afterall, come to London with the expressed
purpose of studying with Pearson, whose Grammarof Science had served as an inspirationsince
Neyman discovered it in 1916.
Since Neyman's 1923/1925 paper did not representa conceptual breakthroughin the theory of
sampling from finite populations, we are left with two questions of historical interest. The first is
to ask how the isolated scholar arrivedat what may well be an independentderivationof some key
results in finite sampling and the second is to ask why Neyman's results, in particular,seem to have
had a more profoundinfluence than did similarones obtainedby others.
Perhapsthe answerto the firstquestionis that,while surelyout of the mainstreamof earlytwentieth
centurystatistics, neitherin Russia nor in Polandwas Neyman completely isolated. Neyman studied
probabilitytheory in Kharkovunder the direction of the Russian mathematicianS.N. Bernstein as
early as 1915 or 1916. Although these activities had been forgottenby Neyman by the time he was
describing his early career to Constance Reid in 1978 (and perhapsthis forgetfulness in his later
life contributesto our image of Neyman's early years as totally isolated), in the mid 1920s Neyman
describedhimself as havingcontinuedhis studiesunderBernsteinthrough1921. He also notes thathe
worked in 1920 underBernsteinto apply the theory of probabilityto experimentationin agriculture
(Reid, 1982, p. 30). ThroughBernstein,Neyman became familiarwith Markov'swork, which also
influencedTchouproff(see also Seneta, 1982, 1985). We know that,before he left Poland,Neyman
workedwith a ProfessorIsseroff on statisticalanalysis of agriculturalexperimentsand even lectured
to the AgriculturalDepartmentat the Universityof Kharkovon the applicationof probabilitytheory
to experimentalproblemsin agriculture.These activitieswere clearlyprecursorsto the papersdealing
with experimentaldesign in agricultureand his paper on finite sampling theory which appearedin
Biometrikain 1925.
Just as Fisher was stimulatedby the challenges of real experimentationat Rothamstedto gather
together ideas that were "in the air" to make his masterly synthesis of the design of experiments
and the analysis of variance, so Neyman could well have been stimulated by the challenges of
experimentationat the Institutein Poland to synthesize ideas of sampling from finite populations
that were "in the air".And that would lead rathernaturally,given the variablecitation practices at
the time, to the kind of papersNeyman wrote in 1923, papersthathad little referenceto the work of
others. The lack of referencesto work in English is particularlyunderstandable,since as his advisor
certifiedto Pearson,Neyman had no access to Englishjournalswhile he was in Poland.
To answer the second question, that of Neyman's profoundinfluence, we need to look not at the
1923 paper (nor at its 1925 republication),but at Neyman's watershed1934 paper,in which he was
able to capturethe essential ingredientsof the problemof sampling,synthesizehis own contributions
and those of others, and effectively demolish the idea of purposivesampling.
Neyman arrivedin London in 1925, shortly after the publicationof Fisher's Statistical Methods
for Research Workers.But the pathsof the two seem not to have crossed duringthe entire academic
year. In March of 1926, Gosset passed throughLondon and met with Neyman, who expressed an
interest in visiting Fisher at Rothamsted.Gosset wrote to Fisher warninghim that Neyman "holds
a letter from me to you asking you to show him anythingthat you think might be useful to him. I
daresay offprints would be useful to him if you could spare them as he finds it hard to trace your
work. I gatherthathe will write to you in April".FisherandNeyman actuallymet for the firsttime in
July in Rothamsted,althoughthereis no recordof what they discussed. Neyman then spent a year in
Paris,after which he returnedto Poland.The next recordof interactionbetween Neyman and Fisher
is linked to Neyman's work in 1932 with Egon Pearsonon the theory of statisticaltests.
and J.M. TANUR
S.E. FIENBERG
244
5
The 1925 and 1927 ISI Discussions on Sampling
Before Neymanreturnedto the problemsof samplingfroma finitepopulationthathe hadaddressed
in 1923, major developments in sampling were presented at the 1925 and 1927 sessions of the
InternationalStatistical Institute (ISI), and published in 1926 and 1928, respectively. Kruskal &
Mosteller (1980) give a relatedand somewhatmore detaileddiscussion.
A substantialportionof the 1925 ISI Meeting was taken up with discussions of the "methodof
representativesampling".In 1924, the Bureau of the ISI appointeda Commission, consisting of
ArthurBowley, CorradoGini, Adolph Jensen, Lucien March, VerijnStuart,and Frantz Zizek, to
study the representativemethod. Jensen served as rapporteurand leader of the discussion at the
meeting. The Commission Report (Jensen, 1926a) contains a descriptionof two methods:random
sampling(with all elementsof the populationhavingthe sameprobabilityof selection), andpurposive
selection of large groups of units (in modernterminologyclusters) chosen to match the population
on selected control variates.The reportdoes not really attemptto choose between the methods. In
one of severalannexesto the report,Bowley (1926) provideda lengthy theoreticaldevelopment,but
failed to describefully how the statisticaltheoryof purposivesamplingworks.Whatis remarkableto
us in this Reportand its Annexes, especially in light of the controversythatIsserlis and Greenwood
were to ignite only the next year,is the singularlack of rto
references the theoreticalworkon sampling
from finite populationsby Isserlis, Neyman, and Tchouproff,althoughthe referencelist in Jensen's
(1926b) Annex did include one referenceto Tchouproff,his 1910 thesis! Tchouproffwas presentat
the meeting but because he was ratherill he appearedto contributelittle to the discussion. He died
soon afterthe meeting.
Yates (1946), in reviewing these proceedings, notes the "lack of any clear conception of the
possibility, except by the selection of units wholly at random,or by the inadequateprocedureof
sub-dividingthe sample into two or moreparts,of so designing samplinginquiriesthatthe sampling
errorsshould be capable of exact estimationfrom the resultsof the inquiryitself". (p. 13). He goes
on to describethe developmentsin randomsamplingthattook place in Englandlinkedto agricultural
experimentationbetween 1925 and 1935, stimulatedlargely by the elements of randomizationin
experimentationand the analysis of variance,both of which he attributesto Fisher.We see this as
anotherindication of the lack of impact of Neyman's early contributionsin these two interrelated
fields.
At the 1927 ISI meeting, CorradoGini presenteda paper on the application of the purposive
methodto the samplingof recordsfromthe 1921 Italiancensus (see Gini, 1928, and Gini & Galvani,
1929), which was to play a pivotal role in the later work by Neyman. They needed to discardmost
of the recordsof the 1921 census before taking the next one, and they proposedto retaina sample
for futureanalyses and reference.To make theirsamplerepresentative,they chose to retainall of the
datafrom 29 out of 214 large administrativedistrictsinto which Italy was divided. They appliedthe
purposivemethod in orderto select the 29 througha process which attemptedto matchthe averages
on seven importantcharacteristicswith those for the countryas a whole. When they did this, they
discoveredthattherewere substantialdeviationsbetweenthe sampleandthe entirecountryfor other
characteristics.This, they claimed, called into question the accuracyof sampling. It remainedfor
Neyman to take up the challenge embodiedin this claim.
6
Neyman's 1934 Paper on Sampling
Neyman took up the challenge in his classic 1934 paper presentedbefore the Royal Statistical
Society, "Onthe two differentaspectsof the representativemethod".We review the paper'selements
and emphasize how Neyman rescuedclusteringfrom the clutches of purposivesamplingand gave it
a rightfulplace in the foundationsof randomsamplingmethodology.
Neyman originally preparedthe paperin 1932 in Polish (with an English summary)as a booklet
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 245
growing out of his practicalexperience. He had been working for the Institutefor Social Problems
on a projectinvolving sampling from the Polish census to obtaindatato describethe structureof the
working class in Poland.As he wrote to Egon Pearson,he used the opportunityand "pusheda little
the theory"(Reid, 1982, p. 105). Publishedin 1933, the original Polish version of the paper traces
a good deal of history (Neyman had clearly learnedhis lesson aboutcitationpractices and was now
unquestionablyfamiliarwith the literature);the 1934 version publishedin the Journal of the Royal
Statistical Society includes even more history and carefully cites many of the people who were to
be in the room for the presentationbefore the Society. Neyman gives enormous credit to Bowley
for his 1926 synthesis of the theory underpinningsampling methods, but also notes the hundreds
of contributionsin the area on which his comparison between purposive and random sampling
builds. There is, however,the curious presentationof optimal allocation with no reference back to
Tchouproff(1923a, 1923b).
The goal of the paper is to comparepurposiveand randomsampling. But elements of synthesis
are prominentas well. Neyman describesstratifiedsampling,noting thatBowley considers only the
proportionatecase but stating that such restrictionis not necessary.He gives a crisp descriptionof
cluster sampling: "Suppose that the population n of M' individuals is grouped into Mo groups.
Insteadof consideringthe populationn we may now consideranotherpopulation,say n, having for
its elements the Mo groups of individuals,into which the populationf is divided .... If there are
enormousdifficultiesin samplingindividualsat random,these difficultiesmay be greatlydiminished
when we adoptgroups as the elements of sampling"(pp. 568-569). This is a new synthesis-earlier
conceptualizationsof clustering had coupled it with purposivesampling. Indeed, Neyman quotes
Bowley as maintainingthat"in purposiveselection the unit is an aggregate,such as a whole district,
and the sample is an aggregateof these aggregates,while in randomselection the unit is a person
or thing, which may or may not possess an attribute,or with which some measurable quantity
is associated"(p. 570). Neyman goes on to explicitly uncouple clusteringand purposivesampling,
saying, "Infact the circumstancethatthe elements of samplingare not humanindividuals,but groups
of these individuals,does not necessarilyinvolve a negationof the randomnessof the sampling"(p.
571). He calls this procedure"randomsampling by groups"and points out that, althoughBowley
did not consider it theoretically,he used it in practicein London, as did 0. Andersonin Bulgaria.
Neyman also speaksof combiningstratificationwithclusteringto form"randomstratifiedsampling
by groups".[Bowley, in his role as the lead discussantof Neyman's paper,notes the innovativeness
of Neyman's suggestion of randomstratifiedsamplingof groupsand acknowledgesthatin fact it was
what he had been drivento use in his work even thoughhe has not fully recognizedthe implications
of what he had done]. Then Neyman refersto the full theoryfor best linearunbiasedestimation3of a
populationaveragedeveloped in his 1933 Polish publication,and he gives the now familiarformula
for the varianceof the "natural"weighted averageestimatorin stratifiedcluster sampling.
In orderto compare Gini and Galvani's method of purposiveselection with his own method of
randomstratifiedsampling by groups, Neyman imbues the purposivemethod with a structurethat
allows one to treat it as if it were based on a special form of randomselection, even though this
was clearly not the way Gini and Galvaniactually selected or conceived of their groups (districts)4
3There was an implicit assumptionin Neyman's work regardingthe uniquenessof the best linear unbiased estimate in
samplingfrom finite populations.This assumptionturnsout to be false. Godambe& Thompson(1971, pp. 386-387) observe
that "Neymanproposedthe use of the Gauss-Markofftechniqueto prove the optimality(UMV-ness)of the
sample mean and
othersimilarestimators.Indeed,duringthe discussion,Fisherconcurredin this method;andit appearsthatthe Gauss-Markoff
technique, now known to be of doubtful validity in sampling theory (Godambe, 1955), was one of the very few points on
which Neyman and Fisher were in agreement".
4Neymanenvisions a universeof districtsdividedup into strataaccordingto the values of one or morecontrolvariates,and
then each stratumis subdividedinto substrataaccordingto the numberof units in the district.Then within each substratum
he speaks of the randomselection of a preassignednumberof districts.Of course, because of the
large size of the districtsin
Gini and Galvani's situation, most stratawould contain zero or one district.This leads to the
sampling bias which Neyman
then illustrates.
246
S.E. FIENBERG
and J.M. TANUR
In doing so, Neyman constructsa sturdycoffin inwhich to burythe method of purposiveselection,
and then he proceeds to slam the lid of the coffin tight and nail it shut, by presentingone statistical
argumentafter another.First, he considersthe conditionsunderwhich the purposivemethod, which
utilizes the regressionof groupmeans on a control variatey, producesunbiasedestimates. He then
notes, on the basis of calculations from Gini and Galvani's own data, that the conditions appear
not to be satisfied in practice.Neyman goes further,however,and points out that Gini and Galvani
make the implicit assumptionthat the groups (clusters) are themselves randomsamples from the
population,somethingthat is decidedly false. Neyman's own methodof randomstratifiedsampling
by groupsdoes not have these deficiencies.Neymancarriesthe argumentone final step by exploring,
using his varianceformula, whetherit is preferableto sample a small numberof large groups (in
effect a varianton Gini and Galvani)or a large numberof smallerunits.The latterturnsout to be the
clear choice and this providesNeyman with the final nail to hammershut the coffin of the purposive
method. Shortly later,the coffin was buried,althoughthe ghost of the purposivemethod continues
to rise until this day in the form of quota sampling.
The immediateeffect of Neyman's paperws to establishthe primacyof the methodof stratified
random sampling over the method of purposiveselection, somethingthat was left in doubt by the
1925 ISI presentationsby Jensen and Bowley. But the paper'slonger-termimportancefor sampling
was the consequenceof Neyman'swisdom in rescuingclusteringfromthose who were the advocates
of purposive sampling and integratingit with stratificationin a synthesis that laid the groundwork
for modern-daymultistageprobabilitysampling.
The heart of the 1934 paper, however, in terms of the amount of space Neyman devoted to
exposition and in terms of emphasisin the discussion by othersat the presentationbefore the Royal
StatisticalSociety, is the materialon confidenceintervals.Therewas much confusion as to whether
this was just Fisher's fiducial method presentedin a slightly differentfashion or a new inferential
approach.Neyman introducedthe idea of coverage in repeatedsamples, but the discussion did not
pick up on its originality.It is somewhatironicthatin his discussionof Neyman(1934), Leon Isserlis,
who only seven years earlierhad condemnedNeyman for not giving enough credit to Tchouproff,
completely overlookedthe fact that Neyman failed to credit Tchouproffin this paper,this time for
proposingthe notion of optimal allocation in stratifiedsampling.We speculatethat Isserlis' failure
to questionthis point arose largely from the troublehe was havingunderstandingNeyman's concept
of confidence intervals,the latterbeing the sole topic of Isserlis' discussion.
Two decades later Neyman (1952a) acknowledgedTschouproff'spriorityin discovering these
results:
"I am obliged to Dr. DonovanJ. Thompsonof the StatisticalLaboratory,Iowa State
College, Ames, Iowa, for calling my attentionto the article of A.A. Tschuprow,"On
the mathematicalexpectationof the moments of frequencydistributionsin the case of
correlatedobservations"publishedin Metron,Vol. 2, No. 4 (1923), pp. 646-680, which
contains some resultsrefoundby me and published,withoutreferenceto Tschuprow,in
1933.
The results in question are the generalformulafor the varianceof the estimate of a
mean in stratifiedsampling and the formuladeterminingthe optimumstratificationof
the sample. These formulaeappearedfirstin a Polish bookletAn Outlineof the Theory
and Practice of RepresentativeMethod,Applied in Social Researchpublishedin 1933
by the WarsawInstituteof Social Problems.Lateron they were republishedin English
in the Journal of the Royal StatisticalSociety, Vol. 97 (1934), pp. 558-625. Finally,the
same formulae, again without a reference to ProfessorTschuprow,were given in the
second edition of my book, Lecturesand Conferenceson MathematicalStatistics and
Probability,Washington,D.C., 1952.
The purposeof this note is, then, to recognize the priorityof ProfessorTschuprow,
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 247
to express my regretfor overlookinghis results and to thankDr. Thompsonfor calling
my attentionto the oversight."
And in the 1970s, Neyman continued in this apologetic vein by reprintingthe recognition of
priorityin an introductionto a book of correspondencebetween Markovand Tschouproff(Neyman,
1981). What seems clear today for those of us who go back for Tschouproff(1923a, 1923b) is thatif
Neyman had read these papers,it would have been difficultto have missed Tschouproff'streatment
of stratificationalthoughwe confess to having to searchto identify the result on optimal allocation.
Thus, the simplestinterpretationfor us is thatwhen Neymanwas developingthe materialfor the 1934
paper in Poland he not only did not have easy access to Tschouproff'spaper,but simply presumed
that everyone would accept the notion of stratification,especially in light of the ISI discussion and
publicationson the topic.
Fisher's lenghty discussion of the Neymanpaperis noteworthyin the presentcontext, not so much
for his challenge that Neyman's confidence intervalswere in some senses fiducial intervalsundera
differentname, butratherfor his observationson the importanceof the parallelismbetween sampling
in economic researchand the role of samplingof finite populationsin agriculturalexperimentation.
The majordistinction,he was reportedas saying, was that:"Ina well-designedexperiment,however,
the mathematics were simplified, and all anxiety was avoided in respect to different systems of
weighting"(p. 616).
Of course, Fisher did recognize that Neyman's goal in creating confidence intervals was quite
different from his own goal of inductive inferences from the sample at hand. As Edwards (1995)
remindsus, Fisherexpessed his skepticismregardingthe value of the coveragepropertyof confidence
intervalsand he remarkedin his discussion thatNeyman's generalization"was wide and handsome,
but it had been erected at considerableexpense, and it was perhapsas well to count the cost" (p.
618).
Our question of why the Neyman paper had such a profoundinfluence comparedto the earlier
work of Tchouproffandothers,was also raisedby Bellhouse (1988) and Kruskal& Mosteller(1980).
We believe that while Tchouproffhad clearly derived a numberof the technical results a decade
earlier,and had droppedthe constraintof constantprobabilitiesof selection, his paperswere abstract
and formal in nature,and his results were far removed from real-worldapplication.Neyman more
clearly laid the groundworkfor statistical practice by his innovativeintegrationof clustering and
stratification,and his clear and convincing exposition of the inferiorityof the purposive method.
Neyman provided the recipe for others to follow and he continuedto explain its use in convincing
detail to those who were eager to make randomsampling a standarddiet for practicalconsumption
(e.g., see Neyman, 1952b, for a description based on his 1937 lectures on the topic at the U.S.
Departmentof AgricultureGraduateSchool).
7
Fisher, Neyman, and The Design of Experiments in 1935
The year after Neyman presentedhis sampling paperto the Royal StatisticalSociety, three publications resolved all doubt about who was to gain full credit for the design of experimentsas an
areaof statistics.To understandthese events surroundingthese publications,however,we move back
several years and trace the interlockingdevelopmentsin this areainvolving Fisher and Neyman.
Following Fisher's series of papers on experimentaldesign in the 1920s, those at Rothamsted
such as Yates and Wisharthelped to propagatehis ideas and move them into actual agricultural
practice, and Fisher temporarilyturned his focus to the completion of The Genetical Theory of
Natural Selection, publishedin 1930, as well as relatedissues of genetics and eugenics and his ideas
on inverse probability.But Fisher and others recognized the need to pull together the new ideas
on statistics and experimentation.In 1930 he lectured on the topic at ImperialCollege in London
and at Chelsea Polytechnic and then he spent the summerof 1931 at Iowa State University giving
248
S.E. FIENBERGand J.M. TANUR
some related lectures. This was to be the beginning of notes that ultimatelyled to The Design of
Experiments,in 1935. Meanwhile,Neyman was workingon a series of paperson hypothesistesting
with Egon Pearson,and on some agriculturalexperimentationproblems.
By the late 1920s, Fisher was becoming restless at Rothamstedand considered applying for
positions at Britishuniversities(Box, 1978, pp. 202-203). Box (1978) andReid (1982) describesome
of the correspondencebetween Fisher and Neyman in 1932 and 1933, primarilyabout Neyman's
paperwith Egon Pearsonon tests of hypotheses.When KarlPearsonretiredfrom UniversityCollege
in 1933, both his position and the departmentwere split in two, with Egon Pearsonbeing appointed
as Reader in Statistics and head of the Departmentof Applied Statistics and Fisher being given
the position of Galton Professorof Eugenics and responsibilityfor the GaltonLaboratory.A major
issue between the two was responsibility for teaching of statistics. Pearson proposed that Fisher
not teach statisticaltheory,and so Fisherrespondedby proposingto teach the logic and philosophy
of experimentation(Box, 1978, p. 259 and Bennett, 1990, p. 192). Here we see anotherimpetus
for Fisher to complete the book on experimentaldesign. In mid 1933, upon hearing of Fisher's
appointmentat UniversityCollege, Neyman wrote to him asking aboutthe possibility of a position
in the Laboratory.But Fisherhad nothingto offer specificallyin statisticsand, in 1934, Neymanleft
Poland andjoined Pearson'sdepartmentfor a threemonthvisiting position, which ultimatelyturned
into a permanentnt
one. It was also at aboutthis time that FishernominatedNeyman for membership
in the InternationalStatisticalInstitute.Thatfirsttermback in London,Neyman lecturedon interval
estimationand Fisher on experimentation,with Neyman in attendance.
In Marchof 1935, Neymanpresenteda paperbeforetherecentlyformedIndustrialandAgricultural
Research Section of the Royal StatisticalSociety (Neyman, 1935) entitled "Statisticalproblemsin
agriculturalexperimentation".The paperwas based on work done in Poland, in co-operationwith
two junior colleagues, Iwasziewicz and Kolodziejczyk.In it, he presentedformal statisticalmodels
for the analysis of randomizedblockanLatin
and
squaredesigns, and he claimed that "theapplication
of the usual tests of significanceto the resultsof experimentsby the two methodsis not yet rigorously
justified, thoug thteeffect of the inaccuracyinvolved may be negligible".In other words (but not
those explicitly used by Neyman), Fisher's claims aboutthe validity of tests were wrong! Neyman
gives the model explicitly; treatmentsaffect individualplots in a row-by-columnlayout differently
(i.e., unit-treatmentnon-additivity).The model and Neyman'sdiscussion of it includes the notion of
an infinite series of repetitionsof the experiment,and it is with respect to this long-rundistribution
that he takes expectations.In an appendix,he shows that, if the averagetreatmenteffect over the
entire experimentalarea is the same for all treatments,the usual estimate of the errorvarianceis
unbiasedfor randomizedblocks whereasfor his Latin squaremodel there is a bias. He repeats the
opening claim that this bias may not be substantial.This means that the F test (referredto by both
Fisher and Neyman as the z test) for treatmenteffects in a Latin square"may ... cause a tendency
to state significant differentiationwhen this, in fact, does not exist" (p. 154). Then Neyman goes
on to examine the test of hypothesis that one treatmenthas a greatereffect than another and he
demonstrates,in a series of examples, that Latin squaresare often less efficient than randomized
blocks. Throughoutthe paper Neyman employed the notions of type I and type II errors he had
developed with Pearson.
Fisher was the lead discussantof the paperand he respondedwith greatsarcasmaboutNeyman's
surprisingresultsnotingthatthose present"hadto thankhim, not only for bringingthese discoveries
to their notice, but also for concealing them from public knowledge until such time as the method
should be widely adopted in practice!"(p.155). Fisher questioned Neyman's apparentinability to
grasp the very simple argumentby which the unbiased characterof the test of significance might
be demonstratedand then he presenteda simple illustrationin the context of a 5 x 5 Latin square.
Here Fisher states ratherobliquely one of his key differences with Neyman, the hypothesis to be
tested, which in the Fisherianformulationis that treatmentshave no effect on yields whereasin the
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 249
Neyman formulationit relates to zero treatmenteffects averagedover all plots. Then Fisher invokes
the randomizationdistribution,which in effect assumes unit-treatmentadditivity.Yates and Wishart
also contributedto the discussion in a more polite, but just as critical fashion, and when Neyman
attemptedto respondbriefly,Fisher interruptedwith the final statement:
"Ithinkit is clearto everyonepresentthatDr.Neymanhas misunderstoodthe intentionclearly and frequentlystated-of the z test and of the LatinSquareand othertechniques
designed to be used with that test. Dr. Neyman thinks that anothertest would be more
important.I am not going to arguethatpoint.It may be thatthe questionthatDr.Neyman
thinksshouldbe answeredis moreimportantthanthe one I have proposedand attempted
to answer.I suggest thatbeforecriticizingpreviousworkit is always wise to give enough
study to the subject to understandits purpose.Failing that it is surely quite unusualto
claim to understandthe purposeof previous work betterthan its author.
Froma purelystatisticalstandpointthe positionis thatDr.Neymanwouldlike a different
test to be made, and I hope he will invent a test of significance, and a method of
experimentation,which will be as accuratefor questions he considers to be important
as the Latin squareis for the purposefor which it was designed."
Neyman laterrespondedat lengthin writingbut his relationwith Fisherhad been shatteredandmost
presentat the presentationof the paperseemed to side with Fisher.The controversythat began with
this exchange raged throughoutFisher's lifetime, and it included not only fundamentalaspects of
experimentaldesign but also two very differentapproachesto testimonyand statisticalinference.
The dispute that erupted over the Neyman paper was bound up in issues of estimation and
hypothesis testing and the difference between the two types of inference. Even with Neyman's
model, the usual F-test in the Fisherianapproachhas the correctnull distributionwhen there is "no
treatmenteffect", because then there is no issue of whethertreatment-unitadditivityholds. Fisher
correctly pointed this out in the discussion. Neyman was insisting on a different null hypothesis,
that the treatmenteffect, throughits interactionwith plots, averages out over the complete set of
plots used in the experiment.This is essentially estimatingthe treatmenteffect in the non-null case,
however,and here it matterswhetherone allows for treatment-unitadditivityor not. Fisherdid allude
to this difference,butin languagethatremainsdifficultto decipher.(Rhetoricwas at a premiumrather
than statisticalclarity.)Nelder (1994) essentially arguesthat Neyman's errorwas in formulatingan
uninterestingnull hypothesis,largely as a resultof the non-hierarchicalnatureof its construction.In
such circumstancesone also needs to look at the alternatives.Thus Nelder arguesthatFisher's attack
on Neyman was irrelevantbecause he dealt only with the null case. Further,the Neyman/Pearson
approachto confidence intervals and tests of hypotheses, which Fisher was never to accept, was
essential to the Neyman argumentin this paper;the ideas were still too new for most present to
understand.Finally, we note thatrandomizationseemed to play little role in Neyman's argumentsin
the 1935 paper whereas it was essential to Fisher's ideas, clearly structuringall of his thinking on
models and the validity of relatedstatisticalmethodsfor the analysis of experiments.
Just two months after Neyman's presentation,Fisher's successor at Rothamsted,Frank Yates
(1935), presentedhis own paperon agriculturalexperimentationbeforethe same section of the Royal
StatisticalSociety and the debate between Fisher and Neyman continued,with Yates becoming the
target for Neyman's criticism. In a lengthy critique of Yates' exposition of the theory of factorial
experimentation,submittedin writing afterthe meeting, Neyman again used his work with Pearson
on tests of hypotheses to suggest that interactionshad low power of detection and that Yates' (and
thus Fisher's) definitionof interactionswere fatally flawed.This discussion seems to have confused
the natureof the contrastsused to estimateeffects in the Fisher-Yatesframeworkand Yates avoided
the possibility of others being similarly confused by alteringslightly the definitions in the printed
version of his paper(which followed Neyman's paperin the Journal).This minor alterationvitiated
250
S.E. FIENBERGand J.M. TANUR
all of Neyman's analysis. Yatesalso respondedrathercausticallyin writingto Neyman'scritiqueand
Neyman then submittedan addendumpointing out why he still thoughtthe Fisher-Yatesapproach
was wrong. The controversybetween Neyman and Fisher (with Yates as an occasional surrogate)
was now full-blown.
The final of the trio of 1935 publicationson experimentation,Fisher'sTheDesign of Experiments,
was the most significant.Fisher completedthe prefacethe month afterYatespresentedhis paper.In
this book, Fisherpresentedan integratedperspectiveon the logic and principlesof experimentation.
After a brief introductionin which he explains why for inferenceshe does not use Bayes' theorem
in the book, Fisher turns to his famous example of "The Lady Tasting Tea", based on an actual
experimenthe proposedmore than 12 yearsearlierat Rothamsted,in responseto the claim by Muriel
Bristol, an algologist, who claimed thatshe could tell the differencebetween a cup of tea into which
the milk had been poured first (her preference)and one into which the tea had been poured first
(which Fisherhad offeredher). HereFishermade his strongestargumentsfor the need to randomize.
In the next chapter he dealt with Darwin's data on cross- and self-fertilization(in which there
was no randomization),introducingthe notion of matching,and this time he used the argumentof
randomizationto "justify"the use of the t-distributionfor the key comparison.Subsequentchapters
dealt with randomizedblocks, Latin squares,factorialdesigns, confounding,partialconfounding,
and the analysis of covariance.The final pair of chaptersdealt with more theoreticalmaterialon
estimation:fiducial inferenceand the measurementof information.
Few of the ideas in the book were totally new, but their integrationand the power of Fisher's
exposition of them helped to establishrandomizedexperimentationas basic to the scientist'stoolkit.
The battle over randomizationcontinued to be waged in British journals, with Gosset, who still
advocated the use of systematic designs such as Beavan's drill strip or sandwich design, being
Fisher's main antagonist.The debateessentially ended with Gosset's deathin 1937, althougha final
critiqueby Gosset appearedposthumouslyin Biometrikain 1938. On the otherside of the Atlantic,
there was no such battle, and in 1936, on the occasion of its 300th anniversary,HarvardUniversity
awardedFisheran honorarydegree at least in partfor this achievement.Justas with Neyman and his
1934 paperon sampling,Fisher's The Design of Experimentsprovideda recipe for othersto follow.
8
Conclusions
Both FisherandNeymanmadefundamentalcontributionsto statisticalmethodologythatunderpins
the modernapproachesto the design of experimentsand the design of sample surveys. In this paper
we have retracedtheirworkon these topics, especiallyduringthe criticalyearsfrom 1921 to 1935, and
we attemptto explainthe pivotalroles of Neyman's 1934paperon agriculturalexperimentationwhich
ultimatelyled to a rift between the two men thatwas neverto be repaired.In the aftermath,Neyman
stakedout intellectualresponsibilityfor samplingand Fisherdid the same for experimentation.The
two fields subsequentlydriftedapart.
Two issues were raised in the discussion of an earlierversion of this paperpresentedat the 50th
ISI Session in Beijing, China. In this discussion, otherschallengedus regardingthe real reason for
the separationof the design of experimentsandsamplesurveysafter 1935, andregardingthe original
of Neyman's contributionsto sampling.We commenton each briefly.
Some have arguedthat it was the fundamentaldifferences between experimentationand survey
work that were truly responsible for the separationof the two fields within statistics and not the
fued between Fisherand Neyman. Forexample,T.F.M.Smith(personalcommunication)arguesthat
randomizationin an experimentsupportsinternalvalidity,to the collection of experimentalunits,
whereas random selection in a survey setting supportsexternal validity, allowing generalization
to those not in the sample. This is technically correct, but only in a narrowsense. The statistical
tools used in the two fields are essentially the same and the real scientific goals of an experimenter
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 251
involve generalizationbeyond the experimentalunits. This is exactly why we have emphasizedthe
importanceof experimentsembedded in surveys and surveys embedded in experiments (Fienberg
& Tanur,1988,1989). The recent interestin the common featuresof experimentationand sampling
suggests that the technical links are important,as many of the early contributorsrecognized in the
1930s. Thus we do not believe thatthe differencesin perspectivebetweenexperimentationas surveys
alone can account for the separation.The clash of two strongpersonalitieswho were determinedto
gain recognition for their inferentialapproaches,in our view, was a majorcontributionto the split.
Otherscontinueto raise the issue of the extentto which Neyman"stoodon the shouldersof giants".
In particular,Chris Heyde (personal communication)raises the question of Neyman's debt to the
Russian school of sampling led by Tschouproff.It is true thatNeyman was educatedin Karkovand
studied probabilitywith Bernstein,as we note above, but he appearsto have had little or no contact
with statistics and the sampling of finite populations.Thus, as we argued above, we believe that
the contributionsin Neyman's 1923 paperrepresentindependent
inen
discovery.In the 1925 Biometrika
version there is a reference to an earlier Biometrikapaper by Tschouproff,but no recognition of
Tschouproff's 1923 Metronpaper.This appearssomewhatsloppy and even in 1934, Neyman did not
single out Tschouprofffor recognition,eitherin generalor in connectionwith optimalallocation.But
Neyman's purpose in 1934 was differentand he broughta new integrativeperspectiveto sampling
in the paperthat differedfrom those who wrote on the topic previously.
Thus while we believe that Neyman was influencedby the work of many others-few of whom
he referenced-he broughtoriginalityto his paperson samplingand the clarityof his exposition was
crucial to the subsequentdevelopmentof sampling.As a result, he rightly became one of the giants
on whose shouldersothershave stood.
We continue to see importantlinks between the fields of experimentationand sample surveys
and we find their roots in the pioneering work of both Fisher and Neyman, during a time of great
intellectual ferment in the 1920s and 1930s. We believe that the statisticalprofession has much to
learn from their creativeideas.
Acknowledgments
A preliminaryversion of partsof this paperdealing with Neyman was presentedat the "International Conference on Statistics to Commemoratethe 100th Anniversaryof Jerzy Neyman's Birth",
Jarchranka,Poland,November25-26, 1994, and appearedas Fienberg& Tanur(1995a). The present
paperhas addedconsiderablyto the materialdrawnfrom thatearlierone. We thankA.W.F.Edwards,
Chris Heyde, John Nelder, Eugene Seneta and Fred Smith for comments on an earlier version of
the present paper, which was presentedat the 50th ISI Session in Beijing, China in August 1995
(Fienberg& Tanur1995b).
References
Bellhouse, D.R. (1988). A brief historyof randomsamplingmethods.In Handbookof Statistics.Eds. P.R.Krishnaiah& C.R.
Rao, Vol. 6. NorthHolland, Amsterdam,pp. 1-14.
Bowley, A.L. (1926). Measurementof the precisionattainedin sampling. (Annex A to the Reportby Jensen). Bulletin of the
InternationalStatistical Institute,22, Supplementto Liv. 1: 1-62.
Bennett,J.H., ed. (1990). StatisticalInferenceandAnalysis:Selected Correspondenceof R.A.Fisher.OxfordUniversityPress,
Oxford.
Box, J.F. (1978). R.A. Fisher,The Life of a Scientist.Wiley: New York.
Box, J.F. (1980). R.A. Fisher and the Design of Experiments,1922-1926. AmericanStatistician,34, 1-7.
Church, A.E.R. (1926). On the moments of the distributionof squared standarddeviations of small samples from any
population.Biometrika,18, 321-394.
Cochran, W.G. (1980). Fisher and the analysis of variance. In R.A. Fisher: An Appreciation.Eds. S.E. Fienberg and D.V.
Hinkley. Springer-Verlag:New York,pp. 17-34.
Eden, T. & Fisher, R.A. (1927). Studies in crop variation.IV. The experimentaldeterminationof the value of top dressings
with cereals. Journal ofAgriculturalScience, 17, 548-562.
252
and J.M. TANUR
S.E. FIENBERG
Edwards,A.W.F.(1995). XVIII-thFisher MemorialLecturedeliveredat the NationalHistoryMuseum,Londonon Thursday,
20th October, 1994: Fiducial Inferenceand the FundamentalTheoremof NaturalSelection. Biometrics,51, 799-809.
Fienberg, S.E. & Tanur,J.M. (1987). Experimentaland samplingstructures:Parallelsdivergingand meeting. International
Statistical Review, 55, 75-96.
Fienberg,S.E. & Tanur,J.M. (1988). Fromthe inside out andthe outsidein: Combiningexperimentaland samplingstructures.
CanadianJournalof Statistics, 16, 135-151.
Fienberg,S.E. & Tanur,J.M. (1989). Combiningcognitive and statisticalapproachesto survey design. Science, 243, 10171022.
Fienberg, S.E. & Tanur,J.M. (1995a). ReconsideringNeyman on experimentationand sampling:Controversiesand fundamentalcontributions.Probabilityand MathematicalStatistics, 15, 47-60.
Fienberg,S.E. & Tanur,J.M. (1995b). Reconsideringthe fundamentalcontributionsof FisherandNeymanon experimentation
and sampling.Bulletin of the InternationalStatisticalInstitute,50th Session, InvitedPapersBook 3, 1243-1260.
Fisher,R.A. (1918). The correlationbetween relativeson the suppositionof Mendelianinheritance.Transactionsof the Royal
Society of Edinburgh,52, 399-433.
Fisher, R.A. (1921). Studies in crop variation.I. An examinationof the yield of dressed grain from Broadbalk.Journal of
AgriculturalScience, 11, 107-135.
Fisher, R.A. (1925). StatisticalMethodsfor ResearchWorkers.Oliverand Boyd, Edinburgh.
Fisher,R.A. (1926). The arrangementof field experiments.Journalof the Ministryof Agriculture,33, 503-513.
Fisher,R.A. (1935). The Design of Experiments.Oliver and Boyd, Edinburgh.
Fisher,R.A. (1952). Statisticalmethodsin genetics. Heredity,6, 1-12.
Fisher, R.A. & Eden, T. (1927). Studies in crop variation.II. The experimentaldeterminationof the value of top dressings
with cereals. Journalof AgriculturalScience, 17, 548-562.
Fisher, R.A. & Mackenzie, W.A. (1923). Studies in crop variation.II. The manurialresponse of differentpotato varieties.
Journalof AgriculturalScience, 13, 311-320.
Gini, C. (1928). Une applicationde la methoderepresentativeaux materiauxdu dernierrecensementde la populationitalienne
(ler decembre 1921). Bulletinof the InternationalStatisticalInstitute,23, Liv. 2, 198-215.
all'ultimocensimentoItalianodellapopolazione
Gini, C. & Galvani,L. (1929). Di unaapplicazionedel metodorappresentative
(1? decembri, 1921). Annali di Statistica,Series 6, 4, 1-107.
Godambe,V.P.(1955). A unifiedtheoryof samplingfrom finite population.Journalof the Royal StatisticalSociety,Series B,
17, 269-278.
Godambe,V.P.& Thompson,M.E. (1971). Bayes, fiducialand frequencyaspectsof statisticalinferencein regressionanalysis
in survey sampling(with discussion). Journalof the Royal StatisticalSociety,Series B, 17, 361-390.
Greenwood, M. & Isserlis, L. (1927). A historical note on the problemof small samples. Journal of the Royal Statistical
Society, 90, 347-352.
Isserlis, L. (1918). Formulaefor determiningthe mean values of productsof deviationsof mixed momentcoefficients in two
to eight variablesin samples takenfrom a limited population.Biometrika,12, 183-184.
Jensen, A. (1926a). Reporton the representativemethodin statistics.Bulletinof the InternationalStatisticalInstitute,22, Liv.
1, 359-380.
Jensen, A. (1926b). The representativemethod in practice.(Annex B to the Reportby Jensen). Bulletin of the International
Statistical Institute,22, Liv. 1, 381-439.
Kruskal,W.H. & Mosteller, F. (1980). Representativesampling, IV: The history of the concept in statistics, 1895-1939.
InternationalStatistical Review,48, 169-195.
Lehmann,E.L. & Reid, C. (1982). In Memoriam,JerzyNeyman 1894-1981. AmericanStatistician,36, 161-162.
Nelder,J.A. (1994). The statisticsof linearmodels: back to basics. Statisticsand Computing,4, 221-234.
Neyman,J. (1933). Zarysteorii ipraktykibandaniastrukturyludnoscimetodareprezentacyjna(An Outlineof the Theoryand
Practice of RepresentativeMethod,Appliedin Social Research).Institutefor Social Problems,Warsaw.(In Polish with
an English summary).
Neyman, J. (1934). On two differentaspects of the representativemethod:the methodof stratifiedsamplingand the method
of purposiveselection (with discussion). Journalof the Royal StatisticalSociety, 97, 558-625.
Neyman, J. (1952a). Recognitionof priority.Journalof the Royal StatisticalSociety, 115, 602.
Neyman, J. (1952b [1938]). Lectures and Conferenceson MathematicalStatistics and Probability. U.S. Departmentof
Agriculture,Washington,DC. (The 1952 edition is an expandedand revisedversionof the original 1938 mimeographed
edition).
Neyman,J. (1981). Introductionto The CorrespondenceBetweenA.A.Markovand A.A. Chuprovon the Theoryof Probability
and MathematicalStatistics,(Kh. O. Ondar,ed.), Springer-Verlag,New York.
Neyman, J., with the cooperation of K. Iwaszkiewicz and St. Kolodziejczyk. (1935). Statistical problems in agricultural
experimentation(with discussion). Supplementto the Journalof the Royal StatisticalSociety, 2, 107-180.
Pearson,K. (1927). Another"historicalnote on the problemof small samples".(Editorial)Biometrika,19, 207-210.
Reid, C. (1982). Neymanfrom Life. Springer-Verlag,New York.
Rubin, D.B. (1990). Comment:Neyman (1923) and causal inference in experimentsand observationalstudies. Statistical
Science, 5, 472-480.
Russell, E.J. (1926). Field experiments:How they are made and what they are. Journal of the Ministryof Agriculture,32,
989-1001.
Seneta, E. (1982). Chuprov(or Tschuprow),AlexanderAlexandrovich.In Encyclopediaof Statistical Sciences, Vol. 1 (S.
Kotz and N. Johnson,eds.). Wiley, New York,477-479.
Seneta, E. (1985). A sketch of the history of survey sampling in Russia. Journal of the Royal Statistical Society, Series A,
The FundamentalContributionsof Fisher and Neymanon Experimentationand Sampling 253
148, 118-125.
of thetheoryof smallsamplesdrawnfroma finitepopulation.
J. (1925[1923b])Contributions
Biometrika,
Splawa-Neyman,
reads"Theseresultswithotherswereoriginallypublishedin La Revue
17, 472-479. (Thenoteon thisrepublication
de la Republique
Centralde Statistique
Mensuellede Statistique,
Polonaise,tom.vi. pp.1-29, 1923").
publ.parl'Office
of probability
J. (1990[1923a]).Ontheapplication
theoryto agricultural
experiments.
Essayonprinciples.
Splawa-Neyman,
andeditedby D.M.Dabrowska
Section9. StatisticalScience,5, 465-472. Translated
& T.P.Speedfromthe Polish
original, which appearedin RocznikiNauk Rolniczyc,Tom X (1923): 1-51 (Annalsof AgriculturalSciences).
'Student'(1923).Ontestingvarietiesof cereals.Biometrika,
15, 271-293.
'Student'(1938).Comparison
betweenbalancedandrandarrangements
of fieldplots.Biometrika,
29, 363-369.
Al.A.(1923a).Onthemathematical
of themomentsof frequency
distributions
inthecaseof correlated
Tschuprow,
expectation
observations
I-III).Metron,2, 461-493.
(Chapters
Tschuprow,Al.A. (1923b). On the mathematicalexpectationof the momentsof frequencydistributionsin the case of correlated
observations(ChaptersIV-VI).Metron,2, 646-680.
Yates, F. (1935). Complex experimentation(with discussion). Supplementto the Journal of the Royal Statistical Society, 2,
181-247.
Yates, F. (1946). A review of recent developmentsin sampling and sample surveys (with discussion). Journal of the Royal
Statistical Society, 109, 12-42.
Resume
R.A. FisherandJerzyNeymansontbienreconnuscommeles statisticiensqui ont etabliles idees fondamentales
qui
soutiennent
et le plandes enguetesparsondage,respectivement.
le plandes experiences
Danscet articlenousrevoyonsles
contributions
centralesde ces hommesfameuxdansles deuxdomainesde recherche.Nous adressonsaussil'effet d'une
controverse
a l'articlede Neyman(1935)ausujetde l'experimentation
quia rapport
agronome
quia aboutia uneseparation
dansle champsdesrecherches
desexperiences
et des6chantillonnages.
[ReceivedMarch 1996, accepted July 1996]