^iW^^>4Pl^^^^
'^^^
0C*<^
14
10V
Mil
3
LIBRARIES
TDflO
CHlfl
DObSflb?^
3
f99r
ALFRED
P.
WORKING PAPER
SLOAN SCHOOL OF MANAGEMENT
INTERNATIONAL SURVEY
OF THE
NEURAL NETWORK RESEARCH COMMUNITY
Preliminary Report
Michael A. Rappa and Kocnraad Debackcre
Principal Investigators
May
25, 1990
WP#
3170-90-BPS
MASSACHUSETTS
INSTITUTE OF TECHNOLOGY
50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETTS 02139
S{AS5ACKl,'SHTT5 INSnTLTE Or TZCHSOLOCTi
INTERXATTCNAL
r^~R\'F:
o?~-^ _
_
Michael A. J-^rzi I'd Kz'Srjn^z Z-ec-EC^ere
Ma\
We are ex:-; —
:
c-i-:-.;
:
15.
VVT#
:??:
— ~ --:
;
:-i-
"•"
-
:
_-:
- ;-
.,
-
^
--
3I~D-5':-5:
-
-;
-
,
..^i^j
the iensthy questionnaire, as vvetl as char enmusiasin for ur.ce?tar.>iirL£ the 50cicl>>£:c2i asc>?cts cr
Roberts ar- R^^ij- .:'•;- 7
_
-_;
.
-
-
-
^
-
,
-
-
\
-
;-:,;.
v^.
- .-:.
For f_-L- ;- -rormaaon, please contact: Piot. MA. Karpa. cuildins E52-53*. 50 MemtXTai Dr..
Camrr.cLce. NL-\ 02139, U.SA. Tel. -:" ^^^oZT: Fa.x oir/Z53-26cO>.
Abstract
This report describes the preliminary results of an international survey
of the neural network research communitv conducted over a threemonth period between February 16 and May 11, 1990. The objective of
the study is to exanune the sociological dimensiorxs of a research
community, including identification and charactenzation ot the phases
that may occur over hme as researchers attempt to develop a radically
new technology. The scope of this report is limited to providing the
reader with a brief overview of the rarionale for the study and some of
the basic statistics regarding the demographics of the survey
respondents. Over the next six months additional reports will be
produced descnbing the complete findings of the statistical analysis.
Introduction
Neural networks are
a
very interesting method of computation that
is
quite different in
its
approach from conventional rule-based techniques. Inspired by theones of neurological
processes in biological organisms in their design, neural networks are typically implemented in
areas such as machine vision, speech recognition, and signal processing— areas in which
traditional computational
intellectual history that
among
activity
methods have proven
difficult.
Their development has an
spans more than forty-years. In recent years, the level of interest
researchers within several disciplines has risen dramatically, and
have resulted
in the
numerous streams
of
commercialization of neural network computer software and
hardware.
This study of neural networks seeks to understand the internal functioning of research
communities and
is
their role in
contnbuting
to the
development
of a radically
new
technology.
It
part of a larger research program, begun in 1986, which currently includes investigations of
twelve research communities. The results of these earlier studies indicate that research
communities exhibit certain
in
structural
and behavioral
understanding the underlying rate of
scientific
and
characteristics,
which may be important
technical progress.
Specifically, these
studies suggest a pattern of evolution within research communities reminiscent of three distinct
phases, as illustrated in Figure
The phases are diagramed
field.
The
relatively
first
low
phase, which
level of effort.
themselves to furthering the
peers,
1.
in Figure
is
1
in
terms of changes in the level of effort within the
coined the "bootlegging" phase,
During
field,
this period, a
criticized.
characterized by a
handful of researchers dedicate
even though their enthusiasm
and indeed, may be severely
is
may
not be shared by their
Typically, they have difficulty securing
Phase
Phase
Bandwagon
Phasa
II
t
Bcx3tlegging
lll-a
nstitutionalization
t:
_o
ilj
"5
>
(S
Time
FIGURE
1:
PHASE MODEL OF R&D COMMUNITY
name
adequate funding (hence the
bootlegging, which implies that researchers struggle to
maintain their research without formal recognition or funding to underwrite the cost of their
work.) Furthermore, there
scientific
value of the
difficult to see at all.
their line of
new
is
no obvious market
field
working
charactistic of this
have
little
pecuniary motivation
phase represents a sharp transition from the
phase
is
the very rapid increase in the
m the community in a relatively short period of time.
typically
Within
to
extremely
is
pursuing
in
work.
referred to as a
it
emerging technology. While the
be the topic of debate, the commercial value
Thus, researchers tend
In comparison, the second
dominant
may
for the
"bandwagon;" hence the name
for this phase.
This
begins
to
make headway
builds wider scientific legitimacy. TTiis second phase
against
is
also
number
phase. The
of researchers
phenomenon
is
sometimes
As the research community grows
becomes more widely distributed across organizations,
this phase, the field
first
its
sectors,
and
countries.
mainstream detractors and
accompanied by the growth
niche markets, which begin to take advantage of the commercial potential of the
of
new
technology.
As the bandwagon progresses,
paths will emerge:
(a)
the
community
researchers continue to
confronting them, allowing the
community
enters the third phase,
make progress
where one
in solving the
to institutionalize itself,
or
(b)
of
two
problems
progress begins
to
"
slow
down such that
'
researchers
perhaps eventually return
Under
scenario
(a),
become discouraged, forang
to the
success
the
conditions prevailing in the
is at
commuruty
first
to contract
and
phase.
hand: the saentific and commeraal value ot the
technology becomes mcreasmgly apparent, markets begin to expand, funding sources are secured,
and the community begins
to institutionalize itself.
The growth
rate in the
becomes more formalized as
participation will begin to stabilize at a lower level. Recruitment
cumcula and graduate programs, which allow students
universities develop standard
specialize in the field.
that specialize in the
New
commuruty
university research centers dedicated to the field
to
and new firms
development and application of the technology are formed. Moreover,
specialized journals, conferences, and professional societies that facilitate communication
among
researchers are established. Standards begins to emerge, easing the
development.
In
sum, what had once been a fringe group begins
way
for
to establish itself
market
within the
academic and industrial mainstream.
The
alternative scenario (b), illustrates a
researchers' expectations, markets
Under
take hold.
community
suffers,
community
do not develop, and
which progress does not meet
in
the institutionalization process does not
these circumstances, the large majority of researchers begin to
in favor of other
more promising
and funding sources begin
to
research topics. Recruitment of
run dry. The result
is
for the
new
community
out of the
researchers
to revert
phase-one conditions. This implies that over longer pteriods of time, communities
several cyclical fluctuations before successfully institutionalizing
filter
back
to
may undergo
itself.
The Neural Network Community Survey
The
objective of the neural
network community survey
sociological dimensions of the different phases
transitions.
In particular, the
and the
survey focuses on Phases
understand the attitudes and behavior of researchers
that
is,
to
I
is
to
probe more deeply into the
factors precipitating the
and
who
II
in
first
understand the similarities and differences which
an attempt
phase
to better
enter the field in each phase:
may
exist
between "bootleggers
and "bandwagoners.
The survey instrument
is
a 12-page questionnaire designed specifically for collecting
information from neural network researchers about their research
activities.
the survey instrument are organized into five sections that deal with:
of the respondent's activities with neural networks;
(2)
(1)
The questions
in
the extent and nature
the respondent's decision to
work
in the
field of
neural networks;
(3) the
and nature of the respondent's communal
extent
as sharing information with other researchers through conferences, papers,
activities,
such
and other
collaborations; (4) the respondent's perceptions regarding the future progress of neural networks
research;
and
(5) the basic
demographic characteristics of the repondent and
his or her
organization.
The planning and design
1990.
During
this
time a mailing
using authorship in
summer
stage of the survey
scientific
list
and
was conducted from November 1989
to
February
population for the survey was constructed
of the target
technical journals, conference proceedings,
workshops, and
schools dedicated to neural network research over a two-year period from 1988 to 1989
(fourteen different literature sources in
total).
The
target population
numbered
A
letter
2,037
individual researchers from thirty-five countries.
The questionnaires were mailed on February
March
Questionnaires were received up until
1.
data were entered into the data
statistical
file
16, 1990.
May
and a system
follow-up
11, 1990, at
file
was mailed on
which time the
(using SPSS)
was created
last of the
to facilitate
analysis.
Preliminary Results
Over
the three-month data collection period, a total of 710 individuals respionded to the
survey (see Figure
satisfactory for
an
2).
This represents an effective rate of return of 38%, which
considered
must be judged
in
budget constraints did not allow for the survey (which was in English)
to
international survey of this size
light of the fact that
is
and scope. The return
rate
be translated into the native languages of the researchers. The majority of the researchers in
the target population lived in English-speaking countries,
and many
of those
who
did not, have
published in English language journals. Moreover, because mailing addresses for the target
population were derived from documents published in 1988-89, the problem of invalid addresses
due
to
employment changes was
are: the length of questionnaire,
which was necessary
inability to
in order to
fairly
which
common. Other
is
factors contributing to the return rate
long by most standards; coding of the questionnaires,
provide the results of the study
to participants;
and the
provide return postage due to the costs involved and the international scope of the
survey (although, return postage was guaranteed). All things considered, the 710 completed
questionnaires
employed.
is sufficient in
number
for
most,
if
not
all,
of the statistical techniques to be
Figures 3
and 4
illustrate the regional
distnbution ot respondents and the
number ot
respondents by country (for the to|>10), respectively. The regional distnbution of respondents
corresponds very closely
sampled population. Most respondents are woricing in
to that of the
the U.S. (60%); about one-quarter are based in Europe
Although
it
plausible that the survey
is
examination of the
scienrific literature
is
and 10% are based
skewed toward U.S.-based
on neural networks
in 1988-89
in the Far East.
researchers, an
shows a roughly
similar
regional distribution of researchers.
The
sectoral distnbution of respondents
of the respondents are
employed
is
shown
in universities,
in Figure
Slightly less than two-thirds
5.
while one-quarter are industrial researchers,
with the balance in government organizations. Here again, the distribution corresponds
target population
and
shown, there are a
firms,
and
to the scientific literature
total of
on neural networks
216 universities represented
among
to the
Although not
in 1988-89.
the respondents, 101 industnal
government organizations.
51
Figure 6 provides the disciplinary distnbution of respondents, which shows that electrical
engineers compose more than one-third of the
scientists, the
combined
second largest category
respondents.
literature
As with
total for
is
EE and CS
is
total.
more than one-half
the physical sciences,
In terms of their position,
where
physicists
together with computer
of
all
respondents. The
make up
the majority of
and regional distnbutions, an analysis of the
the sectoral
on neural networks
When added
in 1988-89
37%
shows
scientific
a similar disciplinary distribution.
of the respondents hold faculty appointments (see Figure
7).
The second most common position
is
that of staff scientist (or researcher),
about 33% of
is
followed by students (17%), engineers (7%), and managers
(5%).
is
all
respondents. This
Although not
illustrated, the
35 years. The youngest quartile
average age of respondents
is
between the ages 22 and
is
which represents
37 years and the median age
30; the oldest quartile
is
between
ages 43 and 69.
Figure 8
shows
the current funding for neural network research broken
down by source
for the
respondents. The largest source of funds for respondents are their employer (averaging about
42%
of total funds), followed by
Slightly
more than
government agencies (averaging about 39% of
one-third of government funding
industrial sponsors,
is
total funds).
defense-related. Personal funds,
and private foundations make-up the balance of funding.
In terms of their level of interest in neural networks. Figure 9
shows
that over
50%
of the
respondents view neural networks as their major or only research interest. In contrast, about
40% view
as
it
one of many research
research agenda. Figures 10
and
11
Only about 10% see
interests.
it
as a minor issue in their
provide a breakdown of the theoretical interests and
DARPA
various application areas of respondents. Using the categories set-out in the
Network Study
which
(1988),
respondents were asked
their research focuses.
and algorithms.
Classification
11% claimed
most
the largest application category, followed
17%
to a great extent
to
The data
10% indicated
"oriented
identified "applied research";
and
"development" in nature. About 19% of
neural networks.
number of respondents entering into
to 1990.
by vision and
of the respondents indicated that their neural
one or more patents related
Figure 12 shows the
from 1950
were
by leammg
the largest theory category, followed
characteristic of their work;
that their activities
the respondents hold
total of
theory and application areas on
activities are to a great extent "basic resccu-ch" in nature;
basic research" as
year,
is
is
Although not shown, 40%
signal processing.
network
Simulation
to indicate the
Neural
the neural network field in each
indicate that entry reached a
peak
in 1986
150 and 160, respectively. However, the decline in entry in the
and
1987, with a
most recent years
is
explained in part by the techniques used to identify the survey sample population and not
necessarily
by an actual decline
In order to identify the first
the cumulative
13).
in the entry of
new
two phases of the community's evolution, a graph
number of respondents over time as
The graph shows both the cumulative number
and cumularive percent (on the
which began about
1984.
researchers.
right axis).
About 25%
It
is
made
of
indicated by their year of entry (see figure
of respondents in the field (on the left axis)
clearly illustrates the rapid
growth
in the field,
of the respondents entered the field prior to 1984,
whereas
about 75% entered from 1984 to 1990. In accordance with the phase model presented in the
introduction, in the subsequent analysis the period from 1950-1983 will
1,
be designated as Phase
or the "bootlegging" phase; and the period from 1984-1990 will be designated as Phase
the
or
"bandwagon" phase.
A
major
p>oint of interest is in
understanding the similarities and differences in researchers
who enter the community during the bootlegging phase in comparison to
the
II,
bandwagon phase. The preliminary
differences.
to their
First,
is
shown
who enter during
analysis suggests that there are several sigruficant
an investigation of respondents' entry
graduate education
those
into neural
network research
relative
in Figure 14, using a scatter-plot of respondents' year of
'
A
entry into the neural network field by their year ot graduation (tor highest degree).
five degree line through the
ongin highlights the distinction between researchers who
working on neural networks before graduating
those
who
(i.e.,
Also, a horizontal line through the year 1983 segments the
started after graduation.
the field after graduating, with 2 containing respondents
phase and 3 containing respondents
and 4 include respondents who
who
started in the
started neural
who
started
15, the
work
bandwagon phase.
who
bar diagram illustrates that
bootlegging phase, the large majority started in the
Similarly, quadrants
bandwagon
containing
1
phase. As can be
among respondents
field as students.
entered
in the bootlegging
network research as students, with
students in the bootlegging phase and 4 containing students in the
seen more dearly in Figure
started
thcv entered the field as students), and
data into the two phases of interest. Thus, quadrants 2 and 3 include respondents
1
tortv-
in the
This contrasts markedly
with the bandwagon phase, where the large majority of respondents entered the
field after
graduation.
A
field, as
also
to indicate the
in the field of neural
of the factors influencing respondents entry into the
analyzed according
phase
diagrams
in Figure 16
exit decision,
importance of a variety of factors on their decision
continuing to work in the
in
to the
in
which the respondent began working
and 17 indicate the mean value
for
"intellectually compelling nature of neural
by the
by
were more influenced by the
networks than
"
their
bandwagon
counterparts; they
were
network researchers"; and they were
the "potential for peer recognition" than their counterparts in the
no
The bar
is statistically significant.
the "positive opinions of other researchers"; they
"recent successes of other neural
in the field.
each group for each factor in the entry
and whether or not the difference between groups
less influenced
to
Again, the responses were
field.
In terms of the entry decision (Figure 16), bootleggers
were
The
networks, as well as the importance certain factors might have on
diminishing their interest
and
made
well as those factors that might influence respondents' exit from the field.
respondents were asked
work
was
preliminary analysis
less influenced
less influenced
bandwagon
by
phase. There
significant difference in the influence of "dissatisfaction with the previous research
is
agenda
or the potential for "solving an impKjrtant sooetal problem" between the two groups. However,
bootleggers were less influenced in entering the field by the "availability for funding" of neural
network research than bandwagon respondents; they were
promising research topics
and they were
';
less
influenced by the "lack of other
they were less influenced by the "potential for financial rewards";
less influenced
by the "opportunity
bandwagon counterparts.
8
to build a
commercial enterprise" than
their
As with
the entry decision, there are also several statistically significant differences
between the two groups
in terms of the factors
which might influence the decision
respondents' commitment to neural networks (see Figure
is
no
significant difference
between the two groups
interestingness of neural networks.
"
17).
The only category
diminish
which there
in terms of a "diminished intellectual
is
However, bootleggers are
lack of funding for their research" than their
in
to
bandwagon
less likely to
be influenced by
counterparts; they are less likely to
be influenced by "increased financial costs of conducting neural network research"; they are
likely to
to
be influenced by "rapid progress
be influenced by "overcrowding in terms of the number of neural network researchers";
networks"; they are less likely
in neural
networks "; they are
to
rewards"; they are
in
less likely to
up with new development
they
in neural
be influenced by "slow progress in solving technical problems
less likely to
activities at their organization";
less
they are less likely
in alternative areas of research";
are less likely to be influenced by "difficulty in keeping
a
be influenced by "discontinuance of neural network
they are less likely to be influenced by "a lack of financial
be influenced by "diminished interest
among
other researchers
neural networks"; they are less likely to be influenced by "negative opinions of supervisors ";
and
lastly,
bootlegger respondents are less likely to be influenced by "opinions of leading
researchers unfavorable to neural networks" than their
Lastly, respondents
were asked
to indicate the
number of years
neural networks before reaching their goals. Figure 18
for researchers entering in the
two phases,
category within each group. The data
bootlegging phase are willing
to
work
show
work
shows
counterparts.
they are willing to
the comparison
work on
between responses
terms of the percent of respondents in each
that
for 10 or
goals. In comparison, the majority (about
are willing to
in
bandwagon
50%
of the respondents entering in the
more years
in the field before reaching their
46%) of researchers entering
in the
bandwagon phase
for as long as 2-5 years before reaching their goals.
Concluding Observations
The preliminary
results of the international neural
many
rate is sufficient
and
that the
population. The
first
stage of analysis conducted so far
sample corresponds
of distinct differences between the initial
to
two phases
network survey suggest that the return
is
of the characteristics of the target
encouraging in that
in the
it
shows evidence
proposed model of the
evolutionary development of research communities. Nevertheless, more conclusive evidence
will require further analysis.
Undelivered
(8%)
Repondents
Non-respondents
(35%)
(57%)
Total
sample
size =
Effective return rate =
2037
38%
nCURE 2: NEURAL NETWORK SURVEY RESPONSE RATE
Middle East
Far East
(2.0%)
(10.0%
& South
Amenca
Nortti
Europe
(63.5%)
(24.5%)
nCURE
.^r
REGIONAL DISTRTBUTION OF RESPONDENTS
50
100
150
200250300350400450
Number
of
Respondents
nCURE 4: NUMBER OF RESPONDENTS BY COUNTRY (TOP-10)
Government
Researchers
(11.6%)
University
Researchers
(63.4%)
Industrial
Researchers
(25.0%)
nCURE 5: SECTORAL DISTRIBUTION OF RESPONDENTS
Neural Networks
Psych/Cognitlve Sci
Mathematics
Biological Sci
& Eng
Computer Science
Physical Science
Eiectncal Englneenng
only interest
major interest
one
of
many
minor interest
keep informed
50
200
150
100
Number
350
300
250
respondents
of
HGURE 9: R ESPONDENTS' LEVEL OF INTEREST IN NEURAL NFTWORKS
simulation
learning
algorittims
mattiematical
new
parallel
arch
neurobiology
implementation
'
'
I
'
'
I
''
'
200
Number
of
300
respondents
nCURElO: RESPONDENTS' INVOLVEMENT IN THEORETTCAL
ASPECTS OF NEURAL NETWORKS
classification
vision
signal processing
speech
robotics
sensor fusion
i
I
I
I
(
50
I
I
I
I
I
1
1
I
I
1
1
I
I
I
I
I
100
150
200
Number
of
1
1
I
1
1
250
I
I
I
[
300
respondents
nCURF 11: RESPONDENTS' INVOLVEMENT IN APPLICATIONS OF
NEURAL NFTWORKS
^UU
03
"O
c
o
Q.
to
O)
cc
<D
n
E
3
1
90
llllll'llllttff | |
4Q
"^1
»
1 1
1 1
40
1
50
«
11
I
U lttfffft M
,
1
f
^ J
1
60
Year
of
llll
. t i 1 . 1 .
l
tll
*
70
1
l l
l
ll
1
1
1
'tltl '*JK
1
80
i
»
90
graduation
nCURE 14: RESPONDENTS' FIRST YEAR OF NN RESEARCH
RELATIVE TO YEAR OF GRADUATION (HTCHEST DEGREE)
100.0
_
75.0
CO
o
=
After graduation
50.0
Before graduation
(N-612;P<.001)
a.
25.0
Bootleggers
nGIIRFIS:
Bandwagoners
RESPONDENT S' ENTRY INTO FIELD
RELATTVF T O GRADUATION
intellectual
-
::::::
compeilingness
n
'
jts-^s^asi^aiesiStiiSiesiK^^
ifx.OOl)
positive opinions
'"
recent successes
<
j WlW^
!
l
!
MW^ ^^ '''
:
^W^WJ
^! ^!
peer recognition
dissatisfaction
societal
problem
funding
availability of
lack of othier topics
financial
buikj
rewards
company
1^
3
2
1
Boolteggefs
Mean
4
6
5
7
score (1»not important; 7=.very important)
Bandwagon Resaarcnefs
HGUKE 16: FACTORS INFLUENCING RESPONDENTS' ENTRY INTO THE
FIELD OF N EURAL NETWORKS
intellectual interestingness
lack of funding
^ .»AM;^)V;!t;i»iVW.».;V^^
^
'iAmii^mmimm>i>'M9.mmmi>.>hmm!i
slow progress
discontinuance
financial
rewards
diminished interest
m
i h ii
iiiiililililii n itii
lIliUUAiUMMlluilllililllAUillUddiliuiMAMMri
i
< i< j^ni ijh li j
^
"
tiij^^sft
jj.«j6ii!t!
wv^ww vvw
i
i
El
Bootleggers
^
Barxtwagon Rearcfiers
EXrr
17:
j(p<.001)
^^^^
^
^
!.!.!
(p<.001)
^ ^^^^
(p<.001)
yj^^^JgBWaMW' fe^^^
i
i
^MiiiHwiMmaw*!
(P<.001)
--.-..:-.... im,.
Y,i;<:tw;w:,;w;:,:u::i
.^ ..
.-J
ip<.ooi)
V
^^^wMAW;Wwtaa
12
Mean
nCLT^E
(p<.052)
a
iwit .wi.w^
supervisor
unfavourable opinions
(p<.001)
tmm^m^iim^'^'mffmmtma
alternatives
keeping up
(P<.001)
(p<.028)
cost Increase
overcrowding
TS.I
B^sA
.
(P<.001)
tp<.001)
1
'
1
'
1
'
6
7
5
4
3
score (l-not important: 7=very important)
FACTORS POTENTIALLY TNFLLTNCIN r. RESPONT)ENTS'
FROM THE HELD OF NEURAL NETWORKS
w//^yy/yy/y/^^^^^
(chi
I
0.0
Q
^
Bootleggers
I
I
I
I
I
I
I
I
I
<
I
square-92.22; p<.001)
I
r
f
I
I
I
I
I
10.0
20.0
30.0
40.0
Percent of
total
respondents
wittiin
I
>
'
I
50.0
group
Bandwagon Resaarcners
HGURE 18: NUMBER OF YEARS RESPONDENT IS WTLLING TO WORK
ON NEURAL NETWORKS BEFORE REACHING GOALS
/
u o
w'
,.
^^o&o
00^ s&b^
Date Due
f^AY Oil:
M'^.-f.
29
II
1994
Lib-26-67
BARCODE