Question 1
• How can we succeed?
– in a Business School Context
– Competition comes from Economics, Strategy,
and Organizational Perspectives
– which draw on more widely employed
reference disciplines
Assumptions to Argue About
• “Competition” implies a zero sum game
• It is unusual to cite cognitive literature (see the
work of experimental economics and behavioral
accountants)
• Our work does not cite economics, strategy, or
organizational research: 3 dissertations:
– Studied effects of different utility theory cost and
probability parameters in softlifting experiment (Peace)
– Studying effects of problem-solving cues on strategy
development (Sampler)
– Group spreadsheet debugging work cited group
problem-solving literature (Joseph)
Assumptions to Argue About
• Those fields are secure
– Colleagues in strategy consider their field as the
champion of identity crisis
– If economics is Arnold Shwarzzeneger, strategy might
be Woody Allen.
– What is Cognitive IT?
• Somewhere between Jean-Claude Van Damme and Regis
Philbin
• Our work does not cite widely-used reference
disciplines
– Which disciplines are we missing? Pure math? If that
is required, we would have schools of Finance/Econ.
So, Regis, How do We Succeed?
• Study Interesting Problems (or problems
that will become interesting)
• Attract something:
– Students
– Colleagues
– Grants
– Reporters
Publicity
• Please visit your school’s PR function
• Much of what we do is interesting to
regular people, too.
• I found new respect from others after:
– Chatting with Chris Arnold on All Things
Considered about applying results of
response time research to Web browsing
(1996—result of a Washington Post op-ed)
WSJ Report
• Pam Sebastian’s
column
• Word of mouth’s
interference on
learning a package
• Ironic Philippe Kahn
juxtaposition
• Also reported in
Computerworld, etc.
• ICIS & CACM articles
CNN TV
• Study cited by CNN,
Business Week,
dozens of newspapers, 4 radio
programs
• To appear in CACM
• Intended to show how
a word processing aid
needs to fit cognitive
abilities of users
Question 2a: Are we Doomed
always to be a Minority?
• Yes, unless we act.
– We must be represented on editorial boards and must
not turn over all of the crown jewels to hostile forces
• One person can make a difference: calmly noted the lack of
any relevant track for HCI/cognitive research in ICIS 2003 to
a program chair; now we have an entire mini-track as a direct
result of that conversation
• Our SIGs will help a great deal.
– We should broaden our data collection beyond
college sophomores at least once in a while
– We should write carefully, write well, and explain
effectively the relevance of our studies
Question 2b: What Happened to the
Experiments of the 70s and 80s?
• Experiments are hard to construct that have:
– Basis in theory, but (like Goldilocks and the 3 Bears):
• Not too soft (supported too little by previous studies): A stretch?
• Not too hard (supported too much by previous studies): Obvious?
–
–
–
–
–
Relevance and Rigor
Realism of manipulation
Representativeness of sample
Reasonable magnitude of findings
Capture of appropriate constructs
• How many studies can do all of these? Is it zero? Perhaps.
• If not zero, do we read them as “guilty until proven innocent?”
– A recent submission: a big flaw: we didn’t cite an article released 90 days
after submission
Question 3: What Should We Do?
• Let’s figure out how to fairly evaluate
experiments: what bed is just right?
• Let’s ask/plead/demand our journals to
provide balanced editorial boards
• Let’s market our work properly:
– Don’t oversell it by promising silly implications
– Don’t undersell it by hiding implications
– Don’t try to sell it if no implications!
What to Do (continued)?
• Make agreements to have regular, formal or
informal research exchanges
– Discuss experiments you are designing.
– Be brutally critical at this stage; the best gift to
your colleagues.
– Withholding criticism is not being kind to them.
• Pilot tests are not just for PhD Theses! Fix
materials; debug procedures; ascertain
magnitude of effect size to choose
appropriate sample size.
What to Do (continued)?
• Before a large investment in the experiment,
inventory support for hypotheses
– If support is meager, perhaps save your energy.
– If support is widespread, emphasize your contribution
• Before a large investment in subjects,
–
–
–
–
Do you own tasks before any other steps.
You will find obvious difficulties. Fix them.
Remove any confounds.
Make sure tasks are debugged.