The Professor, The Hooligan and Gender: A replicated study on behavioural priming
Garima Manglani & Lily Buckley Carolan
PSYC375 : Intermediate Research Methods and Statistics
Michael Phillip
September 13, 2024
Word count: 2720
Abstract
Can simply thinking about a university professor make you smarter? The classic study on
behavioural priming by Dijksterhuis and Van Knippenberg (1998) found priming participants
with an intelligence stereotype (university professor) improved their general knowledge test
results, but priming with a “less-intelligent” stereotype (soccer hooligan) decreased scores.
Suggesting environmental cues may unconsciously influence academic performance.
Interestingly, replication studies produced mixed results. Our study aimed to replicate the
original findings while exploring gender influences on task performance. 149 university
students completed a survey using priming manipulation and a general knowledge test. The
results suggested priming successfully activated stereotypes, with no significant difference
revealed in the general knowledge test scores. However, gender significantly influenced test
performance, with males scoring higher than females, regardless of the priming condition.
Findings contribute to the ongoing debate. Although stereotype primes seem effective, the
mechanism remains complex, and further research is required. (final)
The Professor, The Hooligan and Gender: A replicated study on behavioural priming
Behavioural priming is a phenomenon where cognitively activated concepts
unconsciously influence future behaviour (Dijksterhuis et al., 2014). Research has
demonstrated a wide range of priming effects on human behaviour (Bargh, 2006), including
elderly-related primes slowing walking speed (Bargh et al., 1996), superhero primes
increasing volunteering (Nelson & Norton, 2005), and warmth primes promoting prosocial
behaviour (Williams & Bargh, 2008). Behavioural priming is a well-documented
phenomenon in psychological research.
A particularly notable study by Dijksterhuis and Van Knippenberg (1998) found that
priming participants with an intelligence stereotype (e.g., a university professor) improved
their performance on a general knowledge test compared to those primed with a lower
intelligence stereotype (e.g., a soccer hooligan). These findings suggested subtle
environmental cues might influence academic achievement behaviours like test-taking.
However, recent attempts to replicate behavioural priming effects have yielded mixed
results. Doyen et al. (2012) and Lynott et al. (2014) failed to replicate the elderly-walking and
warmth-prosociality effects, respectively. More significantly, a large-scale, direct replication
of the Dijksterhuis and Van Knippenberg (1998) professor-priming study, conducted by 23
laboratories worldwide with 4000 participants, found no difference in general knowledge test
performance between professor-primed and hooligan-primed groups (O'Donnell et al., 2018).
Despite using methods very similar to those of the original studies, these replications have
not observed results similar to those of the original studies.
These “failed” replications have sparked considerable debate in the field of
psychology (Chivers, 2019), contributing to what some describe as a "crisis of confidence" in
psychological research methods (Pashler & Wagenmakers, 2012). Others argue that poorly
conceived, untestable theories are to blame for the lack of replicability (Szollosi & Donkin,
2021). Still others suggest that these failures may be due, in part, to errors in the replication
studies (e.g., Gilbert et al., 2016).
Several explanations have been proposed for the failed replications of the professorpriming effect. One possibility is that the stereotypes associated with professors and soccer
hooligans have evolved since the original study, potentially affecting the efficacy of these
primes (Shanks et al., 2013; O'Donnell et al., 2018). Perhaps participants simply do not have
stereotypes about the intelligence of professors and soccer hooligans. If the manipulation in
the study does not change the psychological variable of interest (i.e., perceptions of
intelligence), we would not expect it to prime a related behaviour (i.e., general knowledge
performance). This study uses slightly modified materials and methods to replicate
Dijksterhuis and Van Knippenberg's (1998; experiment 4) study.
Main Replication Hypothesis
Based on the original findings of Dijksterhuis and Van Knippenberg (1998), if
behavioural priming effects are real, we hypothesise that these stereotype primes would lead
to increased or decreased performance on a general knowledge task.
Replication Extension
In addition to the original study by Dijksterhuis and Van Knippenberg (1998), we
have chosen to explore the influence of gender and behavioural priming on task performance.
We hypothesise that priming effects on gender will not significantly impact task performance
in the general knowledge condition. The inspiration for expanding on the gender variable has
been drawn from recent studies examining the interaction effects between biological and
social factors on cognitive performance.
Hausmann et al. (2023) conducted a study that investigated how levels of testosterone
levels and gender stereotypes influenced a person’s performance in mental rotation and
verbal fluency regarding sex-sensitive talks. The results of their study suggest that gender
stereotypes play a role in increasing sex-related cognitive variances, specifically in mental
rotation tasks. Furthermore, males have higher performance outcomes than females under
stereotype activation. The study accentuates how gender stereotypes significantly influence
cognitive abilities when combined with hormonal factors. This supports the idea that gender
priming may affect task performance outcomes around stereotype influence but not
necessarily in a general knowledge condition. Such research is crucial to understanding how
social cues may combine with biological factors to alter behaviour in several contexts.
Inspiration was also drawn from Josephs et al. (2003) which explores the idea of how
testosterone levels moderate responses to stereotype threats. The study found that individuals,
particularly women who had a high testosterone level, displayed reduced performance under
the influence of stereotype threat. In contrast, men with higher testosterone levels had a
higher tendency to excel while under positive priming of their stereotype. The research
supports the theory that behavioural priming tethered with gender can influence task
performance distinctively based on genetic predispositions.
The above studies offer exciting information, leading to our inspiration to explore
gender and its effects on behavioural priming. They explore how gender priming can have
broader effects in a working environment. The weight of gender stereotypes, hormonal
influences, and task performance encourages our research extension into whether these
effects have a similar outcome on general knowledge. This stimulates the mind to think about
the limits of gender-based priming effects, contributing to a more refined hypothesis and
research design incorporating psychological and biological factors.
Method
Participants
The participants for this study were recruited from The University of Canterbury
Psychology Statistics 375 course, where they were randomly assigned. A total of 149
participants ranged between 19 to 60 years of age (M = 23.8, SD = 6.40). There were four
gender categories in the study: male (1), female(2), non-binary (3) and prefer not to say (4).
Participants who did not identify as male or female were excluded from the extended gender
research study.
Materials
Manipulation Check Measure
Questions were used to test whether the two priming conditions corresponded to the
predicted intelligence stereotypes. Participants rated the intelligence of a typical person from
five different groups on a 0 (least intelligent) to 100 (most intelligent) scale, where 50 was
considered to represent average adult intelligence. The rated groups included rugby fans,
soccer hooligans, university professors, primary school teachers, and politicians.
General Knowledge Test Measure
Forty questions were used to quiz participants on their general knowledge. The
general knowledge test rated participants as correct or incorrect. A correct answer received a
score of 1, and an incorrect answer received a 0. Test-retest reliability was used to increase
reliability; the results would remain consistent if measures were replicated again. Inter-rater
reliability created consistency across people and was used to increase the reliability of the
extended general knowledge gender hypothesis. Participant criteria excluded participants that
identified in groups 3 and 4, while genders 1 and 2 were exclusively used. Reducing gender
variation between participants and increasing consistency across people. The reliability of the
general knowledge test was assessed using Cronbach's alpha (α = 0.657). Items Q16 and Q17
exhibited negative correlations with the total scale, suggesting potential issues with these
items.
Design
The study adopted a between-subjects design, where each participant experiences
only one condition, and the results are compared and contrasted between the different
conditions. For statistical analysis, a factorial 2X2 ANOVA test would be utilised to study the
individual effects of priming and gender on the general knowledge test. It would also be used
to understand how these two independent variables may influence the dependent variable. To
extend our replication study, we decided to look at gender as a moderator variable.
Procedure
A 15-minute survey was constructed and integrated into the course material at the
beginning of the semester. Participation was voluntary, and while there was no incentive for
completion, it was strongly recommended for better content preparation. Unlike the original
study, participants completed the survey independently and on various devices during the first
few weeks of the semester. Participants read over an information sheet that explained the
experiment’s association with PSYC375, specifically its contribution to later semester
discussions. The instructions outlined the anonymous nature of the experiment, and
participation would not alter future grades. Collected data would be analysed through a
second-party program, aggregate. The information sheet explained three questionnaire
sections, the first consisting of open-ended questions, secondly, general knowledge questions,
and finally, the third section asked participants to rate the intelligence of hooligans and
professors across a scale. Participants were provided with a statement of consent. Participants
were required to confirm their participation through a consent statement, indicating their
acknowledgment that their results would be incorporated into the experiment. The statement
of consent consisted of five statements, and participants were required to click ‘yes’ to
proceed with the survey.
Consent statements consisted of acknowledgement of a predicted 15-minute time
frame, acceptance of survey instructions, understanding of instructions, agreement to
complete the survey independently and an attention check. When asked, participants would
select ‘crocodile’ as their favourite animal. Participants were asked to imagine they were a
typical soccer hooligan or professor. They were then asked to answer open-ended questions
like a soccer hooligan or professor. There were five minutes to complete this task. Secondly,
participants were given 40 general knowledge questions; each question had a 10-second time
limit. After completing the general knowledge questions, an attention check was performed.
Participants were to select their favourite animal out of four options, one option being a
crocodile. Participants were then given a ‘yes or no’ question that asked if listing the
characteristics of a hooligan or professor changed their performance on general knowledge
questions. Participants were then asked to rate the intelligence of a typical soccer hooligan,
politician, university professor, primary school teacher and rugby fan across a scale of 0-100.
Finally, participants were asked to confirm their lack of consultation with AI or other sources
of information. Finally, participants were asked if they had taken part in a study similar to the
one that they just completed.
Results
Analysis of the manipulation check comparing the stereotypical intelligence of five
different groups found that university professors were rated as most intelligent (M = 81.8, SD
= 11.6) and Soccer hooligans were rated as least intelligent (M = 42.1, SD = 19.3) (see Table
1). These results suggest that the manipulation of the variables did, in fact, impact the
stereotypes.
Table 1
Ratings of Typical Group Intelligence
Mean
SD
Rugby Fan
52.1
17.3
Soccer Hooligan
42.1
19.3
University Professor
81.8
11.6
Politician
62.9
19.9
Primary School Teacher
66.0
13.3
An independent samples t-test was conducted to compare the test scores from the
general knowledge test to the professor-primed group (M = 22.0, SD = 4.57) and the
hooligan-primed group (M = 23.3, SD = 4.33). The t-test findings displayed a non-significant
difference in scores (t(140) = -1.72, p =.087), with a small effect size (Cohen’s d = -.29).
Therefore, priming conditions have little effect on general knowledge scores.
Following this, an ANOVA was conducted to analyse the impacts of gender and
scenario on the general knowledge test performance. The results suggested that gender
significantly influenced performance (f(1, 140) = 6.964, p = 0.009). It is important to mention
that neither scenario (f(1, 140) = 1.248, p = 0.266) nor interaction (f(1, 140) = 0.408, p =
0.524) had a significant influence on the test scores.
Discussions
Our research aimed to replicate the findings of the behavioural priming study by
Dijksterhuis and Van Knippenberg (1998), where priming participants with stereotypes
associated with intelligence correspond to their general knowledge test results. In addition to
the replication study, we extended our research by exploring whether gender affected task
performance. Our results propose successful priming, followed by activating relevant
stereotypes, may have occurred. The manipulation check suggests that the associated
stereotypes remain intact to a certain extent. This can be seen as the participants rated a
professor highly compared to a soccer hooligan. These findings suggest that priming did take
place as it activated the relevant stereotypes. These findings are similar to the original study.
On the other hand, no significant difference was found in the general knowledge test
performance between the professor and hooligan-primed groups. These results are similar to
the replication study by O’Donnell et al. (2018). This large-scale research failed to replicate
the original professor priming effect as well. This suggests that while stereotyping may occur,
it does not necessarily mean it drives behaviour change.
Several studies have proposed different theories as to why the results from the
original research have led to inconsistent replication practices. One potential explanation
could be the evolution of the stereotypes associated with the professor and soccer hooligan.
Participants in current experiments may not hold the strong stereotypes as the priming groups
compared to the original participants. In the case where priming manipulation does not alter
perceptions of intelligence, it would be expected that there would be no influence over
behaviours such as test performance (O’Donnell et al., 2018; Shanks et al., 2013).
Another area to be explored in our study is student fatigue. Participants in our
replication study consisted of students enrolled in a level 3 psychology paper. Prior research
indicates that performance on standardized tests is predicted by the time students take a test.
Sievertsen et al. (2016) found that students' test scores decreased by 0.9% of an SD every
hour later in the day. The time of day influences the success of an individual’s results because
as time passes, the overall cognitive cost required throughout the day increases and mental
resources are taxed. Compared to our study, further study could identify the time participants
completed their general knowledge questions and compare the number of correct answers
with the time of day completed. We would expect students to experience a decline in correct
answers as the day progresses.
Cognitive load can be one of the reasons why priming effects may not have been
displayed in our research. When the general knowledge questions progressively increased in
difficulty, participants may have strongly relied on their stereotype priming to make
judgements. When the cognitive load became too taxing, the participants resorted to system
one thinking, leading to a cognitive bias response (shortcuts) to reduce stress on the decisionmaking system (Kahneman, 2011). This theory is supported by Stewart & Raihani (2023),
who hypothesise that high cognitive costs, specifically ones impacted by challenging tasks,
can impact stereotype conformity. Further studies could systematically explore the interaction
effects between Task difficulty and stereotype reliance. Furthermore, research could be drawn
towards exploring whether increasing cognitive load impacts stereotype activation during
challenging tasks.
Gender Extension
It would be valuable in future research to explore the relationship between
stereotyping and non-binary people. Within the current gender extension study, gender was
limited to males and females due to the limited sample size of non-binary and other
participants. A larger sample that included a larger cohort of non-binary people would allow
researchers to explore the impact of stereotyping on minority groups already exposed to
social stereotyping. We would hypothesise that primed non-binary participants would be less
stereotypical than primed cis-gender participants.
Limitations
One particular drawback of this study is that it did not assess potential changes in
stereotypes over time. Further research should explore whether the stereotypes of the priming
conditions have shifted, which could further explain inconsistencies between current research
and the original study. Another area that future research can explore is whether utilising
modern stereotypes could result in different priming effects.
For future research, it would be beneficial to diversify the sample; individuals in the
current experiment were recruited from a single cohort; increasing the diversity of sample
size would allow for a more extensive understanding of intelligence associated with
professors and soccer hooligans. Sample diversity is significant when considering the effect
of stereotypes. Research indicates that stereotypes tend to be harmful when said group is
substantial, resulting in an unlikely chance of reciprocity evolving. However, when
stereotypical groups are positive, participants are increasingly pessimistic regarding
cooperation. Stewart & Raihani (2023) show that the tendency for stereotyping to evolve is
controlled by costs of cognition, indicating the more extensive the cost, the greater the chance
people are stereotyped.
References
Chivers, T. (2019). What’s next for psychology’s embattled field of social priming. Nature,
576, 200-202. https://doi.org/10.1038/d41586-019-03755-2
Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012). Behavioral priming: It’s all in
the mind, but whose mind? PLoS ONE, 7, e29081.
https://doi.org/10.1371/journal.pone.0029081
Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016). Comment on “Estimating the
reproducibility of psychological science”. Science, 351(6277), 1037-1037.
https://doi.org/10.1126/science.aad7243
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
Lynott, D., Corker, K. S., Wortman, J., Connell, L., Donnellan, M. B., Lucas, R. E., &
O’Brien, K. (2014). Replication of ‘Experiencing physical warmth promotes
interpersonal warmth’ by Williams and Bargh (2008). Social Psychology, 45, 216–
222. https://doi.org/10.1027/1864-9335/a000187
O’Donnell, M., Nelson, L. D., Ackermann, E., Aczel, B., Akhtar, A., Aldrovandi, S., Alshaif,
N., Andringa, R., Aveyard, M., Babincak, P., Balatekin, N., Baldwin, S. A., Banik, G.,
Baskin, E., Bell, R., Bialobrzeska, O., Birt, A., Boot, W. R., Braithwaite, S. R., … &
Zrubka, M. (2018). Registered replication report: Dijksterhuis and van Knippenberg
(1998). Perspectives on Psychological Science, 13, 268-294.
https://doi.org/10.1177/1745691618755704
Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section on
replicability in psychological science: A crisis of confidence? Perspectives on
Psychological Science, 7, 528–530. https://doi.org/10.1177/1745691612465253
Sievertsen, H. H., Gino, F., & Piovesan, M. (2016). Cognitive fatigue influences students’
performance on standardized tests. Proceedings of the National Academy of Sciences,
113(10), 2621–2624. https://doi.org/10.1073/pnas.1516947113
Stewart, A. J., & Raihani, N. (2023). Group reciprocity and the evolution of stereotyping.
Proceedings of the Royal Society B: Biological Sciences, 290(1991).
https://doi.org/10.1098/rspb.2022.1834
Appendix A Pre-registration
Project name: Call of Duty
_____________________________________________________________________
Participating Authors:
Name
Email
University Affiliation
School of psychology, speech and hearing : University of Canterbury
1) Garima Manglani
gma152@uclive.ac.nz
95029775
2) Lily Buckley
lbu56@uclive.ac.nz
43443680
_____________________________________________________________________
Questions: keep your answers concise and precise
1. Have any data been collected for this study already? (optional)
Yes. The data for the study has been collected for this replication study.
2. What's the main question being asked, or hypothesis being tested in this study? (optional)
Question: ‘does priming a stereotype (professor, hooligan) affect task performance in
line with traits (intelligence, stupidity) associated with the stereotype?’
H1: based on Dijksterhuis and van Knippenberg (1998), we hypothesise that these
stereotype primes would lead to increased or decreased performance on a general knowledge
task.
3. Describe the key dependent variable(s) specifying how they will be measured. (optional)
In the experiment, participants first spent some time describing a university professor
or soccer hooligan. After that, they completed a 40-question general knowledge test. The
general knowledge test we used was created by selecting items from the video game Who
Wants to be a Millionaire. Using this survey, task performance of participants would be
measured based on accuracy.
4. How many and which conditions will participants be assigned to? (optional)
Random allocation between the two conditions which were the professor and soccer
hooligan conditions.
5. Specify exactly which analyses you will conduct to examine the main question/hypothesis.
(optional)
We will be using a factorial 2X2 ANOVA test for statistical analysis in this between
subject’s design study. We will also be using gender as a moderator variable to determine the
strength of priming on task performance.
6. Outliers and Exclusions. Describe exactly how outliers will be defined and handled, and
your precise rule(s) for excluding observations.
-
Group 3&4 data may be excluded due to lack of sample size.
7. How many observations will be collected or what will determine the sample size? No need
to justify the decision, but be precise about exactly how the number will be determined.
(optional)
-
Psychology class Statistics 375 where participants were randomly assigned.
8. Anything else you would like to pre-register? (e.g., data exclusions, variables collected for
exploratory purposes, unusual analyses planned?) (optional)
Appendix B :Credit Statement
We would like to acknowledge people and resources responsible for their contributions for
this research project. Project administration, data collection and methodology was designed
and handled by the PSYC 375 – Intermediate Research and Statistics team from the
University of Canterbury (UC). Investigations, writing and editing were carried out by Lily
Buckley Carolan and Garima Manglani with help from the teaching team at UC. Reviewing
was carried out by Lily Buckley Carolan. Resources such as AI assessment forms, data,
writing guides and more were provided by the teaching staff at UC. Chat GPT and
Grammarly helped as resources to provide feedback along with pointing out grammatical
errors. All of them together helped to make this project a success.
Appendix C: Declaration of Use for Generative Al in Assessments form
Declaration of Use for Generative AI in Assessments
Full Name
Garima Manglani & Lily Buckley Carolan
Student ID Number
95029775, 43443680
Student Email
Gma152@uclive.ac.nz , lbu56@uclive.ac.nz
Department
School of Psychology, speech and hearing.
Course Code
PSYC-375
Assessment Name
Research Report
Acknowledgement of Assessment Submission
We, Garima Manglani & Lily Buckley Carolan hereby confirm that on 04/10/24.
1. The document I have submitted was written by me.
2. I acknowledge awareness of any updates to the generative AI tools used, up to the date of
this submission. This includes AI plug-ins or assistants included in existing programs, such as
Grammarly. I take responsibility for any fabricated references or factual errors stemming from
the use of these tools.
3. I have informed myself of the limitations and implications of using generative AI and related
technologies, including the reinforcement of biases and propensity for fabrication.
4. I have used these tools ethically, including not uploading confidential, private, personal, or
otherwise sensitive information.
5. When using AI, I have ensured that the work produced is still my own and I understand that
submitting output from a generative AI tool as my own is NOT acceptable. I understand that I
am expected to build on the output, ensuring any submissions are that of my own ideas and
knowledge.
6. To assist with maintaining academic integrity, I have appropriately acknowledged any use of
generative AI in my work (list below as applicable).
7. I acknowledge that any undeclared use of generative AI will constitute academic dishonesty
and will be dealt with according to relevant University policy.
8. I understand that I will be held accountable and liable for any academic misconduct that
arises in breach of any relevant University policy, as well as the consequences of such
infringements.
Acknowledgement of Generative AI Tools Used
Please add more rows to the table as needed so that each tool used in the creation of your
assessment submission are included in the declaration.
Generative AI Tool Used
Purpose of Use
Briefly Explain the Extent of Use
Chat GPT
Explain certain concepts and help
with flow of writing
Grammarly
Spell check
Moderately used in times of confusion
and for feedback purposes
To check the grammatical errors in
the overall assignment before
submission.
(Please List Each Separately)