ACCOUNTABILITY SYSTEMS 1
Calibrating Process and Outcome Accountability Systems in the Workplace to Meet
Fairness and Efficiency Goals
William T. Self
University of Missouri, Berkeley
Philip E. Tetlock
University of Pennsylvania
Barbara A. Mellers
University of Pennsylvania
Gregory Mitchell
University of Virginia
J. Angus D. Hildreth
University of California, Berkeley
ACCOUNTABILITY SYSTEMS 2
Abstract
This experiment explores the impact of process and outcome accountability on racial and gender
biases in a simulated hiring task. The data revealed the risks of over-correction of traditional
biases when organizations implement outcome-accountability systems in workplaces with two
properties: (a) managers endorse egalitarian or pro-minority attitudes; (b) managers recruit from
labor markets with human-capital deficits among disadvantaged groups. By contrast, processaccountability systems led to more meritocratic decisions—and also did not trigger negative
reactions observed under outcome accountability. We conclude by discussing the perils of onesize-fits-all debiasing prescriptions and the need to calibrate recommendations to specific
conditions.
Keywords:
process accountability, outcome accountability, calibration, hiring decisions, bias
ACCOUNTABILITY SYSTEMS 3
The United States is approaching the fiftieth anniversary of the landmark Civil Rights
Act of 1964. In response to tectonic shifts in public opinion on the acceptability of
discrimination (Campbell, 1947; Dovidio & Gaertner, 1986), changes in labor force
demographics, and political and legal pressure (Dobbin, 2009; Edelman, 2004), Fortune 500
companies routinely spend substantial sums on anti-discrimination and diversity programs and
have implemented a wide array of measures designed to promote equal employment opportunity
(EEO), from rank-and-file diversity training to creation of executive-level diversity officers.
Although the societal stakes are obviously high and the issues far from new (e.g.,
Campbell, 1947), we still know little about the relative effectiveness of many recommended
strategies to debias employment decisions, and less still about the interactions among these
strategies (Dobbin, Kalev & Kelly, 2007; Paluck & Green, 2009). The list of proposed remedies
is long and includes encouraging cooperative equal-status contact across group boundaries
(Binder, Zagefka, Brown, Funke, Kessler, Mummendey, Maquil, Demoulin & Leyens, 2009;
Crisp & Turner, 2009; Turner, Hewstone, Voci & Vonofakou, 2008), promoting cross-cultural
friendships (Page-Gould, Mendoza-Denton & Tropp, 2008), acknowledging race rather than
engaging in “strategic colorblindness” (Apfelbaum, Sommers & Norton, 2008; Norton,
Sommers, Apfelbaum, Pura, & Ariely 2006; Correll, Park, & Smith, 2008), confronting
offenders (Czopp, Monteith & Mark, 2006), training or practicing techniques to reduce biased
reactions (Kawakami, Dovidio, Moll, Hermsen & Russin, 2000; Plant, Peruche & Butz, 2005;
Stewart & Payne, 2008; Olson & Fazio, 2006), reducing in-group identification (Crisp & Beck,
2005), taking the perspective of others (Vorauer, Martens & Sasaki, 2009), and even giving
people sugary drinks (Gailliot, Peruche, Plant & Baumeister, 2008).
ACCOUNTABILITY SYSTEMS 4
Most organizations adopt policies designed to create EEO accountability pressures inside
the company via multiple routes, including anti-discrimination sanctions, affirmative action
programs, and holding managers responsible for following structured decision-making processes
aimed at reducing bias. As with many other interventions (Paluck & Green, 2009), there is a
paucity of evidence on the conditions under which these policies have their intended effect of
checking bias against historically disadvantaged groups as well as on the conditions under which
such policies may have unintended effects (Kalev, Kelly, & Dobbin, 2006; Bisom-Rapp, 1999;
Krawiec, 2003).
Research within the flexible-correction framework (Wegener & Petty, 1995, 1997)
underscores how difficult it can be to calibrate debiasing solutions to avoid errors of both undercorrection (failing to correct sufficiently for the original bias) and over-correction (creating a
new bias in the opposite direction). Calibration in employment decisions can be particularly
difficult for two reasons: (1) we cannot always predict what form bias will take in any given
population or context; (2) we do not know the consequences of different accountability systems
(e.g., process versus outcome accountability) on the manifestation of bias.
This article addresses this knowledge deficit by exploring the efficacy of process versus
outcome accountability interventions in checking race and sex bias in hiring decisions. Key
questions examined here include: Which type of accountability is the more effective debiasing
tool, reducing bias without simply inducing reverse-discriminatory biases in the opposite
direction? How robust are these results across variation in the difficulty of the personnel
decisions (the degree to which the decisions require balancing offsetting strengths and
weaknesses in job-relevant qualifications)? How do individual differences in ideological
ACCOUNTABILITY SYSTEMS 5
attitudes regarding equality and in motivations to achieve equality goals in hiring affect reactions
to accountability pressure?
Accountability Interventions to Correct For Biases in Hiring Decisions
Accountability research grew out of famous research demonstrations, such as the work of
Solomon Asch on conformity (1951) and Stanley Milgram on obedience (1963), of just how far
people are often willing to go to avoid disapproval and seek approval (Tetlock, 1985, 1992).
People often go to great lengths to bring their public attitudes and actions into compliance with
local norms, especially when under accountability scrutiny (Cialdini & Trost, 2010).
Furthermore, when people cannot easily guess the views of those to whom they feel accountable,
they often shift into a more self-aware mindset in which they try to anticipate objections that
reasonable others might raise to how they go about making decisions (Tetlock, 1983). This
mode of thinking, known as pre-emptive self-criticism, has in many studies checked judgment
and decision-making problems related to personnel decisions, such as over-confidence, the halo
effect, in-group favoritism, and belief perseverance (Lerner & Tetlock, 1999; Ford, Gambino,
Lee, Mayo & Ferguson, 2004; Ruscher & Duval, 1998). However, it is not clear how larger
accountability interventions in diversity programs would affect bias, particularly with the known
difficulty of calibrating such efforts.
Not all accountability pressures look the same. Specifically, researchers have
differentiated process from outcome accountability. Outcome accountability – pressure to make
the targeted demographic numbers – is a special case of accountability to an audience with
known views that should motivate decision makers to conform (demonstrate public compliance
with numerical goals) but have little effect on the rigor of underlying judgment-and-choice
processes (Tetlock, 1992). In this view, although equality might be achieved, it may be at some
ACCOUNTABILITY SYSTEMS 6
expense of job-relevant qualifications (Ferris, Frink, Bhawuk, Zhou & Gilmore, 1996). It will
also create some risk of reverse-discrimination claims (e.g., Ricci v. DeStefano, 2009),
particularly in a situation where outcome accountability attempts to increase representation of a
group with human capital deficits: in order to “make the numbers,” decision makers would have
to let inclusion trump ability. When the tradeoff between group membership and job
qualifications is zero and there is no correlation between group membership and job
qualification, the undesired side-effects of outcome accountability may be minimal. But as the
correlation becomes increasingly negative (i.e., as disparities in human capital between groups
grow, perhaps due to educational or other historical forms of structural discrimination), reverse
discrimination will become more difficult to avoid.
By contrast, process accountability – pressure to justify one’s decision process – should
focus on individual qualifications and the match between ability and job requirements and
distract them from bias-susceptible non-job-related factors. Although this form of accountability
may check stereotyping and other cognitive biases that cause disparate treatment, it will fail to
address historical effects of systemic discrimination unless affirmative pro-female or prominority factors are built into the decision process.
Because of this weakness regarding redressing systemic discrimination, many scholars
approach process-accountability correctives with skepticism and have argued that strong
outcome-accountability components are essential if corporate policies are to check bias and
encourage greater diversity and fairness in the workplace (e.g., Konrad & Linnehan, 1995). For
instance, Bielby (2008) recommended that companies implement a series of measures that
combine both process and outcome accountability: (1) “managerial equal employment
opportunity (EEO) accountability” that includes explicit evaluations for each manager’s
ACCOUNTABILITY SYSTEMS 7
contributions to meeting EEO goals, (2) “human resource process accountability” that links
managerial compensation to regular audits of whether managers are following detailed and
specific guidelines for personnel decisions, (3) “organizational EEO assessment” that involves
periodic monitoring for group disparities in hiring, pay, promotion, performance assessment, and
turnover, and (4) “workplace climate assessment” in which employees are surveyed regularly on
perceived barriers to career advancement and assessments of racial or gender disparities in
complaints. An organization that adopts such a program demonstrates its commitment to EEO,
but this program may have unintended effects because we do not have a good understanding of
the interactive effects of the program’s components.
Given the uncertainty about the effects of process accountability and dearth of data
examining the effects of process accountability on personnel processes, an emphasis on outcome
accountability is understandable: the elimination of disparities via outcome checks is, by
definition, guaranteed to increase the presence of women and minorities within an organization.
Of course, an emphasis on outcome checks and greater identity-consciousness within humanresource practices may have unintended effects. By setting identity-conscious variables on
“high” to avoid one form of discrimination, companies may trigger reverse-discrimination effects
(Matheson, Warren & Foster, 2000), perceptions of distributive and procedural injustice
(Cropanzano, Slaughter & Bachiochi, 2005; Richard & Kirby, 1998), self-doubt among
beneficiaries of the identity-conscious measures (Unzueta, Gutierrez & Ghavami, 2010; Heilman
& Alcott, 2001), leniency bias in performance ratings of females and minorities (Harber, 1998),
backlash reactions to diversity initiatives and remedial policies (Aberson & Haag, 2003; Kidder,
Lankau, Chrobot-Mason, Mollica & Friedman, 2004), and stereotypes regarding the ability of
disadvantaged groups to achieve on their own (Heilman & Welle, 2006).
ACCOUNTABILITY SYSTEMS 8
Relative to outcome accountability, process accountability should lead to more rigorous
decision-making strategies, forcing individuals to (1) remain conscious of potential biases, (2)
make slower and more deliberate decisions, and (3) weight job-relevant characteristics more
heavily to ensure that sex and race do not enter the decision. Outcome accountability, by
contrast, may result in less rigorous decision-making strategies. The emphasis on an objective
outcome – “making the numbers” – may simplify employment decisions by giving decision
makers normative permission to resolve close calls in favor of traditionally disadvantaged
groups.
Because of divergence in decision-making rigor, process and outcome accountability
have different implications for the risk of triggering over-correction. Process accountability is
more likely to correct traditional bias without triggering a mirror-image bias in the opposite
direction because it focuses attention on carefully weighing individual-level qualifications. By
contrast, outcome accountability is more likely to trigger over-correction of traditional biases,
especially in labor pools in which traditionally disadvantaged groups have human-capital
deficits, because it sets up a situation in which outcome targets and numerical goals can be used
to justify choices that minimize the role of merit.
Hypothesis 1a: Both process and outcome accountability will
reduce bias against women and minorities, but outcome
accountability will risk over-correcting in the form of bias against
white males.
Additionally, these different risks of over-correction associated with process and outcome
accountability can affect the quality of candidates selected. Relative to outcome accountability,
process accountability will motivate more in-depth processing, leading decision makers to more
ACCOUNTABILITY SYSTEMS 9
carefully weight job-relevant characteristics, and will lead to the selection of more objectively
qualified candidates. Outcome accountability will check bias by shifting evidentiary thresholds
for preferring applicants with different category membership, resulting in the selection of less
objectively qualified candidates.
Hypothesis 1b: Process and outcome accountability will produce
differences in the quality of selected candidates, with process
accountability producing more qualified candidates than outcome
accountability.
Individual Differences in Responses to Accountability Pressures
It is important to remember that people do not all share the same attitudes toward fair
treatment – and these attitudinal differences could translate into differential responses to
accountability pressures. Of special relevance here is work done by Dunton and Fazio (1997)
showing that individuals differ in their motivation to inhibit prejudiced responses to minorities.
The motivation-to-control-prejudiced-reactions (MCP) scale is designed to measure individual
desires to be seen as unbiased. The MCP has also recently been found even to predict how
susceptible people are to unconscious or implicit biases. Low scorers are more likely to display
unconscious-bias effects, whereas high scorers are less likely to show unconscious biases even
when under no situational pressure to be fair – and sometimes show over-correction for biases
when under such pressure (Greenwald, Poehlman, Uhlmann & Banaji, 2009; Hausman & Ryan,
2004; Maddux, Barden, Brewer & Petty, 2005; Olson & Fazio, 2004). Additionally, the MCP
has been shown to predict ratings of minority job candidates and hiring decisions (Plant &
Devine, 2001) and to moderate the effect of cognitive depletion on unintended discrimination
(Park, Glaser & Knowles, 2008; Glaser & Knowles, 2008).
ACCOUNTABILITY SYSTEMS 10
Individuals who endorse equality goals and try harder to inhibit biases should be more
responsive to accountability pressures that encourage such aims. The flexible-correction model
(Wegener & Petty, 1995) posits that individuals often can be sensitized to biases with the
potential to sway their decisions and be motivated to “correct” their decisions in order to
eliminate those biases. From a flexible-correction perspective, strong egalitarians may be able to
identify and inhibit their biases, leading to hiring decisions free from traditional sex or racebased discrimination; however, to the extent that they over-estimate their susceptibility to bias in
a given situation, they may over-correct and favor female or black candidates over male or white
candidates, particularly when encouraged by strong accountability pressures that legitimate those
efforts. In effect, some people are more likely to exchange one bias for another.
Hypothesis 2a: Insofar as those who are unsympathetic to
egalitarian goals under-estimate their propensity to bias, they will
be more likely to under-correct their employment decisions,
favoring male and white candidates over female and black
candidates.
Hypothesis 2b: Insofar as those who strongly endorse egalitarian
goals over-estimate their propensity to bias, they will be more
likely to over-correct their employment decisions, favoring female
and black candidates over male and white candidates.
Organizational Consequences of Accountability-Promoting Policies
Accountability pressures – particularly strong forms of outcome accountability – may
also influence how decision makers feel about their company. Schoorman, Mayer, and Davis
(2007) argued that control systems – which include accountability policies – can be effective
ACCOUNTABILITY SYSTEMS 11
means of managing organizational risk. However, they cautioned that control systems perceived
to be too strong can impair trust between organizations and employees by limiting employee
decisions and creating a situation where trustworthy behavior could be attributed to the controls
rather than to the employee. Personnel decision makers may perceive strong (outcomeconstraining) forms of accountability as implying that they are incapable of or unwilling to make
fair hiring decisions, thus eroding trust (e.g., Bartlett, 2009; McEvily, Perrone & Zaheer, 2003;
Sitkin & George, 2005).
Hypothesis 3: Policies that create strong outcome accountability
pressures will lead to lower feelings of trust among those making
hiring decisions.
Method
Participants
Two hundred and ninety-seven undergraduate business students participated in a
laboratory experiment in exchange for nominal payment. Of those participants, eight were
eliminated for not following instructions. Two participants declined to report their sex and race;
84 of the remaining participants were male (29 percent), 203 were female (71 percent), 70 were
white (24 percent), and 186 were Asian (65 percent).
Experiment Design and Procedure
Participants were asked to assume the role of a human-resources manager at a large
company. They were informed that part of their job involved screening potential candidates for
position openings at the company: they were to determine which candidates would be invited to
interview for open positions. In the study task, participants were told they would review
ACCOUNTABILITY SYSTEMS 12
summaries of 75 applicants for an open position as an entry-level manager at a regional office.
No other information about the organization or position was provided.
The experimental design was based on models of motivated reasoning (e.g., Dovidio &
Gaertner, 2000; Kunda, 1990), which posit that, even though most people embrace EEO norms
in principle, they often make judgments that disadvantage women and minorities when they have
discretion in weighting complex decision cues. Additionally, the experimental task was created
to as closely as possible mirror the sorts of triage activities in which human resource
professionals and hiring managers engage when screening resumes (see Messner, Wänke, &
Weibel, 2011, for a similar methodology). Previous research suggests that screening large
numbers of applications to make interview decisions represents the key challenge for personnel
decision makers – and getting past that screening represents the key challenge for applicants
(e.g., Brown & Campion, 1994). Furthermore, the research confirms that decision makers juggle
diverse and multi-faceted information they receive about applicants – some job-relevant and
some not – and use it to varying extents in making triage decisions (e.g., McKinney, Carlson,
Macham, D’Angelo, & Connerley, 2003).
Accordingly, participants were given numerous details about each candidate, ranging
from innocuous identifiers to job-relevant qualifications to potentially bias-provoking
demographic traits: participants saw (1) a name for each candidate (pre-screened to be perceived
as equally common and desirable and unassociated with any specific racial or socio-economic
group), randomly assigned to candidates for each participant; (2) a medium-sized California city
from which the candidate originated, randomly assigned to candidates for each participant; (3)
the number of years of work experience, ranging from three to six years, randomly assigned to
candidates for each participant; (4) scores on separate tests of technical skills and teamwork
ACCOUNTABILITY SYSTEMS 13
skills, consisting of “excellent,” “very good,” “good,” “average,” “poor,” “very poor,” or
“unsatisfactory” for each test, both of which they were told were important skills for the position;
(5) a photo identifying the candidate’s sex and race (extensively pre-screened to be perceived as
of comparable age, equally attractive and professional, and clearly black or white and male or
female), randomly assigned to candidates for each participant (although a given candidate
remained in the same sex and race category for all participants, the specific photograph attached
to the candidate was randomly assigned for each participant).
The study task consisted of sorting the 75 candidates into seven interview categories,
following a Q-sort methodology used to force distinctions between candidates (Block, 1961) and
replicate the way actual human-resource managers triage candidates: (1) “definitely interview”
(eight candidates allowed), (2) “very likely to interview” (ten candidates allowed), (3) “likely to
interview” (12 candidates allowed), (4) “possibly interview” (15 candidates allowed), (5)
“unlikely to interview” (12 candidates allowed), (6) “very unlikely to interview” (ten candidates
allowed), (7) “definitely not interview” (eight candidates allowed). Following this task,
participants completed a questionnaire about the experience.
Accountability Manipulation
Along with the task instructions, participants received one of three accountability
manipulations. All participants were told that the interview decisions were theirs to make and
that “your boss is leaving it up to you to decide how much weight to give each factor.” In the
no-accountability condition, participants additionally were told, “Your judgments will be
completely anonymous and in no way traceable to you.”
In the process-accountability condition, participants were told that they would have to
explain in detail the process by which they made their decisions to an experimenter “who
ACCOUNTABILITY SYSTEMS 14
suspects that discrimination remains a serious problem in workplaces and who will be checking
to ensure that each candidate was judged consistently and fairly and that the principles of equal
employment opportunity are respected.”
Finally, in the outcome-accountability condition, participants were told that they would
have to justify their decisions to an experimenter “who suspects that discrimination remains a
serious problem in workplaces and who will be comparing the ratio of women and minorities to
be interviewed to the female and minority applicants in the overall labor pool – in this case 60%.
The experimenter will scrutinize decisions especially closely when the decisions produce fewer
female and minority interviews than the target goal number.”
Additionally, in order to make the accountability instructions believable, the
experimenter-to-subject ratio never exceeded 1:2.
Labor-Pool Manipulation
The reality of personnel decisions is that context matters, particularly with regard to how
job-related qualifications (applicants’ and potential applicants’ human capital) are distributed
within the labor market and the degree to which they are correlated with membership in various
demographic groups (Heckman, 1993; Heckman, 1999; Sackett, Schmidt, & Kabin, 2001). It is
one thing to ask organizations to achieve representation targets when all demographic groups are
equally qualified; that task becomes more onerous when qualified candidates are overrepresented in some groups and scarce in other. Although specific hypotheses regarding the
effect of labor pools were not offered earlier, in the interest of realism and external validity we
examine the effects of accountability both in labor markets where qualifications are equally
distributed across all demographic groups and in labor markets where white, male candidates are
disproportionately more qualified.
ACCOUNTABILITY SYSTEMS 15
In order to achieve these conditions, the 75 candidates were divided into four
demographic groups: 30 white men, 15 white women, 15 black men, and 15 black women. Each
candidate was systematically assigned one of seven technical-skill scores and one of seven
teamwork-skill scores (see Figures 1 and 2). The middle three scores for each ability dimension
(“good,” “average,” and “poor”) represent a “common set” of candidates that remained the same
across all experimental conditions for analytical purposes (e.g., Mellers & Hartka, 1988). This
common set of candidates consisted of 24 white men, nine white women, nine black men, and
nine black women – identical for all participants – and was used for most analyses.
In the equal-advantage condition, the non-common-set candidates – six candidates from
each of the four demographic groups – were assigned technical-skill and teamwork-skill scores
in such a way that each sex and race had equal scores on average (correlation of 0 between
sex/race and technical/teamwork-skill scores) (see Figure 1). In other words, no sex or race
possessed a performance advantage in the aggregate.
[Insert Figure 1 about here]
In the majority-advantage condition, the non-common-set candidates were assigned
technical-skill and teamwork-skill scores in such a way that the male candidates and the white
candidates had higher scores on average than did the female candidates and the black candidates
(correlation of .21 between sex/race and technical/teamwork-skill scores) (see Figure 2). In this
condition, male candidates and white candidates possessed a performance advantage over female
candidates and black candidates in aggregate. Note that the scores of the common-set candidates
remained unchanged in each condition, such that any labor-pool differences resulted from the
existence of a larger skewed labor pool and not from differences in the abilities of the candidates
included in an analysis.
ACCOUNTABILITY SYSTEMS 16
[Insert Figure 2 about here]
Egalitarianism Scale
At least a day before participating in the experiment, participants completed an online
survey. From this survey, we created an Egalitarianism Scale, derived from four component
scales.
First, participants completed a six-item (α=.78) scale that included questions about how
they make tradeoffs between equality-maximization goals and merit-rewarding goals (see Table
1). Participants who scored high on this scale weighed merit more heavily than equality in
making selection decisions.
[Insert Table 1 about here]
Second, participants completed the Motivation to Control Prejudiced Reactions Scale (17
items, α=.79) (Dunton & Fazio, 1997). Participants who scored high on this scale were more
motivated to control the expression of prejudice in general and race-based prejudice in particular.
Third, participants completed the Modern Sexism Scale (8 items, α=.80) (Swim, Aikin,
Hall, & Hunter, 1995). Participants scoring high on this scale are more likely to acknowledge
ongoing discrimination against women and to endorse support for policies that advance women.
Finally, participants completed a one-item question about their libertarian ideological
tendencies – “How libertarian do you consider yourself to be?” – on a 1-7 Likert-type response
scale (1= “not at all;” 7= “extremely”).
Participant scores from the four measures were entered into a factor analysis. Table 2
reports the results of a Varimax rotation of the principal-components solution. Based on these
results, we computed regression factor scores for each participant, standardized to a mean of 0
and a standard deviation of 1, to represent scores on a combined Egalitarianism Scale.
ACCOUNTABILITY SYSTEMS 17
[Insert Table 2 about here]
Dependent Variables
Interview-decision ratings
Participants assigned each of 75 candidates to one of the seven interview-decision
categories explained above. This gave each candidate an interview-decision score on a scale
from 1 (“definitely interview”) to 7 (“definitely not interview”).
Trust perceptions
Participants responded to five items (α=.59) measuring the extent to which they felt
trusted by the company to make good and fair decisions (7-point scale: 1=strongly disagree and
7=strongly agree): (1) “I felt trusted to select the best candidate for the position;” (2) “I felt
overly constrained in choosing between the candidates” (reverse coded); (3) “Management did
not trust me to review the candidates fairly” (reverse coded); (4) “I was not given enough credit
that I would fairly choose between candidates” (reverse coded); (5) “Management would support
any interview decision I made.”
Results
To explore the effects of accountability and labor pool (between-subjects Accountability
(3) x Labor Pool (2)) on the same set of candidates, we initially focused on interview decisions
about candidates with technical and teamwork-skill scores that were common to all six
conditions. This set of candidates was based on a within-subjects factorial design of Sex (2) x
Race (2) x Technical Score (3) x Teamwork Score (3). Technical and teamwork-skill scores
included three levels each: “good,” “average,” and “poor.” There were more white males (either
2 or 4) than minority-group members in each cell of this common set, so we averaged over
identically-qualified white male candidates. After this averaging, there were four candidates
ACCOUNTABILITY SYSTEMS 18
(one from each combination of race and sex) in each of the nine combinations of technical and
teamwork skills.
Preliminary Data Analysis
In all six combinations of labor pool and accountability, candidates with higher technicalskill scores were more likely to be interviewed than those with lower scores (p < .01 for every
combination) and candidates with higher teamwork-skill scores outperformed those with lower
scores (p < .01 for every combination). This pattern indicates that, across all conditions,
participants attended to and were guided by job-relevant attributes.
Second, we looked at whether bias existed in this population’s decisions. In the equaladvantage labor-pool, participants preferred female candidates (mean = 4.00, sd = .02) over male
candidates (mean = 4.05, sd = .01; F(1,30) = 5.85, p < .05, η2 = .16). Although there was no
significant main effect of race in the equal-advantage labor pool, there was a significant threeway interaction among candidate’s race, technical skills, and teamwork skills (F(4,120) = 3.73, p
< .01, η2 = .11). Participants assigned better ratings to black candidates than to equally-qualified
white candidates when teamwork and technical scores were inconsistent. Black candidates with
good technical scores but poor teamwork scores (or with poor teamwork scores but good
technical scores) received better ratings than white candidates with the same objective
qualifications. This strategy suggests that, consistent with the symbolic anti-racism hypothesis
(1b), blacks received more “benefit of the doubt” in more ambiguous cases (see also Norton,
Vandello & Darley, 2004).
In the majority-advantage labor pool, female candidates in the common set received
marginally higher interview ratings than equally-qualified males. The means for females and
ACCOUNTABILITY SYSTEMS 19
males were 3.98 (sd = .02) and 4.03 (sd = .01), respectively (F(1,31) = 3.83, p < .10, η2 = .11).
There were no significant effects due to race (F(1,31) = .48, n/s).
Hypothesis 1: The Influence of Process and Outcome Accountability
Hypothesis 1a stated that both process and outcome accountability would reduce antifemale and anti-minority bias but that outcome accountability would be more likely to trigger
over-correction. We examined this hypothesis by comparing the minority groups to white males
under different accountability conditions.
Figures 3 and 4 show the differential treatment of the minority groups relative to white
males for the common set of candidates in the equal-advantage labor pool on the left and the
majority-advantage labor pool on the right. For purposes of comparison, blue bars show noaccountability conditions, red bars show process-accountability conditions, and green bars show
conditions with outcome accountability. Figure 3 presents the percentage of times a minority
candidate was preferred to equally-qualified white males out of fifteen comparisons in the equaladvantage labor pool and nine comparisons in the majority-advantage labor pool. If it was
equally likely that the minority group candidate was favored, percentages of pro-minority ratings
would be 50%.
[Insert Figure 3 and Figure 4 about here]
In the process-accountability condition, there were no significant differences in the
treatment of each minority group relative to equally-qualified white males in either of the two
labor pools. All six comparisons did not differ from 50%. However, in the outcomeaccountability conditions, all three groups of minority candidates in the equal-advantage labor
pool and two of the three minority-group candidates in the majority-advantage labor pool were
significantly more likely to be selected over equally-qualified white males. Binomial
ACCOUNTABILITY SYSTEMS 20
probabilities were .04, .0005, and .003 for white females, black females, and black males in the
equal-advantage pool and .002, .002, and .07 for the same three groups in the majority-advantage
pool. Consistent with Hypothesis 1a, process accountability greatly reduced inequalities without
triggering a mirror-image bias against white males, whereas outcome accountability triggered
over-correction.
Figure 4 examines average differences in the size of the pro-minority advantage. This
analysis differs from the previous analysis because it focuses on the magnitude of a minority
group’s advantage or disadvantage (not the frequency) relative to white males. With process
accountability, there were no significant differences in either labor pool between interview
ratings of each of the three minority groups relative to equally-qualified white males. However,
under outcome accountability, all three minority groups received higher average ratings than
white males in the equal-advantage labor pool (t(14) = 4.31, 4.73, and 3.25 for black females,
black males, and white females, respectively) and in the majority-advantage labor pool (t(8) =
10.35, 2.16, and 4.51 for black females, black males, and white females, respectively). Outcome
accountability led to over-correction across labor-pool conditions.
Hypothesis 1b posited that accountability would have consequences for the quality of
candidates selected by participants. Process accountability was expected to induce more indepth processing and result in the selection of better-qualified candidates, while outcome
accountability was predicted to shift attention toward demographic characteristics and away from
job-relevant qualifications. We tested this hypothesis by computing the average ability score for
each candidate in each interview category (since we did not specify that either score was more
important than the other, we treated scores as equally important). Better candidates were those
with higher average ability scores, and more advantageous interview categories are represented
ACCOUNTABILITY SYSTEMS 21
by smaller numbers. Comparisons were made between outcome and process accountability
within a given labor pool (in which qualifications of minorities and majorities were held
constant) but not across labor pools. Support for Hypothesis 1b would be found in higher
average ability scores in the smaller categories (1, 2, and 3), and lower average ability scores in
larger categories (5, 6, 7) for process accountability. That is, Hypothesis 1b posits that, relative to
outcome accountability, process accountability will result in higher-qualified candidates being
more likely to be interviewed and less-qualified candidates less likely to be interviewed.
Results are displayed in Table 3. In several interview categories, there were no
significant changes in qualities of candidates due to accountability. However, when significant
differences in abilities emerged, results were consistent with Hypothesis 1b. When majority and
minority candidates had the same advantages in the labor pool, process accountability resulted in
better-qualified candidates being placed in category 2 and worse-qualified candidates being
placed in category 6. When majority candidates had the advantage in the labor pool, process
accountability led to better-qualified candidates being placed in category 2. In sum, participants
held accountable for the process (versus the outcome) gave better-qualified candidates a greater
chance of being interviewed and worse-qualified candidates a lower chance of being
interviewed.
[Insert Table 3 about here]
Hypothesis 2: Individual Differences and Hiring Decisions
A pro-female advantage emerged as the most persistent form of bias, arising across every
level of accountability and labor-pool conditions, except when participants were held
accountable for their process of selection in the majority-advantage labor pool. Hypothesis 2
predicted that low egalitarians would prefer male and white candidates over female and black
ACCOUNTABILITY SYSTEMS 22
candidates (hypothesis 2a) and high egalitarians would prefer female and black candidates over
male and white candidates (hypothesis 2b). To explore the causal drivers of the observed profemale bias and to test Hypothesis 2, we used a regression analysis to predict interview ratings
from participants’ egalitarianism score, candidate sex, and the interaction between these
variables, controlling for accountability condition, labor pool, candidate race, and candidate
technical and teamwork-skills scores. The interaction between candidate sex and egalitarianism
score was a significant predictor of the participant’s interview rating. High-egalitarian
participants rated female candidates more positively than male candidates, and low-egalitarian
participants rated male and female candidates roughly the same (β = -.02, t = -2.01, p < .05),1
providing partial support for Hypothesis 2b (relative to the pro-female bias that emerged in the
study). The form of this interaction (Figure 5) is displayed using the method suggested by
Cohen, Cohen, West and Aiken (2003) to plot the predicted effects on interview ratings for high
egalitarians (one standard deviation above the mean) and low egalitarians (one standard
deviation below the mean) with male (0) and female (1) candidates.
[Insert Figure 5 about here]
An alternative explanation for the emergence of pro-female bias is suggested by research
on in-group preference (Brewer, 1979; Tajfel, 1982): female participants prefer female
candidates over male candidates and, because there were more female participants in our sample,
there was a bias in favor of female candidates. To test this idea, we predicted interview ratings
from participant sex, candidate sex, and the interaction, again controlling for accountability
condition, labor pool condition, candidate race, and candidate technical and teamwork-skills
scores. Neither participant sex (β=.00, t=.45, n/s) nor the interaction between participant sex and
1
The regression analysis confirmed that female candidates received better ratings than male candidates (β = -.03, t
= -4.77, p < .01), and there was not a significant main effect for participant egalitarianism score (β = .01, t =.99,
n/s).
ACCOUNTABILITY SYSTEMS 23
candidate sex (β=-.01, t=-1.14, n/s) significantly predicted placement of candidates into
interview categories, contrary to an in-group-bias explanation for the pro-female bias found in
the experiment.
Hypothesis 3: Accountability and Trust
Hypothesis 3 stated that strong outcome-accountability pressures would lead to lower
feelings of trust among those making decisions. Consistent with Hypothesis 3, participants felt
less trusted under outcome accountability (mean = 4.76, sd = .10) than under no accountability
(mean = 5.06, sd = .10; F(1,123) = 4.28, p < .05, η2 = .03). This result held true in the majorityadvantage labor pool (F(1,123) = 3.97, p < .05, η2 = .03) but not in the equal-advantage labor
pool (F(1,123) = .91, n/s). Trust levels under process accountability (mean = 4.99, sd = .10) did
not differ from those under no accountability (mean = 5.06, sd = .10) overall (F(1,122) = .23,
n/s) nor with respect to the equal-advantage labor pool (F(1,122) = .00, n/s) or the majorityadvantage labor pool (F(1,122) = .45, n/s).
Discussion
Our findings shed new light on the risks and benefits of alternative accountability
systems for debiasing personnel decisions and achieving alternative political conceptions of
equal-employment opportunity and social justice (Tetlock & Mitchell, 2009). First, under
outcome accountability, participants over-corrected in their biases more than under process
accountability or no accountability, creating bias against white-male candidates. This bias
became especially pronounced in the experimental condition in which traditionally
disadvantaged minority groups suffered from human-capital deficits. Second, the pro-minority
biases under outcome accountability resulted in the selection of less-qualified candidates across
labor pools. These results are consistent with research suggesting that outcome accountability
ACCOUNTABILITY SYSTEMS 24
results in more cursory information processing and a shift in attention from job-relevant
qualifications to job-irrelevant demographic characteristics. Also, participants reacted negatively
to strong accountability instructions, reporting feeling less trusted in the outcome-accountability
condition. These results suggest that strong accountability pressures can carry a social cost for
organizations.
For organizations seeking to debias personnel processes rather than achieve hiring
targets, process accountability is likely to provide a more effective intervention. Our processaccountability condition resulted in better calibrated bias mitigation, reducing bias without
prompting over-correction. This calibration had performance advantages with process
accountability yielding a better-qualified pool of selected candidates. Additionally, unlike under
outcome accountability, process accountability did not create a sense of distrust. Our study was
designed to examine the basic relation of accountability and trust rather than to identify the
nature of that relation; it may be that outcome-accountability pressure is aversive because it
signals organizational views of the decision-maker or because it violates meritocracy-grounded
views of informational and interactional justice (Colquitt & Rodell, 2011). .
These findings build upon our understanding of flexible correction by isolating several
factors that determine the success of accountability as a debiasing intervention. Among other
things, we need to know how strong the informational and normative components of the
outcome-accountability pressures will be, how pervasive the predispositions to discriminate
against particular demographic groups are, and how large the pre-existing demographic-group
differences in human capital are.
Limitations and Future Directions
ACCOUNTABILITY SYSTEMS 25
Although this study advances our understanding of how different accountability
interventions affect bias in personnel hiring, there are a few study limitations that should be
considered when interpreting our findings. First, we utilize a student population rather than
human resource professionals, hiring managers, or other people with experience making the sort
of interview-triage decisions our participants were asked to make. Future researchers may wish
to replicate our experimental paradigm using such working populations in order to understand
whether expertise – or at least experience – with the task alters the results. We predict that the
pattern of results would be robust to participant populations; however, the magnitude and
direction of bias is likely to change with organizational context, legal climate, and applicant mix.
Likewise, accountability research suggests a high level of generalizability in the ways in which
people are influenced by accountability pressure, making it unlikely that a student sample would
be affected in a substantially different way by the study conditions. Finally, our student
population included a wide range of individual differences along the egalitarianism dimension –
as would be expected in the wider public – to identify ways in which person-by-situation
interactions influenced responses to debiasing interventions and task conditions.
Second, we recognize that it is possible to agree about the need to study the consequences
of debiasing interventions but still have concerns about the particular laboratory experiment
presented here. Critics could argue, for instance, that although the current research paradigm may
satisfy the logical requirements for testing aversive bias theories, it does not satisfy the
psychological requirements. In this vein, they could maintain that it is too easy for participants
in complex repeated-measures designs of the current sort to figure out which hypotheses are
being tested – and to position themselves in a socially desirable light. According to this logic,
participants in our control condition were only fair-minded because many were already
ACCOUNTABILITY SYSTEMS 26
predisposed to be egalitarian and virtually all were aware of ambient egalitarian institutional
norms and aware they were making judgments under a behavioral-science microscope. In this
view, we need to create a tougher series of tests that draw on more representative samples of
participants, that rely on between-subjects rather than on repeated-measures designs, and perhaps
that give participants an opportunity, prior to making simulated personnel decisions, to
demonstrate their "moral credentials" (Monin & Miller, 2001), thereby further reducing the level
of evaluation apprehension arguably present even in control conditions.
Unfortunately for managers seeking guidance, there is no objectively right answer to the
question of how difficult it should be to “pass” laboratory experiments or organizational training
exercises. One can defend the current paradigm on mundane-realism grounds by arguing that:
(1) personnel decision-makers in large companies live in a largely repeated-measures world
similar to that of the current study where they focus their attention on a set of particular jobrelevant factors measured in more and less objective ways; (2) a significant fraction of our
participants had low, not high, scores on our individual difference measure of egalitarianism; (3)
personnel decision-makers typically go about their work without previously being exposed to
moral-credentialing manipulations. The only conclusion we can draw here with reasonable
certainty is that no single paradigm will suffice for the multifaceted challenge of assessing the
degree to which subjective personnel decision-making is untainted by factors outside of
legitimate, job-related qualifications.
Practical Considerations
The practical import of the current findings is the need to calibrate debiasing
manipulations to the types of personnel decisions that managers are making and to the types of
populations from which both managers and employees are drawn. If companies are operating in
ACCOUNTABILITY SYSTEMS 27
a labor pool in which there are large human-capital deficits in traditionally disadvantaged
communities, if the companies' managers score high on indicators of egalitarian values (and
display little or no baseline bias against traditionally disadvantaged groups), and if the
companies embrace outcome accountability, our data suggest that these companies will pay a
potentially high price in workforce qualifications and potential efficiency – as well as a price in
employee dissatisfaction. And if such companies assume that outcome accountability is moving
them into a zone of optimal self-correction, a meritocratic world in which decisions are based
solely on job-relevant qualifications, the companies are making a miscalculation. Conversely, if
companies are operating in labor pools in which there are not significant human-capital
differences across groups, if the companies' managers unapologetically subscribe to the implicit
theory that there are large human-capital deficits across groups, and if the companies embrace
only perfunctory process accountability, these companies risk the opposite mistake – and getting
the worst of both worlds in the efficiency-equality tradeoff.
It is unfortunate that so little of the scientific literature on debiasing judgment has found
its way into the actual practices of personnel decision-making and that we know so little about
the effectiveness of debiasing interventions (Cohen, 2007). As noted at the outset, large
companies in North America and Western Europe are under sustained legal and social pressure
to demonstrate that they have created EEO workplaces. Unfortunately, there is a mismatch
between the societal accountability pressures operating on organizations and our understanding
of the individual accountability pressures facing hiring managers that are essential for
establishing and maintaining genuine equal-employment opportunity. Companies embrace
interventions with the laudable goal of reducing bias and ensuring fairness but with little
evidence about what works and about the unintended consequences (e.g., over-correction). Neo-
ACCOUNTABILITY SYSTEMS 28
institutionalist theory tells us what to expect when organizations confront accountability
demands that they do not have the technical wherewithal to achieve (Suchman, 1995): We
should expect organizations to engage in symbolic posturing, to be guided not by scientifically
validated methodologies but rather by hunches about which best practices, consulting groups,
and slogans will strike the most resonant chords among regulators, judges, and potential
plaintiffs.
ACCOUNTABILITY SYSTEMS 29
REFERENCES
Aberson, C., & Haag, S. C. (2003). Beliefs about affirmative action and diversity and their
relationship to support for hiring policies. Analyses of Social Issues and Public Policy, 3,
121-138.
Asch, S. E. (1951). Effects of group pressure upon the modification and distortion of judgment.
In H. Guertzkow (Ed.), Groups, leadership, and men. Pittsburgh, PA: Carnegie Press.
Apfelbaum, E. P., Sommers, S. R., & Norton, M. I. (2008). Seeing race and seeming racist?
Evaluating strategic colorblindness in social interaction. Journal of Personality and
Social Psychology, 95, 918-932.
Bartlett, K.T. (2009). Making good on good intentions: The critical role of motivation in
reducing implicit workplace discrimination. Virginia Law Review, 95, 1893-1972.
Bielby, William T. (2008). Promoting racial diversity at work: Challenges and solutions. In A. P.
Brief (Ed.), Diversity at work (pp. 53-88). NewYork: Cambridge University Press.
Bisom-Rapp, S. (1999). Bulletproofing the workplace: Symbol and substance in employment
discrimination law practice. Florida State University Law Review, 26, 959-1047.
Binder, J., Zagefka, H., Brown, R., Funke, F., Kessler, T., Mummendey, A., Maquil, A.,
Demoulin, S., & Leyens, J. P. (2009). Does contact reduce prejudice or does prejudice
reduce contact? A longitudinal test of the contact hypothesis in three European countries.
Journal of Personality and Social Psychology, 96, 843-856.
Block, J. (1961). The Q-sort method in personality assessment and psychiatric research.
Springfield, IL: Charles C. Thomas.
Brewer, M. B. (1979). In-group bias in the minimal intergroup situation: A cognitivemotivational analysis. Psychological Bulletin, 86, 307-324.
Brown, B. K., & Campion, M. A. (1994). Biodata phenomenology: Recruiters’ perceptions and
use of biographical information in resume screening. Journal of Applied Psychology, 79,
897-908.
Campbell, A. A. (1947). Factors associated with attitudes toward Jews. In T. Newcomb & E.
Hartley (Eds.), Readings in social psychology (pp. 518-527). New York: Holt.
Cialdini, R.B., & Trost, M.R. (2010). Social influence: Social norms, conformity, and
compliance. In D.T. Gilbert, S.T. Fiske & G. Lindzey (Eds.), The handbook of social
psychology (4th edition, Vol. II, pp. 41-88). Boston, MA: McGraw-Hill.
Cohen, D.J. (2007). The very separate worlds of academic and practitioner publications in
human resource management: Reasons for the divide and concrete solutions for bridging
the gap. Academy of Management Journal, 50, 1013-1019.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation
analysis for the behavioral sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
Colquitt, J.A., & Rodell, J.B. 2011. Justice, trust, and trustworthiness: A longitudinal analysis
integrating three theoretical perspectives. Academy of Management Journal, 54, 11831206.
Correll, J., Park, B., & Smith, J. A. (2008). Colorblind and multicultural prejudice reduction
strategies in high-conflict situations. Group Processes & Intergroup Relations, 11, 471491.
Crisp, R. J., & Beck, S. R. (2005). Reducing intergroup bias: The moderating role of ingroup
identification. Group Processes & Intergroup Relations, 8, 173-185.
ACCOUNTABILITY SYSTEMS 30
Crisp, R. J., & Turner, R. N. (2009). Can imagined interactions produce positive perceptions?
Reducing prejudice through simulated social contact. American Psychologist, 64, 231240.
Cropanzano, R., Slaughter, J. E., & Bachiochi, P. D. (2005). Organizational justice and black
applicants’ reactions to affirmative action. Journal of Applied Psychology, 90, 11681184.
Czopp, A. M., Monteith, M. J., & Mark, A. Y. (2006). Standing up for a change: Reducing bias
through interpersonal confrontation. Journal of Personality and Social Psychology, 90,
784-803.
Dobbin, F. (2009). Inventing equal opportunity. Princeton, NJ: Princeton University Press.
Dobbin, F., Kalev, A., & Kelly, B. (2007). Diversity management in corporate America.
Contexts, 6, 21-28.
Dovidio, J. F., & Gaertner, S. L. (1986). Prejudice, discrimination, and racism: Historical trends
and contemporary approaches. In J. F. Dovidio & S. L. Gaertner (Eds.), Prejudice,
discrimination, and racism (pp. 1-34). New York: Academic Press.
Dovidio, J. F., & Gaertner, S. L. (2000). Aversive racism and selection decisions: 1989 and
1999. Psychological Science, 11, 319-323.
Dunton, B. C., & Fazio, R. H. (1997). An individual difference measure of motivation to control
prejudiced reactions. Personality and Social Psychology Bulletin, 23, 316-326.
Edelman, L. B. (2004). Rivers of law and contested terrain: A law and society approach to
economic rationality. Law & Society Review, 38, 181-198.
Ferris, G. R., Frink, D. D., Bhawuk, D. P. S., Zhou, J., & Gilmore, D. C. (1996). Reactions of
diverse groups to politics in the workplace. Journal of Management, 22, 23-44.
Ford, T. E., Gambino, F., Lee, H., Mayo, E., & Ferguson, M. A. (2004). The role of
accountability in suppressing managers’ preinterview bias against African-American
sales job applicants. Journal of Personal Selling & Sales Management, 24, 113-124.
Gailliot, M. T., Peruche, B. M., Plant, E. A., & Baumeister, R. F. (2008). Stereotypes and
prejudice in the blood: Sucrose drinks reduce prejudice and stereotyping. Journal of
Experimental Social Psychology, 45, 288-290.
Glaser, J., & Knowles, E. D. (2008). Implicit motivation to control prejudice. Journal of
Experimental Social Psychology, 44, 164-172.
Greenwald, A. G., Poehlman, T. A., Uhlmann, E. L., & Banaji, M. R. (2009). Understanding and
using the implicit association test: III. Meta-analysis of predictive validity. Journal of
Personality and Social Psychology, 97, 17-41.
Harber, K. D. (1998). Feedback to minorities: Evidence of a positive bias. Journal of Personality
and Social Psychology, 74, 622-628.
Hausman, L. R. M., & Ryan, C. S. (2004). Effects of external versus internal motivation to
control prejudice on implicit prejudice: The mediating role of efforts to control
prejudiced responses. Basic and Applied Social Psychology, 26, 215-225.
Heckman, J. J. (1993). What has been learned about labor supply in the past twenty years? The
American Economic Review, 83, 116-121.
Heckman, J. J. (1999). The economics and econometrics of active labor market programs. The
Handbook of Labor Economics, 3, 1865-2097.
Heilman, M. E., & Alcott, V. B. (2001). What I think you think of me: Women’s reactions to
being viewed as beneficiaries of preferential selection. Journal of Applied Psychology,
86, 574-582.
ACCOUNTABILITY SYSTEMS 31
Heilman, M. E., & Welle, B. (2006). Disadvantaged by diversity: The effects of diversity goals
on competence perception. Journal of Applied Social Psychology, 36, 1291-1319.
Kalev, A., Kelly, E., & Dobbin, F. (2006). Best practices or best guesses? Assessing the efficacy
of corporate affirmative action and diversity policies. American Sociological Review, 71,
4, 589-617.
Kawakami, K., Dovidio, J. F., Moll, J., Hermsen, S., & Russin, A. (2000). Just say no (to
stereotyping): Effects of training in negation of stereotypic associations on stereotype
activation. Journal of Personality and Social Psychology, 78, 871-888.
Kidder, D. L., Lankau, M. J., Chrobot-Mason, D., Mollica, K. A., & Friedman, R. A. (2004).
Backlash toward diversity initiatives: Examining the impact of diversity program
justification, personal and group outcomes. International Journal of Conflict
Management, 15, 77-102.
Konrad, A.M., & Linnehan, F. (1995). Formalized HRM structures: Coordinating equal
employment opportunity or concealing organizational practices? Academy of
Management Journal, 38, 787-829.
Krawiec, K. D. (2003). Cosmetic compliance and the failure of negotiated governance.
Washington University Law Quarterly, 81, 487-544.
Kunda, K. (1990). The case for motivated reasoning. Psychological Bulletin, 108, 480-498.
Lerner, J., & Tetlock, P. E. (1999). Accounting for the effects of accountability. Psychological
Bulletin, 125, 255-275.
Maddux, W. W., Barden, J., Brewer, M. B., & Petty, R. E. (2005). Saying no to negativity: The
effects of context and motivation to control prejudice on automatic evaluative responses.
Journal of Experimental Social Psychology, 41, 19-35.
Matheson, K. J., Warren, K. L., & Foster, M. D. (2000). Reactions to affirmative action: Seeking
the bases for resistance. Journal of Applied Social Psychology, 30, 1013-1038.
McEvily, B., Perrone, V., & Zaheer, A. (2003). Trust as an organizing principle. Organization
Science, 14, 91-103.
McKinney, A. P., Carlson, K. D., Mecham III, R. L., D’Angelo, N. C., & Connerley, M. L.
(2003). Recruiters’ use of GPA in initial screening decisions: Higher GPAs don’t always
make the cut. Personnel Psychology, 56, 823-845.
Mellers, B. A., & Hartka, E. (1988). “Fair” selection decisions. Journal of Experimental Social
Psychology, 14, 572-581.
Messner, C., Wänke, M., & Weibel, C. (2011). Unconscious personnel selection. Social
Cognition, 29, 699-710.
Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology,
67, 371-378.
Monin, B., & Miller, D. T. (2001). Moral credentials and the expression of prejudice. Journal of
Personality and Social Psychology, 81, 33-43.
Norton, M. I., Sommers, S. R., Apfelbaum, E. P., Pura, N., & Ariely, D. (2006). Color blindness
and interracial interaction: Playing the political correctness game. Psychological Science,
17, 949-953.
Norton, M.I., Vandello, J.A., & Darley, J.M. 2004. Casuistry and social category bias. Journal of
Personality and Social Psychology, 87, 817-831.
Olson, M. A., & Fazio, R. H. (2004). Reducing the influence of extra-personal associations on
the Implicit Association Test: Personalizing the IAT. Journal of Personality and Social
Psychology, 86, 653-667.
ACCOUNTABILITY SYSTEMS 32
Olson, M. A., & Fazio, R. H. (2006). Reducing automatically-activated racial prejudice through
implicit evaluative conditioning. Personality and Social Psychology Bulletin, 32, 421433.
Page-Gould, E., Mendoza-Denton, R., & Tropp, L. R. (2008). With a little help from my crossgroup friend: Reducing anxiety in intergroup contexts through cross-group friendship.
Journal of Personality and Social Psychology, 95, 1080-1094.
Paluck, E. L., & Green, D. P. (2009). Prejudice reduction: What works? A critical look at
evidence from the field and the laboratory. Annual Review of Psychology, 60, 339-367.
Park, S. H., Glaser, J., & Knowles, E. D. (2008). Implicit motivation to control prejudice
moderates the effect of cognitive depletion on unintended discrimination. Social
Cognition, 26, 379-398.
Plant, E. A., & Devine, P. G. (2001). Responses to other-imposed pro-Black pressure:
Acceptance or backlash? Journal of Experimental Social Psychology, 37, 486-501.
Plant, E. A., Peruche, B. M., & Butz, D. A. (2005). Eliminating implicit racial bias: Making race
nondiagnostic. Journal of Experimental Social Psychology, 41, 141-156.
Richard, O. C., & Kirby, S. L. (1998). Women recruits’ perceptions of workplace diversity
program selection decisions: A procedural justice examination. Journal of Applied Social
Psychology, 28, 183-188.
Ruscher, J. B., & Duval, L. L. (1998). Multiple communicators with unique target information
transmit less stereotypical impressions. Journal of Personality and Social Psychology,
74, 329-344.
Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High stakes testing in
employment, credentialing, and higher education: Prospects in a post-affirmative action
world. American Psychologist, 56, 302-318.
Schoorman, F. D., Mayer, R. C., & Davis, J. H. (2007). An integrative model of organizational
trust: Past, present, and future. Academy of Management Review, 32, 344-354.
Sitkin, S. B., & George, E. (2005). Managerial trust-building through the use of legitimating
formal and informal control mechanisms. International Sociology, 20, 307-338.
Stewart, B. D., & Payne, B. K. (2008). Bringing automatic stereotyping under control:
Implementation intentions as efficient means of thought control. Personality and Social
Psychology Bulletin, 34, 1332-1345.
Suchman, M. C. (1995). Managing legitimacy: Strategic and institutional approaches. Academy
of Management Review, 20, 571-610.
Swim, J. K., Aikin, K. J., Hall, U. S., & Hunter, B. A. (1995). Sexism and racism: Old-fashioned
and modern prejudice. Journal of Personality and Social Psychology, 68, 199-214.
Tajfel, H. (1982). Social identity and intergroup relations. Cambridge, England: Cambridge
University Press.
Tetlock, P. E. (1983). Accountability and complexity of thought. Journal of Personality and
Social Psychology, 45, 74-83.
Tetlock, P. E. (1985). Accountability: A social check on the fundamental attribution error. Social
Psychology Quarterly, 48, 227-236.
Tetlock, P. E. (1992). The impact of accountability on judgment and choice: Toward a social
contingency model. In M. Zanna (Ed.), Advances in experimental social psychology (vol.
25, pp. 331-376). New York: Academic Press.
ACCOUNTABILITY SYSTEMS 33
Tetlock, P. E., & Mitchell, G. (2009). Implicit bias and accountability systems: What must
organizations do to prevent discrimination? Research in Organizational Behavior, 29, 338.
Turner, R. N., Hewstone, M., Voci, A., & Vonofakou, C. (2008). A test of the extended contact
hypothesis: The mediating role of intergroup anxiety, perceived ingroup and outgroup
norms, and inclusion of the outgroup in the self. Journal of Personality and Social
Psychology, 95, 843-860.
Unzueta, M. M., Gutierrez, A. S., & Ghavami, N. (2010). How believing in affirmative action
quotas affects White women’s self-image. Journal of Experimental Social Psychology,
46, 120-126.
Vorauer, J. D., Martens, V., & Sasaki, S. J. (2009). When trying to understand detracts from
trying to behave: Effects of perspective-taking in intergroup interaction. Journal of
Personality and Social Psychology, 96, 811-827.
Wegener, D. T., & Petty, R. E. (1995). Flexible correction processes in social judgment: The role
of naïve theories in corrections for perceived bias. Journal of Personality and Social
Psychology, 68, 36-51.
Wegener, D. T., & Petty, R. E. (1997). The flexible correction model: The role of naïve theories
of bias in bias correction. In M. P. Zanna (Ed.), Advances in experimental social
psychology (vol. 29, pp. 141-208). Mahwah, NJ: Erlbaum.
ACCOUNTABILITY SYSTEMS 34
Appendix
Figure 1. Equal-advantage Labor Pool
7
6
Wm Wf
Wm Wf
Bm
Bm
Bf
Bf
2Wm Wf 2Wm Wf 4Wm Wf
TECHNICAL
5
Bm
Wm Wf
4
Bm
Bf
Bf
Bm
Bf
Bm
Bf
2Wm Wf 4Wm Wf 2Wm Wf Wm Wf
Bm
Bf
Bm
Bf
Bm
Bf
Bm
Bf
4Wm Wf 2Wm Wf 2Wm Wf
3
Bm
2
Bf
Bm
Bf
Wm Wf
Wm Wf
Bm
Bm
Bf
Bm
Bf
Bf
1
1
2
3
4
TEAMWORK
5
6
7
ACCOUNTABILITY SYSTEMS 35
Figure 2. Majority-advantage Labor Pool
Wm
7
Bm
Wm
Wf Wm
Wf
2Wm Wf 2Wm Wf 4Wm Wf Wm
Wf
6
Bm
5
TECHNICAL
Wm
Bm
Bf
Bm
Bf
Bm
Bf
Wf 2Wm Wf 4Wm Wf 2Wm Wf Wm
4
Bf
Bm
Bf
Bm
Bf
Bm
Bf
Bm
4Wm Wf 2Wm Wf 2Wm Wf
3
Bm
Bf
Bm
Bf
Bm
Bf
Bm
Bf
Wf
2
Bm
Bf
Bm
Bf
Bf
Wf
1
Bf
1
2
3
4
TEAMWORK
5
6
7
ACCOUNTABILITY SYSTEMS 36
Table 1. Merit-Equality Tradeoff Scale
1. I believe the right thing to do is to make sure people are selected for positions based on
merit.
2. I support judging people on merit alone, even if it leads to the under-representation of
certain groups.
3. The most qualified person should always be selected for a position.
4. I believe it is important to consider the need for equal representation from all groups
when selecting people for positions. (R)
5. In order to remedy past discrimination and to prevent future discrimination, I support
establishing numerical goals for selecting people for positions. (R)
6. It is important to me that groups be equally represented in a position, even if it means the
best-qualified person is not always selected. (R)
ACCOUNTABILITY SYSTEMS 37
Table 2. Varimax factor loadings of Egalitarianism Scale
Measure
Merit-Equality Tradeoff Scale
Motivation to Control Prejudiced Reactions Scale
Modern Sexism Scale
Libertarian Ideology
Factor
-.69
.69
.62
-.48
ACCOUNTABILITY SYSTEMS 38
Percentage of Times Minority Favored
Figure 3. Frequency of pro-minority advantage
1.20
1.00
Equal-Advantage
Majority-Advantage
0.80
0.60
0.40
0.20
0.00
White
Female
Black Black Male
Female
White
Female
Black Black Male
Female
Minority Group
No accountability
Process accountability
Outcome accountability
ACCOUNTABILITY SYSTEMS 39
Figure 4. Size of pro-minority advantage
0.2
Average Pro-Minority Effect
Equal-Advantage
Majority-Advantage
0.15
0.1
0.05
0
-0.05
White
Female
Black
Female
No accountability
Black
Male
White
Female
Minority Group
Process accountability
Black
Female
Black
Male
Outcome accountability
ACCOUNTABILITY SYSTEMS 40
Table 3. Averages of teamwork and technical scores (with standard deviations in parentheses)
for candidates in each interview-rating category. (Higher ability averages mean better
qualified.)
RATING
EQUAL ADVANTAGE
Process
1 Best
2
3
4
5
6
7 Worst
5.49
(.51)
4.97
(.14)
4.49
(.26)
3.98
(.24)
3.51
(.26)
3.06
(.23)
2.51
(.52)
Outcome
t-stat
MAJORITY
ADVANTAGE
Process Outcome t-stat
5.47
5.92
.51
(.52)
(.69)
4.92
5.01
**
3.49
(.23)
(.27)
4.51
4.50
-.82
(.32)
(.32)
3.98
3.97
-.37
(.27)
(.25)
3.49
3.51
1.16
(.31)
(.36)
3.13
3.00
-2.83**
(.39)
(.37)
2.51
2.12
.08
(.50)
(.68)
** p<.01; * p<.05
5.92
(.71)
4.95
(.34)
4.53
(.31)
3.97
(.25)
3.54
(.33)
3.01
(.32)
2.08
(.67)
-.03
2.18*
-1.42
-.11
-1.20
-.21
.64
ACCOUNTABILITY SYSTEMS 41
Figure 5. Predicted regression effects showing the interaction between participant
egalitarianism and candidate sex.
-0.04
High Egals
Predicted Effect
-0.03
-0.02
Low Egals
-0.01
0
0.01
Male Candidates
Female Candidates