Casey W. Miller cwmsch@rit.edu
Rochester Institute of Technology
Director, Materials Science and Engineering
School of Chemistry and Materials Science
APS Committee on Minorities (2015-2018)
2015 Inclusive Grant Initiative
Division of Diversity and Inclusion

(Feb 2013)

Invited Diversity Talks; Invite me!

• Broadening Participation Matters
• State of Physics
• Bridge Programs and their Key Features
• Admissions Criteria and Diversity
• Using Minimum Acceptable GRE
Scores Negatively Impacts Diversity in
STEM and does not select students with the most research potential

Broadening Participation
Matters
• US Demographics are changing rapidly
– Minorities represent 1/3 of the US population, but much less in
STEM jobs (e.g., ~5.4% of PhDs)
– Engage URMs in STEM to utilize all facets of the domestic talent pool, address brain drain

Broadening Participation Matters
• Diversity is an Asset
– Diversity enhances innovation
– Collective intelligence from diverse social skills

Physics is among the least diverse of the sciences.
Credit: APS/Source: IPEDS Completion Survey

Physics is among the least diverse of the sciences.
US pop. age 25-29
US pop. age 21-24 average of 2006-2008 Credit: APS/Source: IPEDS Completion Survey

0.7
Physics is among the least diverse of the sciences.
0.6
0.5
Biology
Chemistry
Math & Stats
Earth Sciences
Physics
Engineering
0.4
0.3
0.2
0.1
0
1965 1970 1975 1980 1985
Year
1990 1995 2000 2005 2010
Credit: APS/Source: IPEDS Completion Survey

25%
Physics is among the least diverse of the sciences.
20%
15%
10%
Bachelor
5%
0%
1965 1970
Postdoctoral
Doctoral
1975 1980 1985
Year
1990 1995 2000 2005 2010
Credit: APS/Source: IPEDS Completion Survey & NSF-NIH Survey of Graduate Students & Postdoctorates in Science and Engineering

Small numbers =
Top URM Producers
University of California-Berkeley
University of Michigan at Ann
Arbor
Howard University
Florida Agricultural and
Mechanical University
University of Arizona
Alabama Agricultural and
Mechanical University
Hampton University
Rice University
University of California-Los
Angeles
University of Texas Hlth Sci Ctr
San Antonio
University of Texas at Austin
North Carolina State University at Raleigh
Potential for Impact
5-year Total Degree Production of
Top Producers of Physics URM PhDs
2006-
2010
8
8
8
4
4
6
5
5
5
4
4
4
Top Hispanic PhDs
University of California-Berkeley
Rice University
University of California-Los
Angeles
University of Michigan at Ann
Arbor
University of Texas Hlth Sci Ctr
San Antonio
University of Texas at Austin
Arizona State University Main
Harvard University
Stanford University
University of Arizona
University of Rochester
2006-
2010
6
4
4
4
4
4
3
3
3
3
3
Top African American
PhDs
Howard University
Florida Agricultural and
Mechanical University
Alabama Agricultural and
Mechanical University
Hampton University
North Carolina State
University at Raleigh
University of Michigan at Ann
Arbor
Georgia Institute of
Technology, Main Campus
Harvard University
University of Arizona
University of California-
Berkeley
University of Colorado at
Boulder
University of Illinois at
Urbana-Champaign
University of Maryland at
College Park
2006-
2010
8
6
5
5
4
4
2
2
2
2
2
2
2
Source: IPEDS Completion Survey by Race

Top Priority Actions
1) Increase undergraduate retention and completion via strong academic, social, and financial support.
2) Teacher prep, college prep programs, and
Transition to Graduate Study.

Admissions is typically a two-tier process for all fields

Research and leadership potential
Creativity, Innovation
Enthusiasm, passion for research
Diversity

http://frontporchdenver.com/wp-content/uploads/sites/2/2012/06/coin_toss.jpeg

Many reasons for this,
Admissions is only one http://www.phdcompletion.org/information/Executive_Summary_Demographics_Book_II.pdf

• Grades
– Grade Inflation undermines minority participation http://www.gradeinflation.com/tcr2010grading.pdf

Most URMs attend State Colleges
URM Engineering #BA/BS
University of Florida (240/yr)
Florida International University
Texas A & M University-College Station
University of Central Florida
Georgia Institute of Technology-Main Campus
California State Polytechnic University-Pomona
The University of Texas at El Paso
The University of Texas at Austin
North Carolina A & T State University
The University of Texas-Pan American
Cal Polytechnic State University-San Luis Obispo
The University of Texas at San Antonio
Arizona State University-Tempe
University of California-San Diego
University of Houston
San Diego State University
Morgan State University
Prairie View A & M University
Alabama A & M University
North Carolina State University at Raleigh
Southern University and A & M College
Howard University
Tuskegee University
University of Maryland-College Park
University of South Florida-Main Campus
Virginia Tech (38/yr)
Rank URM Physical Sciences #BA/BS
13
14
15
16
9
10
11
12
5
6
7
8
1
2
3
4
21
22
23
24
17
18
19
20
25
26
Florida International University (85/yr)
Xavier University of Louisiana
The University of Texas at Austin
University of California-Santa Barbara
Texas A & M University-College Station
The University of Texas at El Paso
University of California-Los Angeles
University of Florida
Spelman College
University of California-Irvine
University of North Carolina at Chapel Hill
University of California-Santa Cruz
University of Arizona
University of New Mexico-Main Campus
Florida State University
Georgia State University
Jackson State University
The University of Texas at San Antonio
Columbia University in the City of New York
University of Memphis
CUNY City College
CUNY Graduate School and University Center
Savannah State University
Alabama A & M University
Georgia Southern University
Tennessee State University (15/yr)

• Test Scores
– “Guide to Use of Scores” not followed (or even known of)
– Significant race/gender group differences
– Correlations with Success are questionable

Folder A
GRE-Q: 740 (80%)
Folder B
GRE-Q: 800 (perfect)

http://www.ets.org/gre/institutions/scores/guidelines/
From ETS Guide to Use of Scores:
It is an inexact measure ; only score differences that exceed the standard error of measurement of a given score can serve as a reliable indication of real differences in applicants' academic knowledge and developed abilities.”
Translated to physics-ese:
CONSIDER INSTRUMENT RESOLUTION
S.E.M. ~60 points (on old GRE scale, 200-800).
740 = 800 = perfect!

• About 180 graduate programs are listed in the AIP Graduate Programs book
– ~96% require the General GRE
• about one quarter of these have a stated minimum score for admission, with the median being 700 (~64%)
– ~48% require the Physics GRE
• about half of these have a stated minimum score for admission, with the median being 600 (32%).

From ETS Guide to Use of Scores
Guidelines
– A cutoff score based only on GRE scores should never be used as a sole criterion for denial of admission.
– Any department considering the use of a cutoff score should compile a rationale justifying the appropriateness of such a score for each measure:
(1) evidence that the proposed cutoff score for the measure usefully distinguishes between individuals who are likely to succeed in graduate school and those who are not, and
(2) the impact of the proposed cutoff score on the institution’s goals related to diversity. http://www.ets.org/gre/institutions/scores/guidelines/

From ETS document
"Factors that can influence performance on the GRE general test 2006-2007"
800 166
750
Median (NRC; Physics): 760
75%
700 155
650
600 148
550
500
25%
50%
144
450
400 140
350
300

These performance disparities are:
• Technically not “bias”
• Nearly independent of intended graduate major
• Qualitatively unchanged when controlling for undergraduate GPA
• Qualitatively the same for
•GRE Subject test
•SAT Math
•8 th grade math achievement tests
•4 th grade math achievement tests
• A feature of standardized testing.

Physical Sciences
800
700
600
500
400
300
200
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

Engineering
800
700
600
500
400
300
200
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

Life Sciences
800
700
600
500
400
300
200
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

Business
800
700
600
500
400
300
200
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

Social Sciences
800
700
600
500
400
300
200
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

800
700
600
500
400
300
200
Humanities and Arts
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

Education
800
700
600
500
400
300
200
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

ugGPA = A
800
700
600
500
400
300
200
SOURCE: ETS, "Factors that can influence performance on the GRE general test 2006-2007”

700
650
600
550
500
450
400
350
300
Asian White Hispanic Am. Ind.
Afr. Am.
SOURCE: Total Group Profile Report, College Board, 2009
College-Bound Seniors.

260
250
240
230
220
210
200
300
290
280
270
8th Grade
4th Grade
SOURCE: NCES. The Nation’s Report Card: Mathematics
2009 (NCES 2010-451), National Center for Education Statistics,
U.S. Department of Education.

84
82
80
78
76
74
72
70
90
88
86
SOURCE: U.S. Bureau of the Census, Income, Poverty, and Health Insurance Coverage in the United
States: 2010, Report P60, n. 238, Table B-2, pp. 68-73.; http://www.npc.umich.edu/poverty/#4

• Cut-off scores are used all the time by admissions committee members.
– Sorting spreadsheets is easy.
– Faculty are busy, often reluctantly serving
– Faculty are not trained in selection
– “Low scores must tell you something.”
• But correlation R~0.2-0.3 is only with first year
GPA; by second year, R<0.1
a a NAS: In the Nation’s Compelling Interest: Ensuring Diversity in the Health-Care Workforce (2004)

Asian Am. White Hisp. (oth) Mex. Am. Am. Ind.
1000 3.5
350
14
12
3.0
300
800 12 10
2.5
250
600
10
8
2.0
200 8
6
150 400
6
1.5
100
50
200
4
2
4
2
1.0
0.5
0
200 400 600 800
0
200 400 600 800
Afr. Am.
0
200 400 600 800
Puerto Ric.
0
200 400 600 800
Men
0.0
200 400 600 800
Women
7
25 1000
250
6
20
200
5
800
15 4 600
150
10
5
0
200 400 600 800
GRE-Q Score
3
2
400 100
1
0
200 400 600 800
200
0
200 400 600 800
50
0
200 400 600 800
Only physical sciences & US citz.

Impact of Cut-offs:
%
Δ Representation
Representation
Cut
800 ∑
off one group
Cut
800 ∑
off all groups
Cut-off Asian Am. White URMs Women Men
60
Am. Ind.
700
Hisp. (oth)
9.3%
Other
7.6%
Mex. Am.
250%
82%
77%
20
11%
10
27%
Men
73%
64%
20
0
0
-10
-20
-20
-40
-60
-30
-80 -40
-100 -50
200 300 400 500 600 700 800
(155)
Cut-off
200 300 400 500 600 700 800
(155)
Cut-off

Working with APS
And ETS

GRE Physics Subject Test
7172
467
155
821
Cut-off on GRE Physics
SOURCE: ETS (via APS)

GRE Physics Subject Test
Cut-off on GRE Physics
SOURCE: ETS (via APS)

120.0%
100.0%
80.0%
60.0%
40.0%
20.0%
0.0%
370
Gender Gap Persists ∀ R/E; ∀ Nationalities
120.0%
Other Hispanic, Latino, or
Latino American F
100.0%
White (non- Hispanic) F
White (non- Hispanic) M
Mexican, Mexican
American, or Chicano F
Black or African
American F
Other Hispanic, Latino, or
Latino American M
80.0%
Mexican, Mexican
American, or Chicano M
Black or African
American M
60.0%
40.0%
20.0%
0.0%
370 870 370 870 370 870 370 870
Other F
Other M
870 370
Native Hawaiian, or Other Pacific
Islander F
Native Hawaiian, or Other Pacific
Islander M
870 370
Puerto Rican F
Puerto Rican M
Asian or Asian American
F
Asian or Asian American
M
870 870 370

From ETS Guide to Use of Scores
Guidelines
– A cutoff score based only on GRE scores should never be used as a sole criterion for denial of admission.
– Any department considering the use of a cutoff score should compile a rationale justifying the appropriateness of such a score for each measure:
(1) evidence that the proposed cutoff score for the measure usefully distinguishes between individuals who are likely to succeed in graduate school and those who are not, and
(2) the impact of the proposed cutoff score on the institution’s goals related to diversity. http://www.ets.org/gre/institutions/scores/guidelines/

Correlations with success are questionable

A+
A-
B+
B 9
B-
C+
4.0
3.8
3.6
3.4
3.2
3.0
Similar conclusions:
Michigan; Berkeley
2.0
2.5
3.0
3.5
4.0
20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 ug GPA GRE-Q (%) GRE-V (%) GRE-Physics (%)

Subject Test may tell you a little about preparation , convolved with R/E/G
NOT about potential to become a
PhD-level research scientist (the aim!)

The usual weight given to
GRE scores in admissions exceeds its predictive capabilities and has negative societal impact.

• But it can, if you’re: careless; too busy; or just going through the motions.
• Lowering the bar perpetuates stereotypes exactly by definition
• What bar? Potential for what?

Victoroff and Boyatzis, J. Dent. Ed 77 , 416 (2013):
Correlating clinical performance with admissions criteria and non-cognitive competencies
Cognitive
Non-Cognitive
Didactic
Yes
Maybe
Clinical
No
Yes
Self-Management competencies correlate with clinical grade.
1. Achievement Orientation
2. Adaptability
3. Initiative
4. Emotional Self-Control
5. Trustworthiness
6. Conscientiousness
7. Optimism
“Cognitive ability and knowledge are threshold aspects of professional work, necessary but not sufficient for outstanding professional performance.”

• Non-cogs considered after using filters that negatively impact diversity

• Measurable!
• Results from decades of Industrial-
Organizational Psychology research
– non-cogs predict academic/job performance
– non-cogs show little if any group differences
– non-cogs are orthogonal variables to cognitive constructs (GPA, SAT/GRE)

• ill-defined protocols = haphazard review
– Extra/Zero scrutiny for underrepresented folks

• Coarse Grained Rubrics
– can be implemented now!
– guides review of statements/letters/interviews
• can expedite the review process
• reduce implicit bias ( expectations based on race/gender/name/culture); combats reviewer fatigue
– inter-rater reliability
– develop in conjunction with a social scientist
– add short answer items on app to probe non-cogs?

http://www.vanderbilt.edu/gradschool/bridge/tools.htm

• Situational Judgment Tests
– could be implemented as part of the online application submission process (ease of use!)
– ask the test taker to describe how they would act in a hypothetical scenario; or rank the efficacy of several potential actions
– can be tailored to probe specific constructs or discipline-specific scenarios (hire an expert)
– susceptible to faking
– not terribly expensive

• Normalize components
• Add them
• More effective and efficient than nonalgorithmic methods
• non-cogs are orthogonal to cogs
– weight these equally http://en.wikipedia.org/wiki/Apgar_score

• Traditional admissions process
– No better than a coin toss
– Strongly filters under-represented groups
• Large race/gender gaps on standardized tests
• Grade inflation lowest at state universities
• Non-cognitive assessments can help
– increase the diversity of the applicant pool
– select students who will graduate
• Systematic evaluation protocols can
– limit implicit bias, increase diversity
– increase efficiency of screening applicants
• Get a social scientist to help!