Assessment of Soccer Referee Proficiency in Time-Sensitive Decision-Making
Nathan Jones, Andrew Cann, Saud Almashhadi, Hina Popal
System Engineering & Operations Research, George Mason University
Problem & Need Statement
Context
Referee Call Making Process
Method of Analysis
Junior level referees do not receive assessments
for game flow understanding or fitness attributes
as predictors of call accuracy.
Analysis Part I:
Soccer Game Simulator
An assessment method is needed to evaluate
junior referees based on fitness and/or game flow
understanding attributes.
#
1
Referee Attributes
Attribute Assessment Method
Fitness Test (senior referees only)
2
Call Decision Making (CDM)
Written exam on rules (all referees)
3
Game Flow Understanding (GFU)
Indirectly using on field assessment
(senior referees only)
4
Fitness
Design Alternatives
Fitness Test
Game Flow
Evaluation
Combined
Evaluation
No Assessment
Description
A baseline fitness test equivalent
to those administered to senior
referees
Video performance assessments
conducted by official assessors
Combination of first two
evaluations
Not conducting any referee
evaluations (status quo)
Tests
Fitness
GFU
Fitness
GFU
None
Results
Part I: Soccer Game Simulator
Fitness and GFU range from 0
(worst possible) to 100 (best possible)
Part II: Monte Carlo Analysis
Alternative
Utility
Fitness Test
Game Flow
Evaluation
Combined
Evaluation
No Assessment
0.74926
0.72693
Simulation developed in Java
A two part analysis was conducted to determine alternative utilities:
Utility is defined as the average call accuracy of the top 100 junior referees
identified using each alternative.
Part I: A regression for call accuracy based on fitness and GFU developed using
a discrete event soccer game simulator.
Part II: Utility of each alternative determined through a Monte Carlo analysis
using regression from part I.
95 % Utility
Half-Width
0.00012
0.00028
0.74174
0.00021
0.72099
0.00004
Conclusions & Further Findings
Recommendation
Fitness Test vs. No Assessment (status quo)
Marginal Cost
Fitness Test:
Marginal Utility
Fitness Test:
Further Findings I:
Impact of Teams
$26,990
0.028 Utility
Improvement
It is not cost effective to implement
fitness tests on junior referees.
Simulation output regression analysis (R2 = 99.51):
Call Accuracy (Fitness, GFU) = 0.713491 + 0.000923486
Fitness + 1.28791e-5 GFU - 6.4846e5 Fitness2 + 1.12504e-6
GFU2 + 1.26193e-6 Fitness3 - 6.75305e-9 Fitness4
“Fitness Test” dominates all other assessment based alternatives.
Further Findings II: Recommendation
When evaluating referee quality based on
game performance, team combination must
be considered as a potential confounding
variable in the analysis.
Team combination and game
flow have a significant impact
on referee call accuracy.