THE ROLE OF AUGMENTED REALITY IN SPORT MONITORING
SYSTEMS
Author name – leave blank until after review process
author's affiliation
1st line of address
2nd line of address
Email address
URL for personal/project Web page
ABSTRACT
Many sports have become very reliant on
monitoring systems such as Hawk-Eye in Tennis;
this paper has looked into just how accurate the
technology is, and assessed the effectiveness of the
technological approaches used to implement the
system. The level of accuracy is vital for sport
monitoring systems, in a number of sports such as
Line Calling Decisions in Tennis. Even a small
margin of error can affect the decision of whether
the ball is called in or out. As many high profile
sporting industries have placed a great deal of
dependence upon this technology, if it were to prove
inaccurate, the sporting world would incur
devastating consequences. As governing bodies
would then be under a large amount scrutiny, from
all over the world. Players, coaches and even
spectators would all start to question the decisions
made using this technology made in past. And as
these were designed to rule out human error in such
cases as line calling, any major failings found in the
technology would render them useless.
Keywords
Augmented Reality, Monitoring Systems, Hawk-Eye,
Hot Spot, Image Analysis, Thermal Infra-red
Imaging
1. INTRODUCTION
The use of technology has helped shape the future
of sports today, most of us understand the purposes
of modern systems such as Hawk-Eye, but can you
actually say you know how they work? This paper
will discuss the technical briefing of two monitoring
systems in the sporting industry, the first most
commonly known as Hawk-Eye, which employs a
Permission to make digital or hard copies of all or part of this
work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or
commercial advantage and that copies bear this notice and the
full citation on the first page. To copy otherwise, to republish, to
post on servers or to redistribute to lists, requires prior specific
permission.
4th Annual Multimedia Systems, Electronics and Computer
Science, University of Southampton
© 2010 Electronics and Computer Science, University of
Southampton
technique called Image Analysis. The other, is
called Hot Spot, which implements a different
technique known as Thermal Infra-red Imaging, and
is used in Cricket to detect any change of
temperature caused by the friction of the ball hitting
another object. The technical briefing will examine
several aspects of these systems, looking into the
design motives, technological approach, and level of
accuracy and possible future developments. For
Hawk-Eye, the system can be compared to different
applications of sport, for example, in cricket; the
system would have different purposes than to
Tennis. Then to conclude, the paper will evaluate
the impact the two systems plays in sports today,
judging the accuracy and effectiveness of the
techniques and also give some ideas into possible
future developments.
2. HAWK-EYE
2.1 Design Motives
To this day, Hawk-Eye has proved one of the most
successful and innovative technologies in the
sporting industry, with most notable sports such as
Tennis and Cricket implementing the system. This
technology was created by Dr. Paul Hawkins, who
invented the ball tracking system in order to improve
the quality of sporting decisions, providing a quick,
reliable and accurate system which could support
match officials when making vital decisions in real
game situations.
The technology Hawk-Eye was initially developed
for cricket, with an aim to broadcast reviews of
umpires LBW (Leg before Wicket) decisions. This
would be done by creating a 3D virtual simulation,
where the event could be played back from different
angles to assess the incident. Judging whether the
umpire made the right decisions or not. Since its
original deployment, Hawk-Eye went onto other
sports such as Tennis, used to make accurate and
quick Line Calling Decisions. In recent times, HawkEye has surpassed its original expectations in both
Cricket and Tennis, where now not only is the
technology used for broadcasting purposes, but for
match officials to review the decisions made [7].
2.2 Technological Approach
This section will review the technological
approaches made by Dr. Paul Hawkins when
designing the system, demonstrating knowledge
that will justify how and why the technology was
such a big success. Hawk-Eye uses a range of
sophisticated high speed cameras, precisely located
in evenly spaced out positions around the area of
play, these are used to accurately track the motion
and projection of the balls trajectory as it flies
through the air. (See figure 1)
2.3 Accuracy of the System
Although Hawk-Eye has proved a success with
many sports, there are still flaws to the system. Due
to the camera's limited frame rate, the position of
the ball between frames has to be predicted, which
will have an impact upon the ball’s projected
trajectory. This then brings into question the
algorithm used to determine the balls flight path in
between frames, and this is definitely an area where
there is room for improvement in the future as the
system does not at the moment guarantee 100%
accuracy. In Tennis too there is still need for greater
accuracy as serves can reach up to 150 mph, which
is a significant difference in speed compared to
bowling in cricket. As the ball travels at a faster
distance in between camera frames, this means that
the balls trajectory has to estimate for a longer
distance, reducing the overall accuracy of the balls
true position [5].
Figure 1: Camera Arrangement [14]
Based on the theory of triangulation, the cameras
can locate the position of the ball by using both the
visual images and timing of the data provided.
Triangulation is the process in determining an
unknown point or location, in this case the ball, by
using the position of two fixed points, the cameras,
which are a known distance apart [10]. Analysing
the pixels of each frame in the video feed is used to
determine the approximate location of the centre of
the ball, known as data points or tracklets.
By comparison with at least four cameras, the 3D
position of the ball can be developed from the data
points. Repeating this process for each frame, it
helps to reconstruct the trajectory of the ball in a
fourth dimension, predicting the future flight path. In
order for the video feed to process the simulation at
a high speed, the system requires dedicated video
processors, databases and multiple processing
computers [3].
Figure 2: Hawk-Eye Accuracy Example [5]
According to Hawk-Eye Innovations, the system has
an overall mean error of 3.6mm of its true location,
and this small margin of error could still have an
impact upon a sporting event, for example in LBW
decisions in cricket. Figure 2 shows an extreme
case where hawk-eye could go wrong, where the
small margin of error would have an impact upon
deciding whether the ball would actually go on to hit
the stumps [13]. Munnelly, Fritsch and Clarke all
believe modularization provides a number of
benefits to this system, for instance manageability
and maintainability. Modularization helps to facilitate
the development of new or existing components into
a system, permitting the ease of future growth [8].
And in the future Hawk-Eye may need to upgrade
the technology used in the system to increase its
overall accuracy.
2.4 Application in Cricket
As already stated the Hawk-Eye system was initially
developed for the game of cricket, with its main
focus on reviewing LBW decisions, to provide an
accurate system to support umpires when deciding
the batsman’s fate. In this case, Hawk-Eye would be
used to reconstruct a 3D simulation of the bowling
delivery, used then to assess the trajectory of the
ball. Figure 3 below demonstrates the 3D simulation
of the projected flight path that Hawk-Eye would
present to the umpire, revealing if the ball would go
onto hit the stumps. Hawk-Eye takes into
consideration several external factors which could
affect the flight path of the ball, both Wang and
Parameswaran state that the system can calculate
in a second where the ball is pitched, the extent of
its lateral movement in the air and off the wicket, its
velocity and the effect of the bounce [14].
are benefits that are raised by Dewell. Here are
some of the key statistics Hawk-Eye can provide:

Wagon Wheel - shows where the batsman
has made all his runs on the field.

DeSpin - shows how much a ball has
deviated after pitching compared to a ball
which hasn't spun away.

Pitch Maps - shows where a specific
bowler has pitched the ball on the wicket to
a batsman.

Ball Speeds - compares two of more
bowling deliveries, comparing the speed of
which they are travelling.
2.5 Application in Tennis
After surpassing their initial expectations in Cricket,
Hawk-Eye wanted to imitate their success in other
markets, deciding to implement a new system in
Tennis to allow players to review Line Calling
Decisions. The players are allowed to challenge
three line calls per set, if the player is unsure about
a call made by a line judge, they can opt to use one
of their challenges, where the play is referred and
reviewed by Hawk-Eye. Similar to LBW decisions in
cricket, Hawk-Eye will reconstruct a 3D simulation of
the tennis rally, where the flight path and the bounce
of the ball is replicated to depict whether the ball
bounced on the line or not, see figure 4.
Figure 3: Hawk-Eye LBW decision in Cricket [3]
Dewell believes Hawk-Eye has benefited Cricket
from a number of perspectives, improving the game
from both a marketing and educational view. The
successful impact Hawk-Eye has sustained, through
gaining a reputation for accuracy and reliability, and
has helped viewers have a better knowledge of the
game. Thus overall, making the sport more
attractive to watch, while developing the quality of
both playing and coaching. [6] While Bailey, on the
other hand, is more skeptical of the decisions hawkeye makes, questioning the validity and accuracy of
the system, stating that the system has the luxury of
being a commentator’s tool rather than a missioncritical input device [2].
The technology behind Hawk-Eye can also help
cricket in other aspects, offering key analysis and
statistics of specific bowling and batting
performances for example and therefore providing a
useful tool for coaches, who can utilize the statistics
to improve aspects of a player’s performance, which
Figure 4: Hawk-Eye Line Call in Tennis [15]
Figure 5 presents the Tennis Hawk-Eye system
which enables the technology to work, showing the
modularization techniques used as data travels from
each sequence to formulate the final 3D virtual
simulation. The process begins at the cameras,
which surround the court in an accurate formation
so all lines are covered from numerous angles.
When the player challenges a call made, each
camera has to differentiate the ball during each
frame. But before the coordinates are sent, the
camera PTZ (Pan/Tilt/Zoom) position is taken into
consideration, as each camera’s motion will vary
from its fixed location, impacting upon the
coordinates.
Figure 5: Hawk-Eye Tennis System [9]
Once the frames have located the data point
measurements, the data is sent to the Image Plane
Ball Tracking, where the 2D tracklets of the balls
trajectory can be constructed. One limitation of
employing a Hawk-Eye system is the cost of the
hardware, not all broadcasting cameras used have
the top specification due to limited funds. As a
result, the cameras tend to have slower shutter
speeds, which can affect the shape of the ball
during each frame. The purpose of the Image Plane
is to correct this problem, by obtaining accurate
measurements of both the balls shape and location
in each frame. The 2D tracklets are then sent to the
3D Reconstitution module where the tracklets are
assembled into 3D tracks, using visualisation
software to display and analyse the simulation,
judging whether the ball had hit the line or not [9].
As stated above in the Application for Cricket
section, Hawk-Eye isn’t solely used to aid decision
making. The system can also be used to assess a
player’s performance, by looking at the match
statistics derived from the technology. Figure 6
below shows an example, looking into the serve
direction and the rally hit points of a match between
Myskina and Mauresmo [15]. From this information,
coaches can see how well their player served in the
game and see where they played most of their shots
on the court. If one aspect of their game is
underperforming, coaches can use the statistics as
evidence of how they could improve.
Figure 5: Hawk-Eye Tennis Statistics [15]
2.6 Further Developments:
Hawk-Eye still has the potential for a lot more
growth; recent developments have seen the
technology introduced into Snooker. Using a variety
of techniques in hawk-eye, the system has provided
a useful tool for commentators to help annotate the
player’s strategies in the game. This is made
possible by tracking the white the ball, analysing its
motion to demonstrate some possible example
animated shots the player could take [4]. One sport
which Hawk-Eye is yet to be implemented into is
Football; many critics have expressed a keen
eagerness into using goal line technology, as a lot
of controversy has been portrayed in the media.
Similar to the application in tennis, Hawk-Eye would
be used to determine whether the whole football
had crossed the line.
3. HOT SPOT
3.1 Design Motive
Hot Spot technology was solely designed for the
game of cricket; recent developments have seen the
system become integrated into the official third
umpire review process. The fundamental aim of this
technology is to detect whether the cricket ball
made any sort of contact with the bat. For instance
in LBW decisions, if the ball makes contact with the
bat before pad, then the decision of LBW can't be
given.
In the past, other similar technologies such as
Snicko-Meter have proved both inconclusive and
inaccurate in some extreme cases. This system
uses a combination of high speed cameras, and
microphones attached in the stumps to an
oscilloscope, used to detect any traces of sound
waves as the ball passed the bat in slow motion. But
the interference of other sound waves, for example,
bat on pads, substantiated that the system couldn't
always be trusted. This provided both a key design
motive and a gap in the market for an accurate
cricket system, such as Hot Spot, to be introduced
into the game.
Figure 6: Thermal Radiation from a Cricket Ball [1]
4. CONCLUSIONS
3.2 Technological Approach
Hot Spot makes use of two thermal infra-red
cameras, positioned at opposite ends of the area of
play. Through the process of thermal infra-red
imaging (TIR), the cameras can detect whether the
ball has made any contact on the bat, pads or
anywhere else on the batsman. TIR imaging
provides an instantaneous, remote sensing
technique for converting the balls thermal radiation,
invisible to the eye, into an electrical signal, where
the temperature is amplified and displayed into a
visible image. Thermal radiation indicates the level
of surface temperature distribution of an object [1],
any surface with a temperature above an absolute
zero produces electromagnetic radiation [10].
If the ball has made contact with the bat, the TIR
cameras would be able to identify any thermal
radiation off the bat, caused from the friction the ball
creates on impact. Thermal radiation, also known as
infra-red or heat radiation, can be defined as the
collection of electromagnetic waves in a specific
range of wavelengths in a spectrum, varying from
0.75- 14 micrometres (µm) [11]. Figure 6 displays
the electrical signal of a balls Hot Spot, depending
on the position of the area of impact, the umpire can
use this system to make a decision on whether the
batsman is out or not.
To conclude the paper, a technical briefing was
given on two monitoring system used in sport, Hawk
Eye and Hot Spot as used in sport today. For each
system, the design motive, technological approach,
level of accuracy and possible further development
were given. The paper reviewed the different
techniques used in each system, for example HawkEye bases its system on the theory of Triangulation,
using a range of high speed cameras in fixed
positions to determine the location of the ball. Then
through the process of Image Analysis, the balls
predicted trajectory can be determined by tracking
the data points of the ball in each frame. It also
employs a modularisation technique, splitting the
complex system into smaller processes, facilitating
the room for potential growth and supporting the
ease of management and maintainability.
Although the accuracy of Hawk-Eye has been
questioned, the system has surpassed their
expectations, proving both effective and accurate in
a number of sports. It also offers sport such as
Cricket and Tennis a useful tool for assessing the
game both on and off the field. As sport has now
become both highly competitive and, as in football,
a financially based business it is increasingly
important that decisions made on the pitch are the
correct ones. And any mistakes made widely known
by way of the media. So in future, as technology
grows, Hawk-Eye should look to invest in cameras
which provide a higher frame and shutter rate.
Progressively, this should improve the accuracy of
the system from a mean error of 3.6mm. Hot Spot,
on the other hand, implementing a technique called
Thermal Infra-red Imaging, the only limitation found
thus far is the cost of developing and maintaining of
the system. Future work should be used to build a
cheaper system, which still has the
effectiveness and high level of accuracy.
same
5. REFERENCES
[1] Ahmad I., Prasad N.R., Omkar S.N. and Rajan
K., Quantification of Impact of a Cricket Ball, Space
Technology Cell, Department of Aerospace
Engineering, Indian Institute of Science (2006). p1-5
[2] Bailey B., Real Time 3D motion tracking for
interactive computer simulations. (2007) p13-14.
[3] Baguley G., Stereo Tracking of Objects with
respect to a Ground Plane. University of
Canterbury, Christchurch, New Zealand (2009) p9.
[4] Christmas W.J. et al., A System for the
Automatic Annotation of Tennis Matches, Centre for
Vision, Speech and Signal Processing. University of
Surrey (2004) p1.
[5] Collins H. and Evans R., You cannot be serious!
Public understanding of technology with special
reference to “Hawk-Eye”, Public Understanding of
Sciences. 17 (3), p183-308 (2008).
[6] Dewell R., Multimedia Learning Aids to Improve
Cricket Coaching, University of Gloucestershire, p13
[7] Guo H., and Namee B.M., Using Computer
Vision to Create a 3D Representation of a Snooker
Table for Televised Competition Broadcasting.
Dublin
Institute
of
Technology.
1-2.
[8] Munnelly J., Fritch S. and Clarke S., An AspectOriented Approach to the Modularisation of Context.
In IEEE International Conference on Pervasive
Computing and Communications (PerCom) 2007.
(Dublin, Ireland, 2007) p1.
[9] Owens N., Harris C. and Stennett C., Hawk Eye
Tennis System, The Institution of Electrical
Engineers, p1-4 (2003).
[10] Thurmond V.A., The Point of Triangulation,
Journal of Nursing Scholarship, 33 (3), p253-258
(2001).
[11] Titman D.J., Infrared Thermal Imaging. in IEE
Colloquium Medical Scanning and Imaging
Techniques of Value in Non-Destructive Testing
(2002) p1.
[12] Todtong Y. et al. Observation of Scratch on
Magnetic Media by using Thermal Infrared Camera,
Khon Kaen University, Thailand. (2008)
[13] Rasmussen A.T. and Rasmussen H.M.,
Tracking People in Sports using Video Analysis.
Technical University of Denmark (2008).
[14] Wang J.R. and Parameswarran N. Survey of
Sports Video Analysis: Research Issues and
Applications. University of North South Wales
(2004) p1-4.
[15] White R.B., Improving a Video Based Darts
Match Analyzer, COM3021, p1-2 (2009).