Shoe-Mouse: An Integrated
Intelligent Shoe∗
Weizhong Ye, Yangsheng Xu and Ka Keung Lee
Department of Automation and Computer-Aided Engineering
The Chinese University of Hong Kong
EB2, Shatin,CUHK,HongKong
wzye@acae.cuhk.edu.hk
Student ID ： M9920103
Student
： Kun-Hong Lee
Adviser
： Ming-Yuan Shieh
1
Outline
• ABSTRACT
• INTRODUCTION
• HARDWARE DESIGN
 DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF
INTELLIGENT SHOES
• EXPERIMENTAL RESULTS
• CONCLUSION AND FUTURE WORK
• REFERENCES
2
ABSTRACT
 In this paper, we developed a sensor-integrated shoe as an
information acquisition platform to sense the foot motion.
The system is small, portable and wearable.
 The platform is mainly composed of four parts including a
sensing module, a computing module, a wireless
communication module, and a data visualization module.
3
ABSTRACT
 Based on this platform, we developed a novel input device
called Shoe- Mouse, which can be used by people who have
difficulties in using their hands to operate computers or
devices.
 The platform can be also used for applications such as gait
recognition, human identification, and motion
monitoring.
4
INTRODUCTION
Main attributes of our system
 Firstly, we integrated all the sensors and circuits fully inside
the shoe without adding much weight into the original
shoe.
 It is easy to use and users will notice little difference
between their normal shoes and the proposed shoes.
 Secondly, we built a hardware platform to collect data from
the shoe. This platform is programmable, easily scalable
and easy to be integrated to the other applications.
5
INTRODUCTION
 Thirdly, we applied the system to several successful tasks
based on this platform, especially the Shoe-Mouse.
 By using this interface, we can operate a device with our
feet.
 This invention can be useful for the persons who have to
use computers frequently but have difficulties in using
their hands.
6
HARDWARE DESIGN
Fig. 2. Outline of the hardware design and sensor
7
HARDWARE DESIGN
Sensing the parameters inside the shoe
 This component consists of a flexible instrumented insole
(in Fig. 3) that is worn inside the shoe.
 One important parameter the insole measures is the force
applied between the foot and the shoe at some key points.
 These key points are under the major weight-bearing
points of the foot: the big toe and the heel ( which is
divided into a medial and lateral portion ).
8
HARDWARE DESIGN
Sensing the parameters inside the shoe
Fig. 3. A flexible instrumented insole
9
HARDWARE DESIGN
Sensing the parameters inside the shoe
 The force sensors operate with a voltage source and a fixed
resistor to produce a voltage that changes with the applied
forces.
 This voltage is further processed in order to present a
scaled voltage to the input of an analogto-digital converter.
The digitized value of the force will be sent to the microcontroller .
10
HARDWARE DESIGN
Sensing the parameters inside the shoe
 Besides, a bend sensor is installed between the toe and the
heel.
 This bend sensor is used to measure the degree of bend
between the toe and the heel.
 The output of the bend the sensor also contains rich
information about human motion, especially loading and
uploading of feet.
11
HARDWARE DESIGN
Gathering information from the sensors
 This subsystem is mainly composed of a processor circuit
board (in Fig. 4). It includes a microcomputer, an analogto-digital converter , peripheral components , batteries,
and an attached accelerometer.
 As can be seen from Fig. 4, the circuit board is small and it
can be easily put into the heel of the shoe so that users will
notice little the difference between the normal shoes and
the intelligent shoes.
12
HARDWARE DESIGN
Gathering information from the sensors
Fig. 4. Processor circuit board
13
HARDWARE DESIGN
Data visualization
 As this platform is designed for general applications, we
display all parameters measured from the shoe. The host
computer gets the data from the wireless receiver via RS232.
 Different functions for visualizing the data from different
sensors are developed and compacted. We can also
visualize a walking person by animation (in Fig. 5), which is
mapped from different motion status of the person.
14
HARDWARE DESIGN
Data visualization
Fig. 5. A walking person by animation
15
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Analysis of signals from sensors inside the shoe
 6 parameters are selected and measured:
1) the force sensor installed at the toe
2) the force sensor installed at the left side of the heel
3) the force sensor installed at the right side of the heel
4) X-axis of accelerometer at the heel
5) Y-axis of accelerometer at the heel
6) degree of bend from the bend sensor
16
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Analysis of signals from sensors inside the shoe
Fig. 6 shows the sample data taken from the intelligent shoe
when a person wearing this shoe is walking at a speed
around 5km/h.
Fig. 6. Visulization of data
17
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Smoothing the curve of the data
 In general, errors will be unavoidably introduced into the
system. There are two kinds of errors.
 One is the error introduced from the wireless module.The
other is introduced from the abnormal contact between the
shoe and the human foot, which is the output of the
sensors that do not reflect the person’s intention and it
should be deleted.
18
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Smoothing the curve of the data
 These two errors will cause some abnormal peaks in the
output waveform.
 Here, exponential smoothing [7] is applied to minimize the
effect of the abnormal peaks without affecting the
performance.
19
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Smoothing the curve of the data
 The principle of exponential smoothing can be described
as follows:
St = α ∗ yt + (1 − α) ∗ (St−1 + bt−1) 0≤ α ≤ 1 (1)
bt = γ ∗ (St − St−1) + (1 − γ) ∗ bt−1 0 ≤ γ ≤ 1 (2)
Fig. 7. Result of exponential smoothing
20
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Mapping motion of foot to motion of a mouse-cursor
on the screen
 Based on the shoe-mounted data gathering platform, we use
the following mapping methods to achieve the goal:
1) Use accelerometer to produce motion of mouse cursor:
The accelerometer used here (ADXL202E) can output xaxis
acceleration and y-axis acceleration.
21
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Mapping motion of foot to motion of a mouse-cursor
on the screen
Fig. 8. Rectangle motion of the shoe in a plane (left) and its
corresponding output of x-axis acceleration (right)
22
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Mapping motion of foot to motion of a mouse-cursor
on the screen
Fig. 9. Motion of foot and its corresponding acc output
23
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Mapping motion of foot to motion of a mouse-cursor
on the screen
2) Use force sensors at the toe to produce single click of the
left button:
Force sensor at the toe is naturally mapped into a single click.
 When a user has clicked the shoe using his toe, a force will be
produced from the force sensor installed between the toe and
the shoe.
24
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Mapping motion of foot to motion of a mouse-cursor
on the screen
 single-click of left button of the mouse takes effect.
 This force will persist for about 0.5 seconds.
 The average force during this time is computed.
 If the force is bigger than the threshold, the function of
single-click of left button of the mouse takes effect
25
DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM
OF INTELLIGENT SHOES
Mapping motion of foot to motion of a mouse-cursor
on the screen
3) Use two force sensors at the heel to produce single click of
the right button:
here we use two force sensors to judge whether the user has
pressed the right button
26
EXPERIMENTAL RESULTS
• The toe of the shoe contacts the ground with a
force. By calculating the average value of this
force, we can decide whether the user is intended
to produce a left click of the mouse or not.
• However, not all toe motions can produce the
corresponding click of the mouse.
• We did the following experiments to evaluate the
performance of the click motion.
27
EXPERIMENTAL RESULTS
• A circle with the diameter 30cm was drawn
manually on the screen.
• Each of the three volunteers wearing intelligent
shoes performs a simple click of toe to produce a
point in this circle.
• If the click is successful, it will produce a new
dark point inside the circle.
28
EXPERIMENTAL RESULTS
Fig. 11. Result of click test
29
EXPERIMENTAL RESULTS
• We drew a circle as a test path, letting the users
draw several identical circles in the same place as
accurately as they could.
• Then we computed the deviation between drawn
circles and the original circle.
30
EXPERIMENTAL RESULTS
• If the deviation is bigger than the threshold, we
consider this circle an unsuccessful circle.
• After that, we compared the performance between
the ordinary mouse and our proposed ShoeMouse.
31
EXPERIMENTAL RESULTS
Fig. 12. Performance of drawing a circle using Shoe-Mouse
32
CONCLUSIONS
• In this paper, we have built a shoe-based sensor integrating
platform that can gather a lot of data inside the shoe.
• A novel input device called Shoe-Mouse is introduced by
applying the shoe platform as a user wearable interface.
• The Shoe-Mouse is especially designed for the people who
have difficulties in operating computers using hands. As
proved in the experiments, the Shoe-Mouse performs
satisfactorily in motion control and can partly replace the
ordinary mouse.
33
CONCLUSIONS
• People who do a lot of work using computers can also
benefit from this invention.
• In the future, more experiments will be conducted on
potential user groups to find out more about the
requirements for the systems
34
REFERENCES
 [1] K. N. Tarchanidis and J. N. Lygouras, “Data glove with a force sensor”,IEEE Trans. on
Instrumentation and Measurement, vol. 52, issue.3,pp.984-989, June 2003.
 [2] T. E. Starner, “Wearable computers: no longer science fiction”, IEEE Pervasive
Computing, vol.1, no.1, 2002.
 [3] R. E. Morley, E. J. Richter, J. W. Klaesner, K. S. Maluf, and M. J.Mueller, “In-shoe
multisensory data acquisition system”, IEEE Trans. on Biomedical engineering, vol.48,
no.7, July 2001.
 [4] J. Paradiso, E. Hu, and K. Y. Hsiao, “The cybershoe: wireless multisensor interface for
a dancer’s feet”, in Proceesings of Internaional Dance and Technology, Tempe,
Atizona,1999.
 [5] I. P. Pappas, T. Keller, and M. R. Popovic, “A novel gait phase detection system”, in
Proceesings of Workshop Automatisierungstechnische Verfahren fr die Medizin,
Darmstadt, 1999.
 [6] Stacy J. Morris and J. A. Paradiso , “A compact wearable sensor package for clinical gait
monitoring”, Offspring vol.1, no.1, pp.7-15, January 31, 2003.
 [7] A. Watts, “On exponential smoothing of discrete time series”, IEEE Trans. on
Information Theory, vol.16, issue.5, pp.630-630, Sep, 1970.
35