INTEGRATED AUTOMATED SYSTEM FOR THE IMPLEMENTATION OF
SKIN TESTS AND ALLERGY DIAGNOSIS
S. Karagiannis, Professor*
skar@teipir.gr
G.Syrkos, Professor*
gsyrkos@teipir.gr
C. Balafoutis, Professor*
cbalaf@teipir.gr
A.Dounis, PhD*
aidounis@otenet.gr
T.Chalastras, MD**
chalastras@hotmail.com
D. Papachristos*, Automation engineer
dimpapachristos@yahoo.gr
*Technological Institute of Pireaus, Department of Automation, P. Ralli and Thivon
250, 12244, Aigaleo, Athens – Greece.
**Department of Otolaryngology, G.Gennimatas Hospital, Athens – Greece.
ABSTRACT
This project presents the outcomes of the research which had been accomplished in the terms
of the ARCHIMEDES I program .The above project had been performed at the robotics
laboratory of the Automation department, at the Technological Institute of Piraeus. The
project refers to the design of an integrated system for the implementation of the relevant
allergy test, diagnosis of allergies and suggestion of a treatment. More precisely, the
subsystems that are presented, contain the following: The robotic system that is needed for the
implementation of the allergy test, the expert system for the diagnosis of the allergy and the
suggestion of the treatment and lastly the visual system which helps the previous ones. The
suggested system does not supplant the specialized doctor but it supports his task (diagnosis
and suggestion of treatment) and enhances the waiting period for the examined persons. In
addition it dwindles the possibilities of a fault to be provoked during the implementation of
the test (dermic, prick-puncture), due to the accuracy of the robotic arm. It is regarded as
innovative task (introduction of new technologies in medicine tasks), crucial towards our
country, where the increase of allergy incidents has become a common issue and the cost that
the allergy tests entail throughout Greece hinders the massive use of these methods. On the
contrary the use of such a system lowers the total cost (less human work-hours) and at the
same time due to the technological ability, it boosts the reliability and the accuracy of the
diagnosis.
1. INTRODUCTION
In the terms of the ARCHIMEDES* program, for the development of research projects in the
Technological Institutes, the research project with the following title “Intelligent Robotic
System for the Diagnosis of Allergies” was submitted and authorized. The research aims at
the application of the robotics and the artificial intelligence fields in the scientific field of
Allergiology – Immunology which constitutes a field of the medicine science, where the
implementation of an intelligent system will help to accomplish a faster treatment of the
patents, to minimize the total cost and to support the final diagnosis [8-10].
The automatization of the allergy diagnostic test will offer reliability, great rapidity,
accuracy and flexibility. The use of the robot relates to the execution of the test while the use
of artificial intelligence relates to the diagnosis and the proposal of a treatment. The suggested
system does not supplant the specialized doctor but it supports his task (diagnosis and
suggestion of treatment) and dwindles the waiting period for the examined persons. In
addition it minimizes the possibilities of a fault to be provoked during the implementation of
the test (dermic, prick-puncture), due to the accuracy of the robotic arm [8-10].
2. CONCLUSIONS OF THE RESEARCH PROJECT
During the development of the research project, stages of its study were completed more
specifically there were analyzed and outlined the following parts of the system:
2.1 Robotic System
The selection of the robotic arm will be based mainly on the analysis of the geometrical
design of the robotic arm and on the analysis of its degrees of freedom. From the analysis of
the above criteria we will be able to choose the appropriate kind of robotic arm that suits an
automated process of executing skin allergy tests. The analysis of the above criteria is
described below:
2.1.1 Geometrical Design of the Robotic Arm
The first criterion for the selection of the robotic arm relates to the geometrical design of the
robotic arm, which is going to determine the shape of the robot’s workspace. The latter is one
of the most basic parameters on which the selection of the robotic arm is based, in most
robotic applications. According to the general analysis of an allergy test and according to the
fact that the test will be executed on the front surface of the forearm, it is obvious that the
physical space, that is needed in order to execute the test safely, could have the shape of a
parallelogram or a cylinder. According to the above observation, we decided that, for an
automated allergy test we should use a Cartesian robotic arm or SCARA type robotic arm. In
the following paragraphs we will present the basic characteristics of the two types of robotic
arms that we have already mentioned.
a) Cartesian Robots
After a careful survey and analysis of the relevant references we concluded that the main
(regarding the other robots) advantages of the Cartesian robots are the following:
1. They can move linearly between any two positions in their workspace.
2. They have a quite big workspace ( The dimensions of the workspace’s base of a
common Cartesian robot that is sold in the market, can vary between 100 x 100 mm
and 2000 x 2000 mm)
3. They have great mechanical endurance ( because the endings of both of their axis are
supported)
4. They have a quite good repeatability (usually +/- 0.010 mm) in the majority of their
workspace
5. They have a quite good resolving power
6. They have stable accuracy throughout all their workspace
7. They have an easy kinematical dynamical analysis, and lastly
8. They can be programmed easily.
The basic drawback of the Cartesian robots is that their workspace is relatively small
considering the excessive required 3-D physical space that is needed for their operation. The
axis of the Cartesian robots have a relatively big length that is often exposed to dust and
humidity, something that hinders their maintaining. Lastly, another drawback of the Cartesian
robots is their operating speed on the horizontal plane that is usually lesser than the operating
speed of robots with a turning base. Nowadays there are a lot of Cartesian robots available,
that their specifications meet most of the needs in a great range of industry applications. Thus,
in combination with the appropriate controllers, the Cartesian robots can consist a powerful
candidate for the designed application of the automated execution of the allergy tests.
From the survey of the relevant references we can deduce that specialized Cartesian robots
start to participate in several medical applications like the example of the Bloodbot [13-15],
which had been designed for an automated process in which we could take blood samples
from the surface of the forearm. The Bloodbot, with the support of a specially designed
component can exert controlled force on the front surface of the forearm and after the relevant
analysis it can locate the right vein to take the blood samples. It continues, under the
continuous force control, with the penetration of the chosen vein with a needle ,and finally it
takes the blood samples. Conclusively, it is overt the fact that a specially chosen Cartesian
robot can meet the needs of the robotic system which can be used in order to execute the
proposed automazition of the allergy tests.
b) SCARA Robot
One of the basic advantages of the Scara robots is that the axis of the first three links are not
used (in contrast to the vertical articulated robots) in order to support the weight of the next
links or to support the load that the robotic arm will handle. These weights are supported
from the previous links and that is why the mechanical accuracy is of a Scara robotic arm is
quite high. In addition in a Scara robotic arm the motors of the first two joints can be placed
on the base of the robot. Thus these motors can be more powerful and consequently bigger,
something that, regarding the other robots, results in better operating speed and a better
ability to carry heavier load. Lastly, because of the two parallel links of the Scara robots ,they
can apply compliance in a desired plane and that is why they are ideal for assembly
applications. The Scara robots have been used in medicine too, with successful results. For
instance these kinds of robots are ideal in operations of replacement of the haunch or the knee
as well as in operations related to the treatment of the prostate gland cancer, where the Scara
robots have been used for the placement of “seeds” to the patient. These “seeds” are
radioactive isotopes and their diameter is less than 6.5 mm. The above process is similar with
the one of placing a new nib in a mechanical pencil. According to the above analysis we can
clearly deduce that in order to accomplice an automated execution of the allergy tests, we can
use both the Cartesian and the Scara robots. In addition, in the terms of the survey of these
robotic arms we can proceed to a comparison between the Cartesian robots and the Scara
robots, which will reveal, that the Cartesian robots have a better mechanical endurance and
repeatability rather than the Scara robots, but the latter have a higher operating speed. In
addition the Cartesian robots can be controlled more easily than the Scara robots.
2.1.2 Kinematical Degrees of Freedom
The robotic arm that will be used in an automated allergy test will have to be able to support
the two following tasks: a) It must be able to place the end-effector in several positions in the
physical space and b) it must be able to operate puncture at the chosen positions on where the
antigens have been placed. From the above tasks the first one requires three degrees of
freedom something that leads us to the conclusion that we need a robotic arm with at least
three joints from which at least one should be linear (because in the 3-D plane a position can
be determined more easily in Cartesian or polar coordinates). To decide how many degrees of
freedom the second task requires we have to examine how many movements the needle
system has to accomplish in order to have a successful puncture. It is overt the fact that a
successful puncture can be accomplished with the combination of two (controlled)
movements: One movement along the vertical axis towards the surface of the arm (by which a
controlled force to the skin of the arm will be exerted) and a second (reciprocation) revolute
movement around the vertical axis towards the surface of the arm (by which the surface of the
arm will be slightly “grazed”). Consequently, the second task requires two kinematical
degrees of freedom of which the first one determines a linear movement and the second one a
revolute movement. Because of the fact that these two tasks are not always independent to
each other, in order to have an automated allergy test we could use a robotic arm with tree to
five degrees of kinematical freedom. This fact leads us to robotic arms with three, four, or
even five joints but among them there should be at least one linear and one revolute joint.
From the above analysis we can deduce that in order to have an automated allergy test we can
use a robotic arm with three joints with the condition that the controller of the robotic system
would be perfectly efficient. But because in practical situations there is no perfectly efficient
control, and according to the kind of the task, we will have to choose a robotic arm which will
have at least four degrees of kinematical freedom. On the other hand, too many degrees of
freedom make the kinematical and dynamical analysis more complicated as well as the
control of the robotic arm too. Consequently in order to have an automated allergy test it is
logical to choose a robotic arm which will have four degrees of kinematical freedom.
From the things we have already mentioned above in relation to the Scara robots we can
see that this type of robot meets the criterion of the degrees of freedom, thus this type of robot
can be used in order to accomplish an automated allergy test. The Cartesian robots have three
joints and consequently they do not meet the criterion of the desired degrees of kinematical
freedom an they have to be disqualified. But, the example of the Bloodbot, reveals that if we
add a fourth axis that would allow the revolute movement of the end-effector of the robotic
arm ,then an appropriately modified Cartesian robot could be used for the accomplishment of
the desired automated allergy tests. The question of which is the best choice can only be
answered after comparative studies. For now a first denotation can be based on the above
comparison of the Cartesian robots and the Scara robots. Thus, according to the fact that some
of the characteristics of the Cartesian robots (e.g. repeatability, mechanical endurance and
accuracy, easy control etc) are met in higher degree than in Scara robots, our first choice is to
choose a Cartesian robot which will be equipped with a revolute axis which in contrast to
bloodbot it will be a servo-controlled. Of course, because in the terms of this research
program we have not the ability of purchasing and using a specially modified Cartesian robot,
the study and the analysis of an automated system for the execution of the allergy tests which
uses a Cartesian robotic arm will be accomplished only through a simulation which will be
basic on the widely known software for robotic simulations workspace. These studies have
already started and soon we will be in a position to present the first results.
2.2 Expert System
The proposed expert system constitutes part of intelligent robotic system of diagnosis of
allergies. The application of expert systems in the medicine is very interesting and has created
considerable importance systems of diagnosis. In the first stage of this work have been
implemented the design of an expert system that aims in the procurement of patient medial
background and in the search for suitable skin test selections. Skin testing is the tool used
most widely to diagnose allergies. The medical knowledge of specialized doctor is required
for the development of expert system. This knowledge is collected in two phases. In the first
phase, the medical background of allergic disease is recorded through the creation of
questionnaire in which the patient is called to answer. In the second phase, a set of rules is
created wherein each rule contains in IF part the facts and in THEN part the tests that should
be realized. The inference engine (forward reasoning) is a mechanism through which rules are
selected to be fired. It is based on a pattern matching algorithm whose main purpose is to
associate the facts (input data) with applicable rules from the rule base. Finally, the skin tests
are produced by the inference engine. The development of expert system is implemented in
CLIPS programming environment [11]. In the second stage of the project have been
developed the design of the user interface. The user interface with the expert system is of
fundamental importance as right data entry ensures to an extended degree the proposal of
suitable test from the expert system. Since the user can be either a doctor or a patient and by
no means an expert in information technology tools, visual programming with its abundance
of controls is considered as the most suitable choice for the implementation of user interface.
To this end, Visual Basic 6.0 is selected as the most suitable language for user interface [12].
In this article are presented the methodology of knowledge acquisition and knowledge
representation for the diagnosis of allergy problem, the structure of expert system and the
work of the following stage which will constitute the utilization of fuzzy logic in order to be
faced the ambiguity and the imprecision of the results of skin reaction from the image
processing.
2.2.1 Knowledge Acquisition and Knowledge Representation
The diagnosis of allergic diseases should always begin with the procurement of a patient
history and an appropriate physical examination. Skin testing is the tool used most widely to
diagnose clinical allergies. In this article is presented the design of an expert system aiming at
the search of allergic disease and the selection of suitable skin tests. The medical knowledge
of specialised doctor is required for the development of expert system. This knowledge is
collected in two phases. In the first phase, the medical background of allergic disease is
recorded through the creation of questionnaire in which the patient is called to answer. This
questionnaire includes: 1) the presented symptoms, 2) the hereditary and family history, 3)
search of allergy to inhalants (dusts, pollens etc), 4) search of allergy to foods, drugs, physical
agents and 5) habits and hobbies, vaccinations (immunization), rashes from contacts and
unusual reaction from insect stings. These answers constitute the facts in the expert system. In
the second phase, a set of rules is created wherein each rule contains in IF part the facts and in
THEN part the tests that should be realised. The inference engine (forward reasoning) is a
mechanism through which rules are selected to be fired. It is based on a pattern matching
algorithm whose main purpose is to associate the facts (input data) with applicable rules from
the rule base. Finally, the skin tests are produced by the inference engine.
The problem of the search allergy is implemented by methodology of rule based systems.
One of the well-know methods of representation of knowledge in the expert systems is the
productive representation as the CLIPS (production system). The overall organization of a
production system can be explained with the help of figure 1 where the three basic
components of a production system are: a working memory for data (facts), a production
memory for rules and an inference engine, whose function is to infer new facts from existing
facts and rules, to assert the new facts into the working memory and then to continue this
procedure of discovering new facts via the rules through the new store of facts in the working
memory until no further facts can be inferred.
Changes
Working memory
(facts)
INFERENCE
Matching
Production memory
(rules)
Agenda
Selection
One selected rule
Execution
Changes
Fig .1 Structure of a production system (CLIPS).
2.2.2 Rules for Recognition of the Results of Skin Reaction
A skin reaction is considered positive when it is formed redness, pomphus, and the grad of
positiveness is symbolized with crosses. By using the term pomphus we mean the contoured
prominence of the skin in different sizes and colors (pale, red, cyanic).
The image processing of the skin reaction states the size (diameter) of redness (from skin
test) and the existence or no of pomphus with the corresponding color. This information is
getting clear with language variables. The number of the tests is equal with the number of
identifications skin reactions.
The following rules describe the positiveness or the negetiveness of a skin reaction. These
rules are the knowledge base for the expert system which characterizes a skin reaction. The
characterization of the positiveness or the negativeness of a reaction with linquistic
description represents important information for the doctor.
1st Rule
IF the reaction is strongly positive (4+) with diameter of redness>>20mm and pomphus with
diameter >>5mm THEN the positiveness is characterized with (++++)
2nd Rule
IF the reaction is positive (3+) with pomphus with diameter 3-5mm, and redness with
diameter >20mm THEN the positiveness is represented with (+++).
3rd Rule
IF in the positive reaction (2+) there is only redness with diameter >20mm and pomphus with
diameter <3mm THEN the positiveness is symbolized with (++).
4th Rule
IF the skin reaction in which we can observe only redness with diameter <20mm THEN the
positive reaction characterized with one cross (+).
5th Rule
IF there are no changes in skin after the test THEN the reaction characterized as negative.
2.3 Optical system
There were designed: the first prototype of the optical system which operates with infrared
light for the locating of the veins in the skin, the system of polarized lighting and the system
of the filters. Previous research [1]-[7] has shown the effectiveness of vein imaging via the
use of infrared (IR) illumination. The same approach has been used in our system, but the IR
image is supplemented with normal visual images and augmenting visible, infrared and laser
illumination. More specifically there were developed:


The system of polarized lighting which is used for the locating of the areas which
contain blood vessels and cannot be seen under visible light. For the lighting system
there has been a mechanical construction with diffusers and polarized filters which
operate in the infrared spectrum. With the mechanical construction of the filter the
manual choice, of the axis of the polarity of the infrared light and the adjustment of
the light’s volume is now possible. The lighting is provoked by LEDs which emit in
an appropriate area of the infrared spectrum.
In combination with the lighting system, there has been constructed a system which
would back up the filters in order to be placed in front of the camera (Fig.2). More
specifically, the camera has been equipped with appropriate IR-pass filters, which
allow only the infrared light to pass through them while blocking the visible spectrum
from the camera. Thus the camera is more susceptible to the infrared area, and the
surrounding visible light (and its vacillations) can not affect the image. In addition, a
second polarized filter allows the choosing of photons, with specific axis of polarity,
from the image, something that enhances the results in the case that we have to deal
with instant reflections from the lighting system to the camera’s optics. The camera
that has been used in the construction of the prototype is a simple USB webcam with
a special C.C.D. sensor which has an appropriate design in order to respond to the
photons of the infrared light but it does not have embedded cutting filters of the
infrared light, as it is common in most of the cameras in the market. The parts that
were blocking the incoming of the infrared light have been removed from the focus
system of the camera, hence with the presence of the filter system the camera evolved
operating characteristics similar to these of an infrared camera. The drawback of this
solution is that the focus optics are not designed to operate in the infrared spectrum
but in the visible spectrum, and the use of infrared light ends up in various lengths of
focusing for the specific type of grass, something that results in a focus of inferior
quality in the images, and the above fact aggravates the computing part of the project
by adding more computations.
Fig.2 Image with infrared filter and without appropriate lighting of the environment.


A special base for the placement of the arm with appropriate background has been
designed, for the easier exportation ,of the interesting areas, from the image and for
the appropriate calibration during the placement of the camera (distances and angles)
so that the inverse transformation of the points of the image to real dimensions would
be possible.
A lot of expectant approaching algorithms have been tested and there have been
selected the best depending on the implementation and the operating. An issue of
major importance was the automated recognition of the optical characteristics of the
camera and of the zoom lenses that have been used (intrinsic and extrinsic optical
parameters) in order to be able to valuate the real distances and dimensions from the
image, and to minimize the optical deformations that are caused from the lenses
system and the filter system. More specifically:
1. In the basic process of the image, there have developed the algorithms related
to the prior-process and the prior-processing algorithms that undertake the
task of normalizing the images that are taken from the camera and to
eliminate the elements of noise and the non-interesting areas (background) so
that we can have a minimized volume of data for the next stages. In addition
we provoke an initial amplification of the images which is based on the
contrast and the balancing of the unequal light of the examined area.
2. A lot of approaches have been tested and to the issue of locating the natural
characteristics of the skin, like the areas with hair, veins e.g. The final
laboratorial pattern operates mainly with morphologically processing nonlinear algorithms, which yielded satisfying results.
3. Algorithms have been developed for the dispersion of the control points in
permissible areas (Fig.3). These algorithms accomplish an optimum
dispersion of the control points in areas that are characterized as permissible
areas from the stages of characterizing the skin areas. After observation the
need, of a more accurate characterization of the various areas which would be
based in natural characteristics ,has arisen, so the research must aim to the
fulfillment of that goal.
Fig.3 Permissible areas for the allergy test.
2.4 Interconnection of the Subsystems
Totally the automated system consists of the following cooperating subsystems [5], [6], [7]:
-- Subsystem for the importation of medical history (expert system)
-- Subsystem of the robotic arm for the execution of the tests (robot)
-- Subsystem of the robotic vision and diagnosis (optical expert system)
--Subsystem for a treatment suggestion (expert system)
The subsystems can be interconnected according to the following block-diagram:
Choice Position
Diagnostic
System
Image Processing
Therapy System
Robotic Arm and Tools
control
Robot Controller
Expert System
Camera
Webcam
robot
Fig.4 Interconnection of the automated system for the execution of the skin allergy test.
3. CONCLUSIONS
The development of an automated allergy test represents a crucial and innovative project in
our country, where the increase of the allergy incidents is a common issue and the high cost
hinders the application of diagnostic tests throughout Greece. On the contrary, the use of such
a system dwindles the cost and in the same time, due to its technological ability, it increases
the reliability, the credibility and the accuracy of the diagnosis. The implementation of the
system will free the medical and paramedical staff from the boring tasks.
The proposed system represents a project of applied research in a pivotal sector, like the one
of medicine field, which in the same time initiates new potentials for a collaboration of the
robotics and medicine, the information science and the artificial intelligence.
REFERENCES
[1] Herbert D. Zeman, Gunnar Lovhoiden, Harshal Desmhukh, Design of a Clinical Vein Contrast
Enhancing Projector, Proceedings of SPIE, Vol.4254, 2001, pp.204-215.
[2] Gunnar Lovhoiden, Harshal Desmhukh, Herbert D. Zeman, Clinical Evaluation of Vein Contrast
Enhancement,
Proceedings of SPIE, Vol.4615, 2002, pp.61-70.
[3] Gunnar Lovhoiden, Harshal Desmhukh, Carlos Vrancken, Yong Zhang, Herbert D. Zeman, Devin
Weinberg,
Commercialization of Vein Contrast Enhancement, Proceedings of SPIE, Vol.4958, 2003, pp.189200.
[4] Herbert D. Zeman, Gunnar Lovhoiden, Harshal Desmhukh, Optimization of subcutaneous vein
contrast enhancement, Proceedings of SPIE, Vol.3911, 2000, pp.50-57.
[5] Gunnar Lovhoiden, Harshal Desmhukh, Herbert D. Zeman, Prototype vein contrast enhancer,
Proceedings of SPIE, Vol.5318, 2004, pp.39-49.
[6] Philippe Schmid-Saugeon, Joel Guillod, Jean-Philippe Thiran , Towards a computer-aided
diagnosis system for pigment skin lesions, Computerized Medical Imaging and Graphics, Vol.27,
No.1, 2003, pp.65-78.
[7] Ph. Schmid and S. Fischer, Colour Segmentation for the Analysis of Pigmented Skin Lesions,
Proceedings of the Sixth International Conference on Image Processing and its Applications,
Vol.2, 1997, pp.688-692.
[8] St. Karagiannis, I. D. Krikos, E. Karachisaridis, D. Papachristos, Automatic System of Diagnosis
Allergy, Conference Proceedings, 1st Conference Modern Control System Technologies,
Technical Chamber of Greece, Athens, Dec. 1996.
[9] St. Karagiannis, I. D. Krikos, E. Karachisaridis, D. Papachristos, Robotic Application in
Immunology, Conference Proceedings, Technology and Automation, TEI Piraeus, May 1996.
[10] St. Karagiannis, I.D. Krikos, E. Karachisaridis, D. Papachristos, Control Algorithm of
an Automatic System of Diagnosis Allergy with the use of Robot, Conference Proceedings,
Computer Systems and Circuits, Naval Academy, Piraeus, July 1996.
[11] St. Karagiannis, A. I. Dounis, T. Chalastras, S. Frousiniotis, D. Papachristos “Building Expert
System and User Interface for Allergy Skin Tests”, Conference Proceedings 2nd International
Conference IT &Q, Spetses, June 2005.
[12] St. Karagiannis, A. I. Dounis, T. Chalastras, P. Tiropanis, D. Papachristos, “Design of Expert
System for Search Allergy and Selection of the Skin Tests Using Clips”, International Journal of
Information Technology, Vol. 3, No. 2, pp. 74-77, 2006, ISSN 1304 -2403, World Enformatika
Society.
[13] A. Zivanovic and B. Davies, A Robotic System for Blood Sampling, IEEE Transactions on
Information Technology in Biomedicine, ISSN: 1089-7771, Vol.4 No.1, pp.8-14, March 2000.
[14] A. Zivanovic and B. Davies, An Automated System for Vein detection and Needle Insertion
Proc. First Int. Workshop on Haptic Devices in Medical Applications, pp.54-65, Institute for
Process Control and Robotics, University Karlsruhe, D76128 Karlsruhe, Germany, June 1999.
[15] A. Zivanovic and B. Davies, The Development of a Haptic Robot to Take Blood Samples
from the Forearm, Lecture Notes in Computer Science (Medical Image Computing and
Computer-Assisted Intervention - MICCAI 2001), W. Niessen and M. Viergever (Eds.),
Issue 2208, Springer-Verlag, ISSN 0302-9743, pp. 614–620, 2001.
*
This research has been conducted within the framework of the "Archimedes: Funding of research
groups in TEI of Piraeus" project, co-funded by the European Union (75%) and the Greek Ministry of
Education (25%).