From: AAAI Technical Report FS-94-05. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved.
Faculty
The Artificial
of Computer Science
Intelligence
Course at the
in the Polytechnic
University
Asunci6n G6mez
Knowledge SymmsLabontmy
StanfordUniversity
701 WelchRoad, Building C
Palo Alto, CA,94304, USA
Tel: (415) 723-1867
Fax: (415) 725-5850
Em~:gome~@hpp.mnford‘edu
Amn@fi.upm.es
Abstract
piper preNnuthe experience of teaching an
Artificial Intelligencecounteat the Facultyof Computer
Science in the Polytechnic University of Madrid,
Spain.3]re objectiveof this couneis to introducethe
to this field, to l~parethemto conUibute
to
the evohnion
of the technology,andto qualifythemto
solve problemsin the real worldusing Artificial
Intelligence technology. The curriculum of the
Artificial Intelligencecourse,whichis integratedinto
the Artificial Intelligence Department’sprogram,
allowsmto educatethe studentsin this seine mi~the
monoBrapluc
teacMngmethod.
1. Introduction
Thetransfer of knowledgeand technologybetween
universities and compmfies
has becomea key factor
in the progress of the most developedcountries.
Rightnow,in the Patificial Intelligence(ADfield in
Spain, A/technologyis in a process of transition
fromthe universities andresearchlabcratori~to the
marketplace. In this transition, universitiesandthe
knowledgetaught in them, play a very important
role. Sometimes, the quality of business
appfications is the direct outcomeof our teaching
inaice,.
"Fnis paper mmmarizes
in section Twothe state of
AI in Spain. Section Three describes the basic
concepts that the program covers, and the
interactions of the AI courses with the whole
curricula of the Faculty of ComputerScience.
Section Four shows howthe instruction of the
Artificial Intelligence course at the Faculty of
ComputerScience at the PolytechnicUniversity of
Mmhidis perfom~using the monographicteaching
method.Section Five includes an evaluationof the
success of the monographk
method.FinAlly, section
Six analyses its advantagesand howthis method
improvesthe students’ capabilities to use AI to
solve problemsin the real world.
of Madrid
Natalia Juristo
Facultad de Infonn~ca
UniversidadPoli~m/cade
~mpus de Montegancedo m
28660Boadilla clel Monte
Madrid,Spain
Tel: (34-1) 336-6922
Fax: (34-1) 336-7412
Email: Natali~fi.upm.es
2. TheState of Artificial Intelligence
in Spain
Thefirst step in solving a problemis to makeclear
its existence.In this sense, if wewuntto transfer AI
technology from universities
and research
laboratories to the marketplace, it is necessaryto
analyse its evolution over the time. Thatmeans:to
learn fromthe past to knowthe previoussuccesses
anderrors, to identify the strengths and weaknesses
that AI has r/ght now, and to project where AI
should go and why.
This approachwill give us a realistic perspective
about howhealthy AI is nowin Spain. If we know
our weaknessesin the universities, in the research
laboratories,
in the companies, and the
commumcationproblems amongthem, and if we
comparethe Spanishsi,,~on with the international
circumstances, then we can gather enough
information to carry out a multidisciplinary
approachto treat these weaknesses.In this way,the
university, as the engine of the transfer of
knowledge
andthe pioneerof newresearchlines, c4m
play the mainrole in fltis set of private andpublic
interactions among
iustimtious.
The following summarizesthe most important
points in a recent study (Juristo, Mat~,& Pazos
1994) about the past, present and future of
knowledgeengineering in Spain (the survey also
presentsa descriptionof the past, lnesent andfuture
ofA~in general).
Themosti mport~terrors in the past werethat:
80%of the productsdevelopedwere prototypes,
63%of the failures were due to a bad task
selections, and the remainder was due to a
wrongexpert evaluationand/or the use of ad hoc
methodologiesfor knowledgeacquisition and
expert systemsdevelopment.
b) At this moment, the Faculty of Computer
Science carries out two complementary
approachesto transfer AI technologyfromthe
laboratories to the marketplace. Thefirst one
focusesour efforts in the selection of the task
and the developmentof a sound methodology
with an associated life cycle. Thesecondone,
knownas CETFICO(Center of Technology
Transfer in Knowledge
Engineering),wasset up
with the aims of: technological exploration;
identification andclassification; research and
education;technologytransfer and transition,
involving technologymaturation; technology
disseminationandtechnologyinsertion.
In the samesurvey, Stink showsthe relationships
betweenthe numberof undergraduatesin Computer
Science programsin different schools (from 12 to
2,700) and different AI-related courses (from I
800). In the 1993/1994academicyear in the School
of Computer
Scienceat the PolytechnicUniversitity
of Madrid, there were 2,500 students in the
undergraduate program, 400 of them took the
Artificial Intelligence course, and 300 the
Knowledge
Engineeringand Expert Systemscourse.
The Artificial Intelligence course coventhree
hours per week over nine months. The two
objectives of this courseare: to guaranteea solid
instruction in the foundationsof AI, and to apply
the conceptslearned to solve problemsin the real
world. Its prerequisites are: Mathematics,Logic,
Prognmmfing,
and Statistics. Thecurriculumof the
Artificial Intelligence course is divided in the
followingDidacticUnits:
c) Thefuture can not be predicted,but weexpecta
fall integration betweenSoftwareEngineering
and KnowledgeEngineering products in the
marketplace in SpAin_
3. Artificial Intelligence Courses
The Faculty of ComputerScience’s curricula is
comd~tentandcoherentwith the computingcurricula
proposed by ACMand IEEE in 1991 (ACMHEEECS 1991). The whole curricula cover not only
undergraduatebut m advancedand deepcurricula in
diffment computerscience areas for graduates in
computerscience and other disciplines. In its
undergraduate
curricula, the Artificial Intelligence
courseis the first AI coursethat an undergraduate
student c4mtake in this area in the fourth (of six
years) of the ComputerScience undergraduate
curricula. Tiffs courseis oneof the 14 coursesthat
the Artificial Intelligence Departmentof the
ComputerScience Faculty often to undergraduate
students.
In a teaching AI survey, D. Strok (Strok 1992)
says: "On average, each School offers two
undergr~,-~and tluee graduateAI-relatedcourses",
and "while most Schools don’t require an
undergr~_
_,_,,,,. AI course,as manyas tlnee qumten
of
computerscience majorstake it as an elective". In
comparison, the Faculty of ComputerScience’s
curriculaoften:
Unit L
Unit IL
Unit HI.
Unit IV.
Introduction
to
Artificial
Intelligence
Lisp
Formulation and Modelization of
problemsin AI
Search: blind search, heuristic
sea~h and advenmy se~
Knowledge
Planning
Unit V.
Unit VI.
Unit VH.
Unit VHI. Natural Language and Automatic
Translation
The relationships with other undergraduateAI
coune~ age:
a) After this course, the student must take the
mandatory course named Knowledge
Engineeringand Expert Systems. It deals with
methodologies to build expert systems and
uncemiutymsnageme~
a) Twomandatory courses called Artificial
b) 1~e students maychoosesomeof the following
Intelligence and KnowledgeEngineering and
Expert Systems, five optional courses, and
mmy m to undergrKlu~ ~ndents.
optional coursesin their fifth and sixth year.
For the fifth year:. Computational
Perception,
Models and Simulation and Computational
Theory. For the sixth year: Complexity of
Algorithmsand AlgorithmicLogic.
b) For graduate students, the graduate curricula
offer a MSccourse in Knowledge
Engineering
(G6mez-P6rez
&Juristo 1993) and 14 courses
for Fa.D. m~dents.Thesecourses provide adeep
and advanced analysis in AI areas like:
KnowledgeEngineering, Methodologies for
KBS,Software Engineering and Knowledge
Engineering, Robotics, KnowledgeSharing,
Fuzzy Logic, Logic Programming, Neural
Netwodr.s,and Lemnino.
4. The Monographic Method
Althoughthere is muchliterature about different
teachingmethodsfor different goals, environments,
and subjects, the ideal teaching methodis not yet
known.In the courseof ArtCicialIntelligencein the
82
Faculty of ComputerScience at the Polytechnic
University of Madrid,westarted four years ago an
optional method called the MonographicMethod
(Pazos 1988). The Monographic Method (MM)
divides the Artificial Intelligence course into
modulescalled DidacticUniu. This teaching method
is consistent with Bloom’s Taxonomy(Bloom
1956) about educational goals. This methodis an
active learning methodfor students becausethey
play the mainrole in it, workinghard to apply AI
concepts in problemswhich resemble real world
applications. The methodis performedalong the
followingsteps:
.
home.This task implies the decompositionof
the probleminto subpmblems,modularization
of tasks, howthe theory that they havestudied
at homecan be applied to solve the practical
monographicwork, and so on.
.
Generaloverviewof the didactic unit, its place
in the Artificial Intelligencefield, andthe kind
of problems that can be solved with the
techniques coveredby the unit. Thelecturer
providesto the studentsthe basic andnecessary
knowledgeof the didactic unit in a clear,
cohepmt, open to discussion, and extendible
MasterLesson. The masterlesson explains the
basic techniques in each didactic unit, the
conceptual differences between them, and
provides references to extend the knowledge
acq~ in class, l~e students will apply the
conceptslearned in class and by themselvesin
the resolution of problemswhichresemblereal
world problems. Each master lesson should
coverthe followingfive subjects:
4.1 Focus the attention of students on the
relevantpoints.
4.2 Connect the monographicwork with the
knowledgestudied in class and at homeby
themselves.
4.3 Solve some theoretical
questions.
.
I.I Goals.
1.2 Presentationand expositionof the contents
in an ordered way, mixingtheoretical and
and conclusions of the addressed
1.3 Summary
aspects.
.
and practical
Whenthe monographic work is done, the
groups summarizethe worksperformedin the
previous steps by the lecturer, the works
accomplished
in groups, in class, and at home.
This step is carried out followingthe following
script:
5.1 The lecturer reviews in a few minutesthe
activities to be solved in the monographic
work, whythey wereproposed,the goals to
be reachedand whichof themare going to
be analyzedanddiscussedin class.
practice
knowledge.
.
Wheneach group has discussed and analysed
the problem, the discussion and analyses
involve the entire group. During this step,
some groups explain during the class their
partial conclusionsandthe lecturer presents a
partial summaryof the useful results.
Interactions betweengroupsare useful to show
and comparedifferent approachesto solve the
sameproblem.Theobjectivesof this step are:
1.4 Refenmces
to be censulted.
5.2 Each group, itself, will agree howto
explain the activities proposed by the
lecturer.
1.5 Thelecturer proposes a Monographic
work
relatedwiththe subjectof the didacticnnlt~
A monographicworkcovers practical, and
sometimestheoretical, aspects that allow
the student to learn in deepthe techniques
covered by the didactic unit using the
refelenc~
~
5.3 The lecturer selects randomlysomegroups
and, for each group, a spokesperson is
randomlychosen. The spokespsrson will
explain howhis/her group has perfonned
eachoneof the activities.
5.4 Thelecturer mmnnarizes
the advantagesand
limitationsof the different approaches.
student, in an individual and active way,
learns by himtelf or herself when(s)he starts
the monographicwork using the reconnnended
refermces.Thesereferences include classic and
recognized papers and books, and sometimes
newlines of research in the area coveredby the
didacticumit.
StepsTwoto Five are repeatedfor each didactic
un~in the Artificial Intelligencecourse.
.
Afterthis study, studentsgatherin smallgroups
of tinge or four in order to analyzeanddiscuss
the questionsindividuallystudiedand solvedat
83
Whenallthedidactic
units
hasbeentaught,
each student performsan individually written
examabout the wholetheoretical course. If the
results are favorable the methodends and the
student passes successfully the Artificial
Intelligencecourse.
S. Evaluation of the Monographic
Teaching Method
The monographicmethod(MM)has been applied
both the Artificial Intelligent (AI) courseand the
Knowledge Engineering and Expert System
(KE&ES)course. In the AI course, during 1993
there wereabout 400 students, while in the KE&ES
coursethere were300.
For the evaluation (Juristo 1993) of the method
weused two similar experiments. Onefor the AI
course, and the other for the KE&ES
course. We
perfonued
theexperiments
withtwodifferentcourses
to msme
that:
a) Theresults of the experimentfor the AI course
weregenerated becauseof the method,and not
because of any special attribute of the AI
subject.
werethe samefor
b) Theresults of the experimmt
both courses,andthereforeccmect.
T’dl herewehavedescribed the premisesof our
experiment.To carry out an exper/ment,besides the
inemisesweneed: first to establish the hypothesis
wewantto verify; then, the empirical checkingor
contrastof the hypothesis.In our case:
a) Weapplied the statistic inference (Freeman,
1970)to carry out the experiment.
consists in dividing the students
b) Theexlmitmm
of the AI coursein four groupsof 100students
each. In two of them weteach following the
monographic method. In the two others we
teach through the old traditional teaching
method.For the KE&~
coun~we created three
groupsof 100 students each. In only one group
we followed the monographicmethod.In the
two others weapplied the old method.
c) Thenull hypothesis(H0)stands that the results
of the experiment will be the same for the
groups applying the monographicmethodtlum
for the othen.
4 Tuealternative hypothesis(HI), whichis the
onewewentto verify, says that the results will
showa difference betweenthe groupsusing the
MM
and the others.
e) Tobe exigents,weaskedfor a significancelevel
of 0.01 (it is usually enoughwith 0.05 ~ts
significancelevel for experiments).NOTE:
It is
not possible to stablish an exact limit to the
questionwhenthe probabilityof a result is low
enoughto reject the null hypothesis. However,
traditionally, a result is considered rare or
uncommon
whenit rams out five of 100 times.
That is, whenthe result has a probability of
0.05. WhenH0 is rejected because it has a
probab’dityof 0.05 or more,it is said that the
result is significative at the level 0.05, and,
therefore,0.05is the significancelevel.
To analyze the results of the experiment, we
perform two tests on the marks obtained by the
studentsin the different groups.Theresults of the
tests were:
Bythe parametrictest, the marksobtained by
the students in the MM
groups(2 for AI, 1 for
KE&ES)
were 2.5 points/10 points higher than
in the other groups. Thedifference amongthe
marks in the
MM groups
was
0.3points/10points.
b) Usingthe non-parametrictest of X2 the results
were even better. For the X2 test with three
degreesof freedomand0.01 sienificance level,
the critical point is 11.3. Thevalueobtainedin
our case for the X2 test applied betweenany
MM
group and auy traditional group was more
than twicethe critical point. Thevalueobtained
for the X2 test betweenthe monographic
groups
or betweenthe traditional groupswasalwaysno
significative. Therefore,there exist significant
differences betweenthe results obtainedusing
the MM
and those got using the traditional
method.
c) The line (curve, function) representing the
marksobtained by all the students, using the
traditional methodwasslanting (bias) towards
the fails, while using the MM
was slanting
over the highest marks. The meaningof these
results is that the MMis more efficient
regardingthe successof the students, than the
traditional evaluation through exams,in more
than 600%.That is, the evaluations of the MM
are passed six times morestudents than the
examsof the traditional method.
Fmally,wewouldlike to point out that according
to our information, most of the american
universities use teaching-learningmethodshalf-way
the MM
and the traditional spanish methods,being
closer to the MM.Whilein Europe,the methodsare
also half-waybut uemerthe traditional.
Monograpldc
future, make easier their adaptation to the
continuouschangesof this area.
Me~od
Marks:O to 5
5to7
7to9
9to10
10
42
31
17
Remand
Frecue~y
1~ AI Group 10
2n~ AI Group 12
1st KE&ESGroep 9
43
44
40
30
29
32
17
15
19
Expecud
Frecuency
.
Students apply A/technology to solve specific
and constrained problems in the real world.
Consequently, they have the ability to detect
what kind of problems can be solved using AI
technology. They are also more qualified than
before to work as a team in companies that
require AI by itself or AI applied to Software
Engineering
products.
Acknowledgments
Tradlflolai
Method
Marks:Oto 5
Fzpected
Frecmm~
Remarked
grecu4mcy
3th
#th.
2nd.
3th.
45
AI Group 43
AI Ghroep 46
KE&KS
Greep 50
KE&ES
Group 41
5to7
7to9
9 to lO
33
16
6
Thanks to the "Minigterio de Educacion y Ciencia"
in Spain for the grant to be at the Knowledge
SystemsLaboratory in Stanford University.
References
35
33
36
32
17
15
11
19
5
6
3
8
ACM/IEEE-CS. 1991. Report of the ACM/IEE~
CS Joint Curriculum Task Force. Computing
Curricula 1991.
Bloom, B. 1956. Taxonomia of Educational
Objectives:
Handbook: Cognitive Domain. D.
Mckay. NewYork.
6. Conclusions
Freeman, H. 1970. Introduccion a la Inferencia
Estadistica . Trillas. Mejico.
"llte experienceacquuedin the last four years allows
us to conclude that although some points can be
improved, the results achieved by the students who
optienally chose the Artificial Intelligence course
using the monographic method is more than
satisfacto W."Ihe most important characteristic that
the monographic method provides is that it
combinestwo opposite teaching slrale~es.
G6mez-l~rez. A., and Juristo, N. 1993. Integrated
Curricula for a MScin KnowledgeEngineering and
for a MScin Software Engineering. In Proceedings
of the International
Computing Congress.
Hyderabab.India. 309-319.
The first strategy starts when the lecturer
describes techniques and how these techniques
can be applied to solve several kinds of
problems.
Juristo,
N. 1993. Evaluaci6n del Metodo
Monogr~ico. Informe Intemo. Facultad de
lnform(ttica. Universidad Polit~cnica de Madrid.
The second strategy starts when the lecturer
proposes to the students a practical, and a few
times theoretical, monographicwork. At this
time, rite student has to look for newtechniques
that attempt to solve the problemposed.
Juristo, N.; Mat6, J.L.; and Pazos, 1. 1994.
Knowledge Engineering in Spain: Research,
Development and Demostrtion (R&D2) through
time. In J. Liebowitz. WorldwideExpert Systems:
Activities andTrends. PergmnenPress.
I~ advantages of the combination of thesetwo
slrategiesare:
Pazos, J. 1988. Proyecto Docente para Concursara
Catedrdtico de Universidad. Fa~ltsd de lnf~a
de la Universidad Po~ca de Madrid. Madrid.
,
o
.
2.
Mad~Spa~
Spain.
Each student is motivated during the whole
learning process. In the monographicmethod,
students play the mainrole.
Strok, D. 1992. TeachingArtificial Intelligence: An
Expert Survey. IEEE Expert. April 59-61.
The students are qualified to follow the
evolution of the technology further into the
85