User Modeling: The Long and Winding Road
Gerhard Fischer
Center for LifeLong Learning and Design (L3D), Department of Computer Science
and Institute of Cognitive Science
University of Colorado, Boulder, CO, USA
Abstract. The long and winding road of user modeling is grounded in different epistemological assumptions exploring different dimensions of the problem. User-modeling
research has explored different domains, identified important distinctions underlying different approaches within user modeling research, and created a number of challenging
research problems. These issues are explored in the context of high-functionality applications and how our research over the last ten years has addressed the problems of making
high-functionality applications more usable, more useful, and more learnable with a variety of different user modeling approaches.
1. Introduction
User modeling is one of a number of research areas that intuitively seem to be winning propositions and worthwhile investments based on their obvious need and potential payoff. One area
comparable to user modeling is software reuse. The approaches seem to be appealing, natural,
theoretically justifiable, desirable, and needed (e.g., reuse can be justified by the fact that complex systems develop faster if they can build on stable subsystems). But in reality, progress in
these areas has been slow and difficult, and success stories are rare.
The research area has
– explored different domains such as: natural language dialog, human computer interaction, intelligent assistants, information retrieval, and high-functionality applications;
– identified important distinctions such as: adaptive versus adaptable components, explicit
versus implicit modeling techniques, user models versus task models, canonical versus
individual models, and long-term versus short-term models;
– created a number of challenging research problems, such as how to: (1) integrate different modeling techniques; (2) capture the larger (often unarticulated) context and what
users are doing (especially beyond the direct interaction with the computer system); (3)
identify user goals from low-level interactions; (4) reduce information overload by making information relevant to the task at hand; (5) support differential descriptions by
relating new information to known information and concepts; and (6) reach a better balance for task distributions between systems and users.
Some of these challenges will be illustrated in the context of the work that we have done to
make high-functionality applications more usable, more useful, and more learnable.
Gerhard Fischer
1
UM’99
2. High-Functionality Applications
High-functionality applications (HFAs) (such as Unix, MS-Office, Photoshop, Eudora, etc.)
are used to model parts of the world and not just to implement algorithms. They are complex
systems because “reality is not user friendly”. If you ask a 100 different people what features
they would like to have in a particular application, you end up with a very large number of
features. The design of HFAs must address two problems: (1) the unused functionality must
not get in the way and (2) unknown existing functionality must be accessible or delivered at
times when it is needed.
We have conducted a variety of empirical studies to determine the usage patterns of HFAs,
their structure, their associated help and learning mechanisms. All of these studies have led us
to the identification of the qualitative relationships between usage patterns of HFAs as illustrated in Figure 1.
The ovals represent users'
knowledge about the system's
'
D4
information space. D1 represents concepts well known,
easily employed, and used
D4
regularly by a user. D2 conD3 D2
D1
tains
concepts
known
vaguely and used only occasionally, often requiring
passive help systems. D 3
Figure 1: Levels of Users' Knowledge about a System's Information represents concepts users
believe to exist in the system,
Spaces
some of which lie outside the
actual information space. The
rectangle D4 represents the
actual information space of a system. As the functionality of HFAs increases to D4', little is
gained for users unless there are mechanisms to help them relate the additional functionality to
their needs.
The area of D4 that is not part of D3 is of specific interest to research in user modeling.
This is system functionality, whose existence is unknown to users. For the “D4 and not D3”
domain, information access (the user-initiated location of information when they perceive a
need for an operation) is not sufficient, but information delivery (the system volunteering information that it inferred to be relevant to the users’ task at hand) is required. Active help
systems and critics are required to point out to users functionality that may be useful for their
tasks and to help users to avoid to get stuck on suboptimal plateaus.
Figure 1 shows usage patterns of HFAs without taking the users’ tasks into account. There
is no reason for users to worry about additional existing functionality in D 4, if this functionality is not relevant to their tasks. However, if the system does provide functionality in D 4
related to users’ tasks, it is desirable to avoid having users unable to perform the task or do so
in a suboptimal or error-prone way because they do not know about this functionality. In
Figure 2 the gray rectangle T represents the information that is relevant to the users’ task at
hand, and the dots represent different pieces of functionality. Passive intelligent support systems supporting information access can help users to explore pieces of functionality that are
contained in D3 and T, whereas active intelligent systems supporting information delivery are
Gerhard Fischer
2
UM’99
needed for the functionality contained in T and not in D 3. The
T
D4 functionality in D4 outside of T is
often offered by push systems
such as “Did You Know” (DYK)
systems [Owen, 1986] or MicroD3
soft’s “Tip of the Day” [Horvitz,
D2 D1
1997], which throw decontextualized concepts at users.
“Experts” and Expertise in
HFA. “Experts” (users who know
everything about a system) no
Figure 2: Functionality and its Relevancy to the Task at Hand
longer exist in HFAs. Being an
“expert” is at best an attribute of
4 a specific context, rather than a
D1, U 2
D1, U 1
personal attribute. The different
Equation
Mail Merge
Editor
spaces of expertise (determined
by individual interest) are illustrated in Figure 3. In this multi, U3
D
kernel model, {D1, U i} means
1
D1, U 4
Referencing
the area of functionality that is
Collaborative Writing
well known to a particular user
Ui; for example: U 1 knows
about the equation editor; U2
Figure 3: Distributed Expertise in HFAs
about mail-merge functionality;
U3 uses a bibliography system
for references, and U4 knows about collaborative writing tools.
D
3. How Our Research Address the Problems Created by HFAs
Our research related to user modeling has attempted to address the challenges created by
HFAs. Active help systems [F ischer et al., 1985] were an early attempt to analyze the behavior
of users and infer higher-level goals from low-level operations [Horvitz, 1997; Nardi et al. ,
1998].
HFAs, as argued above, came into existence as environments that are useful for a large
number of different users (see Figure 3 ). In order to reduce their complexity, HFAs have often
migrated to a collection of domain-oriented subsystems each with their own templates, forms,
and their own associated wizards thereby being able to provide additional support for user
modeling and assistance not available in more general systems.
In our own research, we have taken this approach further by developing domain-oriented
design environments [Fischer et al., 1998]. These are environments which model specific domains (such as computer networks, user interfaces, kitchens, and voice dialog design [Sumner
et al. , 1997]) by allowing designers to engage in authentic tasks from their own respective
work practices. Domain-oriented design environments make computers invisible and enable
designers to communicate with domain-specific concepts, representations and tools. The domain orientation of these environments makes HFAs more usable, more useful, and more
Gerhard Fischer
3
UM’99
learnable by bringing the objects closer to the conceptual world of their users. In support of
user modeling, domain-oriented design environments provide
– specification components [Nakakoji, 1993] to allow users to enrich the description of
their tasks, and
– critiquing components [Fischer et al., 1998] to analyze and infer the task at hand in order
to be able to detect and identify the potential for a design information need and then present stored knowledge for designers. We have used the artifact (including its partial
construction and partial specification) combined with domain knowledge in the design
environment as an index to infer high-level goals from simple user actions. The existence
of the specification component has allowed us to complement generic critics (which can
be defined at design time) with specific critics whose behavior is dependent on the information provided by individual users at use time.
Domain-oriented design environments integrate a number of components, and the research
built around them has tried to developed principles and arguments for the trade-off between
adaptive and adaptable approaches [Fischer, 1993; Thomas, 1996]. Figure 4 shows a screen
image from the Voice Dialog Design Environment [Sumner et al. , 1997], which supports adaptation mechanisms to allow users to select specific rule sets for critiquing and to determine
the intervention strategy for the intrusiveness of the critics ranging from active behavior (every
step is immediately critiqued) through intermediate levels to passive behavior (users have to
explicitly invoke the critiquing system). Similar solutions can be found in modern spelling
correction programs. The need to customize and tailor critiquing systems to individual users’
objectives has also been explored in the domain of business graphs and use of color [Gutkauf,
1998].
Domain-oriented design environments pose a number of challenging problems for user
modeling. Contrary to Intelligent Tutoring Systems [Burton & Brown, 1982], they model
(open-ended) domains in support of self-directed learning and user-directed activities within a
domain. They are able to exploit domain models for user modeling [Hollan, 1990; Mastaglio,
1990]. At design time (when the system is developed), domain models including generic critiquing knowledge and support for specification, and end-user adaptation is provided. In these
open-ended systems, the task that users engage in cannot be anticipated: they have to be inferred or articulated at use time. Design environments support both adaptable and adaptive
components and tune system behavior to the specific needs of individual users and their tasks
[Gutkauf, 1998; Nakakoji, 1993].
Gerhard Fischer
4
UM’99
VM-Residential
Figure 4: Adaptation Mechanism to Control Different Critiquing Rule Sets
and Different Intervention Strategies
HFAs create challenging learning problems that are representative for numerous complex
systems. As illustrated with the above figures, nobody learns these systems completely, but
users acquire some base functionality and learn additional functionality on demand. Usermodeling techniques can effectively support learning on demand [Fischer, 1991] by helping
users to identify opportunities to learn additional functionality relevant to their task at hand and
to avoid people becoming stuck on suboptimal plateaus. User modeling techniques based on
logged user data can support the organization-wide learning of HFAs [Linton et al. , 1998].
4. User Modeling and Human-Computer Collaboration
Some of the beginnings of the long and winding road of user modeling were derived from the
need and desire to provide better support for human-computer collaboration. Collaboration in
this context is defined as “a process in which two or more agents work together to achieve
shared goals” [Terveen, 1995]. Some fundamental issues (such as shared goals, shared context,
control, (co)-adaptation, (co)-evolution, and learning) can be derived from this definition.
Human-computer collaboration can be approached from two different perspectives: an emulation and a complementing approach. The emulation approach is based on the metaphor that to
improve human-computer collaboration is to endow computers with “human-like abilities”.
The complementing approach is based on the fact that computers are not human and that huGerhard Fischer
5
UM’99
man-centered design should exploit the asymmetry of human and computer by developing new
interaction and collaboration possibilities [S uchman, 1987].
Historically, the major emphasis in user modeling has focused on the human emulation approach (see for example, [Kobsa & Wahlster, 1989], and the section “user and discourse
modeling” in [Maybury & Wahlster, 1998]). However, based on the limited success of the
emulating approach, the interest has shifted more and more to the complementing approach
[Bobrow, 1991; Fischer, 1990]. There is growing evidence that the problems of user modeling
in the complementing approach are more tractable, more feasible, and more desirable, as evidenced by their increasing influence in the design of commercial high-functionality
applications [Horvitz, 1997]. As the complexity of commercially available HFAs grows, and as
we see more computational assistants (such as agents, advisors, coaches, and critics) appear in
widely available commercial applications, a detailed, state-of-the-art analysis and understanding of how people learn, work and collaborate with and around HFAs will provide us with new
requirements for the design of user modeling components.
In the long and winding road that is not only behind us but also before us, our understanding of the trade-offs, the promises, and the pitfalls between adaptive and adaptable systems,
between push (information delivery) and pull (information access) technologies, and between
contextualized information representation and serendipity will hopefully lead us to new ideas
and new insights in the design of future human-centered systems supported by adequate user
modeling techniques.
Acknowledgments. I would like to thank the members of the Center for LifeLong Learning
and Design (L3D) who have contributed to the ideas and perspectives expressed in this paper.
A special thank you goes to my former students who, in their Ph.D. theses, explored important
issues in user modeling as they occur in high-functionality applications, domain-oriented design environments, and critiquing systems.
References
Bo b ro w, D. G. (1 9 9 1 ) " Di men s i on s o f In t eract i o n ," AI Ma g a z i ne, 1 2 (3 ), p p. 6 4 -8 0 .
Bu rt o n , R. R. & Bro wn , J. S. (1 9 8 2 ) " An In v es t i g ati o n o f Co mp u t er Co ach i n g fo r In fo rmal Learn i n g
Act i v i t i es." In D. H. Sl eeman & J. S. Bro wn (Ed s .), In t el l i g ent Tu t o r i n g S ys t ems , Acad emi c Pres s ,
Lo n d o n - New Yo rk , p p . 7 9 -9 8 .
Fi s ch er, G. (1 9 9 0 ) "Co mmu n i cati o n Req u i remen t s fo r Co o p erat i v e Pro b l em So l v i n g Sy s t ems ," In f o r ma t i o n S ys t ems , 1 5 (1 ), p p. 2 1 -3 6 .
Fi s ch er, G. (1 9 9 1 ) "Su p p o rt i n g Learn i n g o n Deman d wi t h Des i g n En v i ro n men t s ." In L. Bi rn b aum
(Ed .) In t er n a t i on a l Co n f er en ce o n t he Lea r n i n g S ci en ces (Eva n s t o n , IL), As s o ci at io n fo r t h e Ad v an cemen t o fCo mp u t i n g i n Ed u cat i o n , p p . 1 6 5 -1 7 2 .
Fi s ch er, G. (1 9 9 3 ) "Sh ared Kn owl ed g e in Co o p erat i ve Pro b l em-So l v i n g Sys t ems - In t eg rat i n g Ad ap t i v e an d Ad ap t ab l e Co mp o n en t s ." In M. Sch n ei d er-Hufs ch mi d t , T. Ku eh me, & U. Mal i n o ws k i
(Ed s .), Ad a p t i ve Us er In t er f a ces - Pr in ci p l es a n d Pr a ct i ce, El s ev i er Sci en ce Pu b l i s h ers , Ams t erd am, p p . 49 -6 8 .
Fi s ch er, G., Lemk e, A. C., & Sch wab , T. (1 9 8 5 ) "Kno wl ed g e-Bas ed Hel p Sy s t ems ." In L. Bo rman & B.
Cu rt i s (Eds .), Pr o ceed i n gs o f CHI'8 5 Co n f er ence o n Hu ma n Fa ct o r s i n Co mp u t i n g S ys t ems, ACM,
New Yo rk , p p . 1 6 1 -1 6 7 .
Fi s ch er, G., Nak ak o j i , K., Os twal d , J., St ah l , G., & Su mn er, T. (1 9 9 8 ) "Emb ed d i n g Cri t i cs i n Des i g n
En v i ro n ment s ." In M. T. May b u ry & W. Wah l s t er (Ed s.), Rea d i n g s in In t el l i g en t Us er In t er f a ces ,
Mo rg an Kaufman n , San Fran ci s co, p p . 5 3 7 -5 6 1 .
Gerhard Fischer
6
UM’99
Gu t k au f, B. (1 9 9 8 ) Imp r o vi n g Des i g n & Co mmu n i ca t io n o f Bu s i n es s Gr a p hs th r o u g h Us er Ad a p ti ve
Cr i t i q u i n g, Ph .D. Dis s ert at i o n , Mat h emati cs an d Co mp u t er Scien ce, Un i v ers i t ät -GH Pad erb o rn ,
Pad erb o rn , German y .
Ho l l an , J . D. (1 9 9 0 ) "Us er Mo del s an d Us er In t erfaces : A Cas e fo r Do mai n Mo d el s , Tas k Mo d el s , an d
Tai l o rab i li t y ." In Pr o ceed i n gs o f AAAI-9 0 , Ei g h t h Na t i o n a l Co n f er en ce o n Ar t i f i ci a l In t ell i g en ce,
AAAI Pres s/ Th e MIT Pres s , Cambri d g e, MA, p . 1 1 3 7 .
Ho rv i t z, E. (1 9 9 7 ) " Ag en t s wi th Bel i efs : Refl ect i on s o n Bay es i an Met h od s fo r Us er Mo d el i n g." In C.
P. A. J ames o n , C. Tas s o (Ed .) Pr o ceed i n gs o f t h e S i xt h In t ern a t i o n a l Co n f er en ce o n Us er Mo d el i n g (UM'9 7), Sp ri n g er Wi en -New Yo rk , Sard eg n a, It al y , p p . 4 4 1 -4 4 2 .
Ko b s a, A. & Wah l s t er, W. (Ed s .) (1 9 8 9 ) Us er Mo d els i n Di a l o g S ys t ems , Sp ri n g er-Verl ag , New Yo rk .
Li n t o n , F., Ch arro n , A., & Jo y, D. (1 9 9 8 ) OWL: A Reco mmen d er S ys t em f o r Or g a n i z a t i o n -Wi d e Lea r n i n g , at h t t p : // www.mi t re.o rg / t ech no l o g y / t ech _ t at s / mo d el i n g / o wl / Co ach i n g _ Soft ware_ Sk i l l s .p d f.
Mas t ag l i o , T. (1 9 9 0 ) A Us er -Mo del l i n g Ap p r o a ch t o Co mp u t er -Ba s ed Cr i t i qu i n g, Ph .D. Dis s ert at i o n ,
Dep art men t o f Co mp u t er Sci en ce, Un i v ers i t y o f Co l orad o at Bo u l d er, Bo ul d er, CO.
May b u ry , M. T. & Wah l s t er, W. (1 9 9 8 ) Rea d i n g s in In t el l i g en t Us er In t er f a ces , Mo rg an Kau fman n ,
San Fran cis co .
Nak ak o j i , K. (1 9 9 3 ) In cr ea s i n g S h a r ed Un d er s t a n d i ng o f a Des i g n Ta s k Bet ween Des i g n er s a n d
Des i g n En vi r o n men t s : Th e Ro l e o f a S p eci f i ca t i o n Co mp o n en t, Ph .D. Dis s ert at i o n , Dep art men t o f
Co mp u t er Sci en ce, Un i v ers i t y of Co l o rad o at Bo u l d er, Bo u l d er, CO.
Nard i , B. A., Mi l l er, J. R., & Wri g h t , D. J. (1 9 9 8) "Co l l ab o rat i v e, Pro g rammab l e In t el l i g en t Ag en t s ,"
Co mmu n i ca ti o n s o f t h e ACM, 4 1 (3 ), p p. 9 6 -1 0 4 .
Owen , D. (1 9 8 6 ) "An s wers Fi rs t, Th en Qu es t i o n s ." In D. A. No rman & S. W. Drap er (Ed s .), Us er Cen t er ed Sys t em Des i g n , New Per s p ect i ves o n Hu ma n -Co mp u t er In t er a ct i o n, Lawren ce Erl b au m
As s o ci at es, In c., Hi l l s d al e, NJ , p p . 3 6 1 -3 7 5 .
Su ch man , L. A. (1 9 8 7 ) Pl a n s a n d S i t u a t ed Act i o n s , Camb ri d g e Un i v ers i t y Pres s , Camb ri d g e, UK.
Su mn er, T., Bo n n ard el , N., & Kal l ak , B. H. (1 9 9 7 ) " Th e Co g n i t i v e Erg o no mi cs o f Kn o wl ed g e-Bas ed
Des i g n Su pp o rt Sy s t ems ." In S. Pemb ert o n (Ed .) Pr o ceed i n gs o f CHI 9 7 Co n f er ence o n Hu ma n
Fa ct o r s i n Co mp u t i n g S ys t ems , ACM/ Ad d i so n -Wes l ey , p p . 8 3 -9 0.
Terv een , L. G. (1 9 9 5 ) " An Ov erv i ew o f Hu man -Co mp u ter Co l l ab o rat i o n ," Kn o wl ed g e-Ba s ed S ys t ems
Jo u r n a l , Sp eci a l Is s u e o n Hu man -Co mp u t er Co l l a b o r at i o n , 8 (2 -3 ), pp . 6 7 -8 1 .
Th o mas , C. G. (1 9 9 6 ) To As s i s t th e Us er : On th e Embed d i n g o f Ad a p t i ve an d Ag en t -Ba s ed Mech a n i s ms , R. Ol d en bu rg Verl ag , Mü n ch en /Wi en .
Gerhard Fischer
7
UM’99