ARTEMIS: Learner-Centered Design of an Information
Seeking Environment for K-12 Education
Raven Wallace, Elliot Soloway, Joseph Krajcik, Nathan Bos, Joseph Hoffman, Heather Eccleston
Hunter, Dan Kiskis, Elisabeth Klann, Greg Peters, David Richardson, and Ofer Ronen
University of Michigan
School of Education
610 East University
Ann Arbor, MI 48104
313-647-7877
ravenmw@umich.edu
ABSTRACT
Learners use software for different reasons and with
different skills and motivations than other users. Using
concepts of learner-centered design (LCD), we developed
a user interface for supporting learners as they use digital
information resources in inquiry-based science
classrooms. Learner needs are categorized in five areas:
content knowledge, technology knowledge, strategic and
metacognitive knowledge, and motivation.
Results of
research on problems encountered by students as they
engage in information seeking are used as the basis for
applying LCD, by identifying some specific problem areas
learners encounter: engaging in a process, generating
search terms, staying on task, and evaluating information.
Solutions offered through the Artemis interface are
described.
Keywords
Learner-centered design, information seeking, digital
libraries, K-12 Education
INTRODUCTION
In the fall of 1995, researchers from the University of
Michigan Digital Library (UMDL) began observing use of
digital resources in 6th and 9th grade classrooms to gain
understanding of classroom uses and learner needs for
information seeking.
Initially, students used Web
browsers and search engines in their science classrooms to
complete inquiry-based assignments. Our observations
convinced us that standard Web tools are not adequate for
learners as they engage in information seeking in
classrooms. For example, search engines return too many
hits, and too many of them are useless or irrelevant;
students lack a stable context in which to work and are
easily lost or distracted; and Web browsers are
Wallace, R., et al, (1998). ARTEMIS: Learner-centered
design of an information seeking environment for K-12
education. Proceedings of CHI98: ACM Conference on
Human Factors in Computing Systems, Los Angeles, CA,
pp. 195-202.
School of Information
550 East University
Ann Arbor, MI 48104
313-936-1562
soloway@umich.edu
disconnected from software which lets students use what
they find. Artemis is an interface designed in response to
our classroom observations, to support K-12 students
access and use information in the University of Michigan
Digital Library.
This problem is of particular interest for two reasons.
First, we are addressing a user population – K-12 learners
– which is often overlooked in the user-centered design of
general software tools. Second, the growing enthusiasm
for connecting schools to the Internet, and the increasing
ubiquity of connected classrooms, create a pressing need
for closer scrutiny of the design and use of tools which
contribute to educationally sound uses of the vast and
heterogeneous resources available on the Internet. This
paper describes the rationale for the design of Artemis,
including an overview of the UMDL; explanation of our
conception of learner-centered design; description of the
contexts for initial implementation of Artemis and the
learning problems associated with those contexts; and
explanation of the solutions Artemis provides to the
problems of learning with digital information resources.
The design which has been through three major revision
cycles, has followed an iterative process, with revision of
the software based on observation of use in classrooms
and test environments.
THE UMDL/MYDL PROJECTS
Artemis is part of the University of Michigan Digital
Library Project (UMDL), a project of the Digital Libraries
Initiative. The project's mission is to create an architecture
and software infrastructure for development of a digital
library open to multiple, heterogeneous collections. The
structure and contents of collections are constrained only
by their ability to communicate with agents in the UMDL
infrastructure about contents, search protocols, and other
metadata. The UMDL project includes testbed collections
in the domains of earth and space science.
Within the agent architecture of UMDL, multiple
interfaces are allowed, each with its own agent which
communicates with other agents in the UMDL
infrastructure to find information, to negotiate prices for
copyrighted information, to browse collections and
thesauri, and to register materials. [1] Just as different
collections can be specialized by content domain or user
characteristics, so interfaces to UMDL can address
particular user groups. Artemis is an example of such an
interface, designed particularly for K-12 students learning
through a process of inquiry.
Middle Years Digital Library (MYDL) focuses on
deployment of UMDL in middle school classrooms,
following the initial deployment of UMDL in high school
classrooms.
Between 1995 and 1997, researchers
observed over 1000 students in over 30 sixth and 15 ninth
grade classrooms as they learned to find and use digital
information resources. Classes began by using Web
browsers and resources while UMDL was being
developed, with initial tests of Artemis beginning in
February, 1997. [2]
TWO KEY ASPECTS:
INFORMATION SEEKING
Learner Needs
LEARNER
NEEDS
AND
The principles of learner-centered design (LCD) [3]
recognize the fact that students differ in a number of ways
from professionals who use computational software.
Table 1 provides a summary of the dimensions of learner
needs which require particular scrutiny as part of a
process of learner-centered design (LCD). Each
dimension is more fully explained below.
Unlike professionals using software to do their job,
learners do not initially know the content with which they
are engaging. They lack domain knowledge and must be
supported as they engage in inquiry, in ways that allow
them to build coherently on any background knowledge
they have. A related area of difference between learners
and other users is in their technology knowledge.
Students may need particular help in using new
technologies to insure that the tool is the background, not
the purpose of the learning. In order to learn content they
may need strategies particular to the type of work they are
doing. For example, they may need to learn to read a
graph, or to interpret a map. Thus, students differ in
terms of their strategic knowledge both generally (i.e.,
strategies for reading and comprehending text) and
specifically within the domain of investigation (e.g.,
reading a map.)
Another area of difference between learners and other
users is in metacognitive knowledge. This refers to
students knowledge of strategies for monitoring their
progress and thinking about their thinking. Metacognitive
strategies include asking themselves such questions as
“Have I seen this before?” “Where am I in this process what do I need to finish?” “Do I understand this?” [4]
Metacognitive knowledge implies the abilitiy to assess
one’s content knowledge and learning strategies. This is
clearly a type of knowledge which students lack in varying
Domain (content) knowledge: How can the software help
the learner recall prior knowledge and build on what she
knows?
Technology (tool) knowledge: How can the software meet
learners at varying levels of technological expertise in ways
that keep the learner focused on substance, not on the tool?
Strategic knowledge: How can the software help the
learner use strategies he possesses, and learn new strategies
appropriate to the task at hand?
Metacognitive knowledge: How can the software assist
the learner in keeping track of her goals, monitoring her
progress, etc.?
Motivation: How can the software help the learner stay
engaged and motivated?
Table 1: Five Dimensions of Learner Needs
degrees, and which teachers provide in the normal course
of teaching.
Finally, learners differ from other users in terms of their
motivation. While professionals in a field are committed
to their work and use software tools to accomplish a task,
students are often not similarly motivated. Technology
must help sustain engagement by supporting complex
activities so that students can focus on substantive
cognitive issues and problem solving.
These five dimensions are considered as they apply to the
particular requirments of learners as they engage in
information seeking in an electronic environment.
Information Seeking
Information Seeking in Inquiry
Our research has focused on providing tools, pedagogy,
and contexts for students to engage in inquiry, with
information seeking as one part of the process. Inquiry
can include many types of investigations from
experiments to observations, from using primary resources
to reading what others have written. Key to the concept of
inquiry is that students engage in sustained investigations
of questions of interest to them. [5, 6] The major
challenge of including digital information resources in
inquiry-based learning is to provide tools which allow
students to embed information seeking in a sustained
process. This means allowing for not only one-shot
queries when the user is looking for a simple answer, but
also complex exploration of information spaces when the
user is trying to understand a complex problem. [7]
Typically, the information seeking process includes
posing a question, exploring for information, refining the
question, finding information, organizing and evaluating
what is found, synthesizing information, and, finally,
using the information. [2, 7] These steps are not followed
linearly, but rather, feedback from each phase may used to
reiterate a previous step, or to jump to a new step. For
example, the cycle of exploring, refining the question, and
looking for information may be repeated many times until
Information
Seeking
Dimensions of Learner Needs
Domain knowledge
Engaging a in
process
Generating
search terms
Staying on task
Evaluating
information
Technology
knowledge
Persistent Workspace
Driving Question Folders
Broad Topics
Collections
Overall design supports
growth in technology
knowledge by building on
common design metaphors
Strategic
knowledge
Metacognitive
knowledge
Motivation
Driving Question Folders
Past Results
Past Results
Persistent Workspace
Recommendations
Broad Topic
Past Resultss
Broad Topics
Broad Topics
Persistent Workspace
Collections
Persistent Workspace
Past Results
Driving Question Folders
Collections
Driving Question Folders
Collections
Collections
Driving Question Folders
Collections
Past Results
Recommendations
Table 2: Artemis Features: Dimensions of Learner Needs and Information Seeking
a satisfactory, focused question is formulated. For a user,
each piece of the information seeking process might be
thought of as a separate task, and different software tools
might be used for different pieces of the process.
Historically, computer systems have provided tools for
information seeking in pieces, focusing on limited aspects
of the process. For example, search engines address the
phase of looking for information; word processors address
using information; Web indexes help with browsing;
bookmarking programs are for organizing. However, for a
learner, whose purpose is to learn from the information
encountered, not just to find a specific piece of
information, the task is the entire process, and it must be
supported as a whole. The support should allow the
student to focus on the content, not on the technology. For
students, especially at younger ages, simultaneous use of
multiple pieces of software is not an option. In addition to
cognitive issues, the hardware available in schools often
cannot effectively support the information seeking process
through use of multiple programs.
Empirical Basis for Understanding Information Seeking
As a first step toward understanding how to support
information seeking for students, we implemented Webbased units in 6th and 9th grade science classrooms. We
used three major sources of data:


Process video uses a video recorder to capture
everything that students see on their computer
monitors through a video-out connection, and
combines this with audio recording of conversations
between students stationed at the computer. Process
video allows fine-grained studies of students
collaboratively searching for information.
Classroom observations follow focus groups of
students over time in the course of their classroom
activity (which includes Artemis searching). Student
groups are monitored across inquiry-based science
projects, as well as longitudinally over the year.
Following students over time allows study of students'
developing competencies, and preferences-over-time,
and also allows us to assess the larger impact of
Artemis-provided information in inquiry-based
science.

Structured interviews of students and teachers were
used in conjunction with other observations.
Observers watch students in the course of a search
recorded on process video, or over the course of a
longer project, and question students or teachers on
aspects of their search, inquiry processes, and
attitudes toward the tools.
During the one to three week projects, students posed
questions and looked for information on the Web and in
their school libraries to complete their assignments, which
varied from writing a report to creating a multimedia
document. Detailed observations of focus students were
obtained by recording their on-line activities with process
video. We followed ten pairs of middle school students
doing four projects, and six pairs of high school students
doing three projects during the 1996-7 school year.
Analysis of the video records is ongoing, but initial results
of analysis of these records, along with observations of
similar projects during the 1995-6 school year,
highlighted four major aspects of information seeking in
which students need support. [8-11] These are described
below, and are related back to the five dimensions of
learner needs. Table 2 gives an overview of these aspects
the information seeking as they related to the dimensions
of learner needs, indicating the Artemis feature which
addresses each dimension.
Engaging in a process. Students often interpret the task of
information seeking as one of getting one "right" answer
or a perfect source. Rather than trying to solve a problem
and increase their understanding of the content area, they
work to finish, reducing their goal to "schoolwork." This
is a problem related to all areas of student motivation and
knowledge.
They need some base level content
knowledge to be able to ask an interesting question and to
deal with content they encounter. They need strategic
knowledge of a variety of types. For example, they need
strategies for reading on-line material, for planning their
progress toward completion of their project, and for
keeping track of what they find.
Generating search terms. Lacking background
knowledge, students often have trouble generating any
keywords other than those used in their question. More
generally, keyword selection is problematic because of the
nature of language and of search systems: e.g., exact string
matching ignores meaning and choice of keywords exactly
determines what is returned to the user. [12, 13, 14] An
inability to generate synonyms, either because of a lack of
background knowledge or because of a misunderstanding
of the significance of the particular keyword chosen, can
completely thwart a student’s progress. This problem
relates to student's content, technological, and strategic
knowledge.
Staying on task. Students can be easily distracted in an
environment such as a computer-rich classroom or lab in
which accountability is low because of the variety of tasks
students are engaged in and the high ratio of students to
teacher. Lack of success can decrease student interest in
staying on task, and lack of understanding of content can
make it difficult for students to focus. This is a problem of
domain knowledge and motivation.
Evaluating information. Accustomed to the textbook
model, in which students merely look up the answer and
take it as given, students may have neither skills,
knowledge, nor inclination to critically evaluate
information they find. This is a problem of content,
strategic, and metacognitive knowledge.
LEARNER-CENTERED DESIGN: ADDRESSING KEY
ASPECTS
The incorporation of features which address the needs of
learners within the problem space of the learning at hand
is central to LCD. These features can be of two types:
learner supports and scaffolds. We use the term “learner
supports” to refer to features that are inherent in the
software, but which are specifically included to support
learning. Examples of learner supports might be a spell
checker which helps students identify and correct
misspellings, or a graphical overview of an information
space which helps the learner understand the knowledge
domain in question. Learner supports may be useful to
professionals, but are in some sense essential for learners.
They are passive supports, in that they do not interactively
intervene in the student’s behavior, but rather are
available as an integral part of the software.
“Scaffolding,” on the other hand, refers to features which
support learning, but which fade as the learner grows in
knowledge and skills. [15] An example of scaffolding
might be an interactive help screen which prompts the
student through the steps of a process with decreasing
frequency as the student becomes successful. Scaffolding
can also be passive, for example, functionality providing
simplified views of a problem. The distinguishing
characteristic of scaffolding is that it fades over time,
becoming unavailable or invisible to the user as the user
becomes more expert. The purpose of scaffolding is to
enable the learner to do complex things on her own once
the scaffolding has faded. A vivid example of scaffolding
is training wheels on a bicycle: they provide support for
the beginner, can be altered as the child learns, and finally
are removed altogether when no longer needed.
In the design of Artemis, we include learner supports, but,
in the initial releases, no scaffolds. Our intention is to
continue observation and analysis of Artemis in use, and
consider addition of scaffolds as we understand more
about the information seeking needs and contexts of
learning of students in inquiry-based environments.
Learner-centered design methodology employed in the
design of Artemis includes analyzing the learning context
and learner needs in light of research on learning and
teaching. In this instance, the context is information
seeking with digital resources in science classrooms. The
special problems of this context are described above. The
five dimensions of learner needs, along with the identified
aspects of information seeking, provide the basis for
understanding the task from the perspective of the learner.
Our methodology requires that we work closely with
teachers in classrooms to iteratively modify our designs
based on actual uses by students.
ARTEMIS DESIGN
Our observations of students using Web browsers for
inquiry confirm that information seeking is a complex
process which is often not attended to in K-12 education.
[7,10,13] We designed classroom contexts and Web
environments to support students’ reflective use of Web
resources, and yet students short-circuited our efforts as
they sought to make sense of their assignments and of the
inherent difficulties of finding useful information in a
complex environment. The design of Artemis is in large
part a response to the difficulties we observed in the four
general categories above, and in many small, specific
instances. Overall, we found that it is imperative for
students to be able to accomplish multiple tasks within a
single computer environment so that their work does not
become fragmented, and so they can pick up where they
left off when they return to the computer day after day.
We sought to design learner supports which made it
possible for students to use Artemis as part of a reflective
approach to using digital information, concluding that
providing scaffolds early in the design cycle might be too
limiting. Our stance as educators led us to believe that the
role of the teacher along with the classroom and school
contexts determines how even the best software is used,
and we opted not to try to predict what scaffolds would
become most important until we had extensive
Figure 1: The Artemis Interface
observation of the software in use in the classrooms of
expert teachers. The most important learning supports of
the Artemis interface in its initial design, and their relation
to the needs of learners and the particular problems of
information seeking, are described below, shown in Figure
1, and summarized in Table 2.
Persistent Workspace
The Persistent Workspace provided through Artemis
retains student work from session to session.
In
particular, a record of searches with their results (in the
form of live links to actual documents) and the Driving
Question Folders (described below) with live links to
documents are stored on a UMDL server and are available
through a user name from any computer with access to the
Web. The Persistent Workspace is important because it
can help students engage in information seeking as a
sustained process over time, an important aspect of
inquiry and one which is quickly forgotten if students
focus only on query, submitting keywords and “getting
good hits.” Even for reflective students, fifty minute class
periods make it difficult to make progress in finding
useful information, especially if they need to start over
each day. Seeing what they have done before can not only
help students avoid redundant searches, but can also
reinforce the notion that they are engaged in a process of
exploring, finding, and evaluating information, not racing
to finish.
information, which students have trouble bridging. In fact,
we have observed that students act as if anything they find
about the subject they are searching for is acceptable: they
do very little evaluation of content. The idea of a
persistent workspace is to give them a way to retain
information about what they have already done, and to
work with information as they need it. The persistence not
only provides information over time, but also reinforces
the idea that information seeking is more than just getting
the right keywords
Driving Question Folders
Driving Question Folders (Figure 2) are created by
The Persistent Workspace supports growth of students
strategic and metacognitive knowledge, as well as helping
them build content knowledge.
In classrooms, we observed a disconnect between
querying and actually looking at and evaluating
Figure 2: Driving Question Folders
what they know, or what they are trying to learn. Strategic
knowledge is supported by providing them with ready
ways to organize what they found, as well as to refine
their question. These are strategic skills which have
proven difficult for students to use on the Web.
Metacognitive knowledge is supported through the
persistence of the Driving Question Folders, allowing
later reflection on what they found, where they are in
accomplishing their goals, and what they need to do next.
Past Results
Figure 3: Past Results in Artemis
students to store links to items they find interesting. They
function like a bookmark file, but are a visible part of the
interface. Items are placed in a folder by dragging and
dropping from the Search window. Students can make
multiple folders which can reflect different areas of
interest, or refinements of an initial question. Because
they are part of the Persistent Workspace, the folders are
accessible day after day, from any computer the student is
working on.
An important concept of inquiry-based learning is that
students work on questions of interest to them, sometimes
described as "authentic" or "driving" questions. We use
the term driving question, defined by one middle schooler
as "A question that drives you crazy until you get the
answer." In posing such a question, the student may start
with a general area of interest, and narrow it down as she
finds interesting information. Or, alternatively, she may
find that her interests change as she encounters new
information, and change her question altogether. These
folders allow flexibility in storing links, and may help
students focus on the substance of what they find. They
can create multiple folders, reflecting a refinement of their
original question, or a new area of interest. The folders are
part of the persistent workspace, and thus can be used
across multiple work sessions, adding items or looking in
more detail at items already found. This provides support
for engaging in a process and may help students stay on
task by making organizing easier. It also allows them to
readily compare and evaluate resources about a specific
driving question. In a later release, a note pad will be
available in the driving question folder.
Driving Question Folders support learning content
knowledge by helping students focus on the question.
Needing to put things in folders helps them think about
what information fits with their question and what other
related questions they might usefully pursue. We have
observed that without a strong focus, students often
ramble around in content without paying much attention to
The Past Results window (Figure 3) keeps a "live" list of
the student's searches. Clicking on an item in the Past
Results window shows the results in the Search window.
The results are actually stored on the UMDL server as
part of the student's persistent workspace, so what they see
in the Search window is identical to their original search.
In addition, the broad topic and keywords used when the
search was submitted are shown, so the student has a
record of how he has searched, as well as what he has
found.
In observing student search behavior on the Web, we
found that students sometimes submit the same query
repeatedly, and are often unaware that they have done so.
One reason for their repetition may be that they come to
see information seeking as a task of creating a perfect
query, one which gets them exactly what they want. This
is reinforced by the unmanageable number of sites
returned through Web search engines, leading to the
feeling that there is no point in looking at sites until you
are dealing with a manageable number. However, Web
search engines typically offer little support for actually
refining a query.
We also observed that students developed a strategy of
recording keywords as if they provided an index to Web
sites. That is, if they found a good site, instead of writing
down the URL or creating a bookmark, they recorded the
keywords they used to get to the site. This is an
interesting strategy which works in the short term and says
much about their mental model of the Web. The Past
Results feature in a sense formalizes their strategy by
actually making their past searches into an index of what
they found.
Past Results can help students avoid useless repetition,
contributing to their staying on task. It can also be used in
an appropriate pedagogical context to help them use
different keywords, by making what they have already
done easy to review. Finally, this feature can contribute to
students' understanding of information seeking as a
process, by allowing them to see their own process over
time. Past Results thus supports development of strategic
and metacognitive knowledge, addressing all four aspects
of information seeking on which we have focused.
Broad Topics
The Broad Topics window (Figure 1), which alternates
with Past Results in one corner of the Artemis window,
consists of a list of topics organized by domain. The
topics serve two types of functions, one internal to the
UMDL system, and the other directed toward the learner.
Within UMDL, broad topics are actually a controlled
vocabulary used to register materials in some collections
within the library. As explained below, the broad topics
shown were selected to register the particular collection
designed by UMDL researchers for K-12 classrooms, and
they are taken from the SEARS headings, a well known
subject heading system for school libraries. Agents within
the UMDL architecture translate these broad topics to
topics in other vocabularies (that is, other sets of terms
used to register collections), and can automatically
broaden or narrow searches if no results are found.
From the perspective of the user, broad topics present a
hierarchy of terms which can be browsed or searched as
the first step in creating a query. UMDL uses the broad
topics selected by the user as keywords when a search is
submitted. If specific topics are also entered, UMDL
searches on those keywords as well. The topics are
searchable - entering a word results in highlighting all
instances of that word in the Broad Topics window.
From the perspective of learning, the Broad Topics
window is intended to help students generate keywords
and recall prior knowledge, and it gives them a view of
the structure of the content area they are exploring.
Students who ask questions that interest them often end up
exploring content areas about which they know very little.
The result is that they can have a lot of trouble thinking of
any terms outside of the language of the question they
have posed. That is, if the question is, “how hot is lava?”
they may try to use the keywords “hot” and “lava” for
their search. For them, Broad Topics works something
like an on-line thesaurus, letting them explore related
terms, jogging their memory and perhaps their interest in
related topics.
We see Broad Topics as a support for learning content
knowledge as described above, and also for development
of strategic and metacognitive knowledge, and helping
with student motivation. It presents a strategy for
generating search terms and gives students a concrete way
to think about what they know and what they need to
know. As support for motivation, it can provide them
with alternatives - different terms with which to search possibly avoiding some of the frustration of getting stuck
using search terms that aren’t working without being able
to think of alternatives.
students; a collection of reference materials including the
Encyclopedia of Science and Technology, and Grolier's
Encyclopedia Americana; and the “Recommendation
Collection” (which is described in the next section.) The
Web itself is also a collection of UMDL by virtue of
having a UMDL agent which searches the Web when
queries are submitted. Once a query is submitted, returns
are grouped by collection, showing the number of returns
within each collection. This provides an initial grouping
of returns, helping make the return set manageable.
The Web Collection consists of Web pages identified by
librarians as particularly suited for students in the pilot
deployment of Artemis, 6th and 9th grade earth and space
science students. Librarians located, evaluated, and
abstracted readable material with appropriate content.
They also looked specifically for information that was
designed to take advantage of the web environment, not
simply marked up textbook pages. They sought pages
with interactivity, animated illustrations, forms to
communicate and respond with web authors. These sites
are registered in a special UMDL collection using
controlled vocabulary from the Broad Topics list. The
Web Collection is designed to insure that students have
successful searches as a way to help motivate them to stay
on task.
Observations of students using general Web search
engines indicate that students get frustrated by the
numerous, often irrelevant, sites returned from even fairly
specific searches.
Irrelevant results also lead to
distraction from their task, as students follow useless
links. The Web Collection approach is similar to the tried
and true approach of libraries, in which collections
appropriate to the user population are made available.
Each site in the Web Collection includes an abstract
written by the librarians. The abstract appears when the
student picks the link from the search results list, allowing
them to get a quick overview of the site without the time
cost of actually linking to the page. This feature is another
way to help keep students on-task. In addition, it provides
an aid to their evaluation of the site.
Collections
UMDL consists of multiple collections, related by a focus
on earth and space science. In the testbed are a journal
collection from UMI (ProQuest Direct); the “Web
Collection,” a collection of Web sites created for K-12
Figure 4: Making a Recommendation
The various feature of UMDL collections support students
in engaging in a process, staying on task, and evaluating
information.
Recommendations
The Recommendations feature lets students and teachers
recommend good resources. Figures 4 and 5 show how
recommendations are set up, while the results of
recommendations are shown in the search window of
Figure 1.
In our classroom experiences with students using the Web,
one approach which was beneficial was to create our own
Web pages with good starting points selected by UMDL
librarians in collaboration with teachers. This experience
resulted in the Web collections described above, and the
Recommender. The Recommender automates the process
of making targeted collections, and incorporates them into
the UMDL as a registered collection.
A teacher or student can recommend resources by using
the window shown in Figure 4. The recommended sites
are compiled into a recommendation collection which is a
searchable collection within UMDL. Using the Profile
window (Figure 5) to set criteria, a user making a query
can ask for recommendations based on characteristics of
the person making the recommendation or the topic. For
example, a class list of recommended resources can be
created by the teacher. She can have her class set their
preferences to get her recommendations. When they enter
queries, the system searches her recommendations for
matches to their query, and these appear as search results
in the Recommendation collection.
Recommendations is another feature which can be used to
help students evaluate information they find. On the one
hand, they can read evaluations from others about a site
they are interested in. On the other hand, they may be
asked to make recommendations to others, a process
which involves explicit evaluation. This supports both
strategic and metacognitive knowledge. We also expect
that being able to contribute recommendations, and see
what peers and teachers think about sites will motivate
students as they work.
Figure 5: User Profile for Recommendations
EVALUATION
Evaluation of the first release of Artemis as described in
this paper is being conducted during the 1997-98 school
year in middle and high schools. Log file data and
process video records of student use are being collected
and analyzed for system functionality and interface
usability. For example, we are looking at patterns of use
by high school students to determine proportion of time
spent submitting and sorting through searches compared
to time spent reading and evaluating material found; and
we are looking for use of specific supports such as the
Driving Question folders and Past Results. Feedback
from field data is being used to rework the interface for a
new release in the spring of 98.
DISCUSSION
There are very high expectations for the value of the
digital resources, and particularly the Internet, in
education. Unfortunately, currently the Internet is used,
for the most part, as a mechanism to deliver electronic
versions of textbooks: “go to the following Web site and
answer the following 5 questions.” In order to truly
realize the potential of the digital information resources
for learning and teaching, learners will need software that
supports them in their use of those resources, and enables
students to go beyond simply answering teacher-given
questions, supporting them as they engage in substantive
inquiry.
Artemis is one attempt at using learner-centered design
ideas to produce a learning environment that does support
students in such inquiry activities. As the foregoing has
documented, the design is well rationalized; we have
attempted to take into consideration cognitive, social, and
motivational issues in the design of Artemis’ functionality
and supports. Our next round of data on the use of
Artemis will enable us to revise, yet again, the interface
and better hone the learner-centered design guidelines and
processes.
ACKNOWLEDGMENTS
The authors wish to thank the students and teachers who
participated in the UMDL/MYDL research projects from
1995-1998.
This work was supported by the
NSF/DARPA/NASA
Digital
library
Initiative
(Cooperative Agreement IRI-9411287); by a grant from
NSF NIE for the Middle Years Digital Library (RED9554205) and by the University of Michigan.
REFERENCES
1.
Atkins, D.E., Birmingham, W. P., Durfee, E. H.,
et al., Toward Inquiry-Based Education Through
Interacting Software Agents. Computer, 1996. 29(5): p.
69-76.
2.
Wallace, R., J. Krajcik, and E. Soloway, Digital
Libraries in the Science Classroom: An Opportunity for
Inquiry. D-Lib Magazine, 1996
(http://www.dlib.org/dlib/september96/umdl/09wallace.html).
3.
Soloway, E., M. Guzdial, and K.E. Hay, Learnercentered design: The challenge for HCI in the 21st
century. Interactions, 1994. 1(2): p. 36-48.
4.
Paris, S.G. and P. Winograd, How metacognition
can promote academic learning and instruction, in
Dimensions of thinking and cognitive instruction, B.F.
Jones and L. Idol, Editors. 1990, Lawrence Erlbaum
Associates: Hillsdale, N. J. p. 15-51.
5.
National Research Council, National Science
Education Standards. 1996, Washington, DC: National
Academy Press. 262.
6.
Blumenfeld, P.C., et al., Motivating Project-based
learning: Sustaining the doing, Supporting the Learning.
Educational Psychologist, 1991. 26(3 & 4): p. 369-398.
7.
Kuhlthau, C.C., The process of learning from
information, in The virtual school library: Gateway to the
information superhighway, C.C. Kuhlthau, Editor. 1996,
Libraries Unlimited, Inc.: Englewood, CO. p. 95-104.
8.
Wallace, R., On-line information seeking in a
sixth grade classroom. In revision, 1997.
9.
Hoffman, J.L., Kupperman, J., Wallace, R. OnLine learning materials for the science classroom: Design
methodology and implementation. Paper presented at
AERA 97, Chicago.
10.
Wallace, R. and J. Kupperman. On-line search in
the science classroom: Benefits and possibilities. Paper
presented at AERA 97, Chicago. (ERIC # ED407252)
11.
Soloway, E. and R. Wallace, Does the internet
support student inquiry? Don't ask. Communications of
the ACM, 1997. 40(5): p. 11-16.
12.
Greene, S.L., Devlin, S.F., Cannata, P. E.,
Gomez, L. M., No IFs, ANDs, or ORs: A study of database
querying. International Journal of Man-Machine Stuidies,
1990. 32: p. 303-326.
13.
Furnas, G. W., Landauer, T. K., Gomez, L. M.,
Dumais, S. T. The vocabulary problem in human -system
communications, Communications of the ACM, 1987. 30:
p. 964--971
14.
Marchionini, G., Information seeking in electronic
environments. Cambridge Series on Human-computer
Interaction, ed. J. Long. 1995, Cambridge: Cambridge
University Press. 224.
15.
Wood, D., J.S. Bruner, and G. Ross, The role of
tutoring in problem-solving. Journal of Child Psychology
and Psychiatry, 1975. 17: p. 89-100.