A Survey of Current Research Questions in
Educational Technology Research and Development
Jake Burghar and Jennifer Turns
December 1999 (Draft)
One way of generating research questions that could be addressed in PETTTs research on The
Arthritis Source is to look at the types of research being explored in prominent educational
technology journals. This report will discuss an analysis of the current research published in one such
journal, Education Technology Research and Development. This analysis is intended to answer two
questions: (1) What kinds of questions are being asked in current research in this field, (2) How could
these categories of questions could apply to PETTTs research on The Arthritis Source.
Methods
A brief survey of article titles in Educational Technology Research and Development was conducted,
in which the table of Contents from 1997 to the present was analyzed to find general trends in the
kinds of research questions being asked in this field.
After reading through the selected Table of Contents, five categories of research questions were
generated. These question categories were based on first impressions of the nature of the research in
the publication, and it is important to stress that they are just suggestions. The articles being
surveyed were not actually read, though some of their abstracts were reviewed to get a better idea as
to their purpose.
Once question categories had been defined, each of the article titles from the dates being studied was
placed each into one of the categories. When this process was completed, the question categories—
and the placement of the titles in those categories—were revised to better fit my understanding of the
research.
The definitions of the categories and the placement of the individual articles within each category was
then reviewed by Jennifer Turns. The descriptions of the categories themselves were clarified, and
the placement of some of the articles were arbitrated. The resulting categories and their descriptions
are listed in Question One.
From these question categories and their associated articles, potential research questions and article
titles for PETTTs research on The Arthritis Source were generated. These questions are listed in
Question Two.
Question One: What Kind of Questions Are Being Asked
In grouping the article titles in Education Technology Research and Development, five question
categories were created. These categories and their proposed relationships to one another are
illustrated in the Figure 1, below. In this diagram, the solid lines represent feed-forward processes in
the development and implementation of educational technology and associated theory, while the
curved, dashed lines represent feedback processes in the same development cycle.
Figure 1: Proposed relationships between the question categories
In this representation, there is a linear flow from theoretical understanding to the application of that
understanding. After a theory has been applied, it can be assessed either by its self or in relation to
the environment in which it has been implemented. The applied theory and assessment stages feed
into improvements in the design process, which in turn feeds into theoretical understanding.
In the following pages, each of the five categories is described and the articles that were selected as
fitting each category are listed.
How can we theoretically understand educational technology?
The titles of these articles suggested that their content discussed abstract theories in educational
technology or created theory from multiple examples. Many of these explanations appeared to be
“conceptual frameworks” to be used by educational technology researchers and developers. These
frameworks seem to be either specific to educational technology or theories taken from other areas of
study and applied to the edtech. Other titles suggested that the article had more to do with the
advancement of research techniques and the development of the educational technology field.
A good example of an article that belongs in this category is “Activity Theory as a Framework for
Designing Constructivist Learning Environments.” This article appears to be taking a theory from
psychology and applying it to a specific type of educational environment.
The articles selected for this category are:







Draft
Enhancing relevance: Embedded ARCS Strategies vs. Purpose. 1997. 45(1)
The Role of Context in Learning and Instructional Design. 1997. 45(2)
User Design in the Creation of Human Learning Systems. 1997. 45(3)
The Field of Educational Technology as Reflected Through its Definitions. 1997. 45(3)
Grounded Practice and the Design of Constructivist Learning Environments. 1997. 45(3)
Cognitive Efficiency: Toward a Revised Theory of Media. 1997. 45(4)
The Real World on a Short Leash: The (Mis)Application of Constructivism to the Design of
Educational Technology. 1998. 46(3)
Page 2 of 7












The Pursuit of Useable Knowledge in Instructional Technology. 1998. 46(4)
Potential Knowledge Management Contributions to Human Performance Technology Research and
Practice. 1998. 46(4)
Activity Theory as a Framework for Designing Constructivist Learning Environments. 1999. 47(1)
Conceptual Frameworks for Understanding Information Seeking in Open Ended Information Systems.
1999. 47(1)
New Research Paradigms in Instructional Technology: An Inquiry. 1999. 47(2)
Toward a Learning Technologies Knowledge Network. 1999. 47(2)
Reassessing Training Programs: A Model for Identifying Training Excesses and Deficiencies. 1999.
47(2)
Interactive Technology and Authoring Tools: A Historical Review and Analysis.
Patterns of Understanding with Open-ended Learning Environments: A Qualitative Study. 1997. 45(2)
Instructional Design Models for Well-Structured and Ill-Structured Problem Solving Learning
Outcomes. 1997. 45(1)
Multimedia: Differences in Cognitive Processes Observed with EEG. 1999. 47(3)
Distance Education Finds a New Use for ‘No Significant difference.’
How can theory be applied to educational technology?
The titles of these articles suggested that their content discussed how theoretical understanding of
educational technology could be used to interpret or design real world applications. This category
also included titles that appeared to discuss some advancement in technology or possible applications
for an existing technology. The articles in this category that appear to be more theory based were
placed under this heading because their focus appeared to be more grounded in applied, real world
practices.
A good example of an article that belongs in this category is “Embedding Cooperative Learning into
the Design of Integrated Learning Systems: Rationale and Guidelines.” This article appears to be
taking theoretical ideas about cooperative learning and applying it to integrated learning systems.
The articles selected for this category are:












Draft
The Effects on Student Achievement and Attitudes When Using Integrated Learning Systems with
Cooperative pairs. 1997. 45(1)
The Relative Effectiveness of Learning Options in Multimedia Computer-Based Fifth-Grade Spelling
Instruction. 1997. 45(2)
Using Three Dimensional Interactive Graphics to Teach Equipment Procedures. 1997. 45(4)
Cognitive Strategies and Learning from the World Wide Web. 1997. 45(4)
A Computer-Mediated Support System for Project-Based Learning. 1998. 46(1)
Technologies for Lifelong Kindergarten. 1998. 46(4)
Embedding Cooperative Learning into the Design of Integrated Learning Systems: Rationale and
Guidelines. 1998. 46(3)
Knowledge Constructing Through Hypermedia Authoring
Visual Displays and Contextual Presentations in Computer-Based Instruction. 1998. 46(3).
Designing Technology to Support Reflection. 1999. 47(3)
Applying Constructivism: A Test for the Learner-as-Scientist. 1999. 47(3)
Designing Communities of Learners for Asynchronous Distance Education. 1998. 46(4)
Page 3 of 7
What are the Effects of a Given Technology or Practice?
The titles of these articles suggested that their contents assess the functionality of an educational tool
or practice. This category does not cover the success of tools in a particular setting, instead it
evaluates the tool or practice itself.
A good example of an article that belongs in this category is “Children’s Vocabulary Acquisition in a
Foreign Language Through Watching Subtitled Television Programs at Home.” This article discusses
a particular effect, level of language acquisition, associated with a specific technology, subtitled
television.
The articles selected for this category are:








The Effects of Instructional Content and Reasoning Complexity on Mathematics Problem Solving.
1997. 45(3)
The Effects of Distance Learning Classroom Design on Student Perceptions. 1997. 45(4)
Effects of Cooperative Versus Individual Learning and Orienting Activities During Computer-based
Instruction. 1998. 46(1)
Learning and Remembering from Thematic Maps of Familiar Regions. 1998. 46(1)
Children’s Recall of the News: TV News Stories Compared with Three Print Versions. 1998. 46(1)
Learner Preferences and Achievement Under Differing Amounts of Learner Practice. 1998. 46(2)
The Effects of Cooperative Learning and Learner Control on Students’ Achievement, Option
Selections, and Attitudes. 1998. 46(2)
Children’s Vocabulary Acquisition in a Foreign Language Through Watching Subtitled Television
Programs at Home. 1999. 47(1)
How can the development process be improved?
The titles of these articles suggested that their content focused on research questions whose outcomes
inform the development process and techniques to be used in the development of educational
technology and instructional materials in general. Most of the titles appeared to describe the
development of a particular case, though some appeared to have a more general scope.
A good example of an article that belongs in this category is “Instructional Design Project Success
Factors: An Empirical Basis.”
The articles selected for this category are:












Draft
ETR&D Development: An Analysis of Content and Survey of Future Direction. 1997. 45(3)
Revisioning models of Instructional Development. 1997. 45(3)
Research on Instructional Development. 1997. 45(3).
User Centered Design and Usability Testing of a Web Site: An Illustrative Case Study. 1997. 45(4)
Instructional Design Project Success Factors: An Empirical Basis. 1997. 45(2)
An Instructional Design Case Event: Exploring Issues in Professional Practice. 1998. 46(1)
Meeting with the SME to Design Multimedia Exploration Systems. 1998. 46(2)
What is an Expert Instructional Designer? Evidence of Expert Performance During Formative
Evaluation. 1998. 46(3)
Blending Authentic Work Projects and Instructional Assignments: An Adaptation Process
The Cognitive Approach to Training Development: A Practitioner’s Assessment.
Instructional Practices of Teachers Enrolled in Educational Technology and General Education
Programs. 1998. 46(3)
Kirkpatrick Plus: Evaluation and Continous Improvement with a Community Focus. 1998. 46(4)
Page 4 of 7


An Expert Teacher’s Thinking and Teaching and Instructional design Models and Principles: An
Ethnographic Study. 1998. 46(2)
Do Superior Teachers Employ Systematic Instructional Planning Procedures? A Descriptive Study.
1998. 46(2)
What factors affect the successful implementation of a technology?
The titles of these articles suggested that their content discussed factors and practices in the
implementation of educational technology. Their focus appeared to be on the interaction of a
technology and its users’ environment, discussing both functionality and factors affecting the
technology’s success.
An obvious example of this category is "Factors Associated with the Amount of Use and Benefits
Obtained by Users of a Statewide Educational Telecomputing Network.” The Telecomputing
network is the technology being implemented, and the Statewide Educational Telecomputing
Network is the environment providing the “factors” used in the analysis. A less obvious example is
“Implementing Individual and Small Group Learning Structures with a Computer Simulation,” in
which the technology is a computer simulation is the technology and the factors being studied are
individual and small group learning.
The articles selected for this category are:










Factors Associated With Amount of Use and Benefits Obtained by Users of a Statewide Educational
Telecomputing Network. 1997. 45(1)
Technology and Educational Empowerment: Students’ Perspectives. 1997. 45(2)
Emergent Patterns of Teaching/Learning in Electronic Classrooms. 1998. 46(4)
User-Design: A Case Application in Health Care Training. 1998. 46(4)
Impacts of a University-Elementary School Partnership Designed to Support Technology Integration.
1999. 47(1)
Implementing Individual and Small Group Learning Structures with a Computer Simulation. 1999.
47(1)
Software for Managing Complex Learning: Examples from an Educational Psychology Course. 1999.
47(2)
Organizing Principles for Science Education Partnerships: Case Studies of Student Learning About
‘Rats in Space’ and ‘Deformed Frogs’. 1999. 47(2)
Applying the ARCS model to the Design and Development of Computer-Based Modules for
Manufacturing Engineering Courses. 1999. 47(2)
Exploring the Potential of a Computer Tool for Instructional Developers. 1999. 47(3)
Question Two: How Can These Questions Be Applied to PETTTs
Research on The Arthritis Source
The second purpose of the report is to stimulate a conversation about how the categories of questions
defined above could be applied to PETTTs Research on The Arthritis Source. Although it is possible
to think of many questions that fit into these question categories, we are only interested in questions
that further PETTTs research agenda. Some criteria for choosing research questions for PETTT are:

Draft
Will the research be consistent with PETTTs goals?
Page 5 of 7


Will the research question be a valid one in the context of The Arthritis Source?
Will the research be consistent with our personal interests?
Some possible questions for each of the question categories are listed below.
How can we theoretically understand educational technology?

What are the pedagogical issues in creating technology to instruct a wide variety of
learners?

What type of metaphors should be used to explain interactive educational resources? (If
not an “interactive encyclopedia,” then what?) This sort of metaphor would be helpful to
both the user and the developer, and it could help promote a sustained interaction with a
resource.

How doe users perceptions of the author of a website affect short and long term
relationships with the site. In particular, we could study what users think of the authors’
of The Arthritis Source? What do users perceive their own relationship with the site to be
because of this “author image”?
How can theory be applied to educational technology?

In conjunction with the authorship question above, how do web design decisions affect
the web-authors’ “presence” (e.g., Rick Matsen, UW Medical Centers, etc.) What is the
effect of externalizing the relationship: letting users know how other users view the site
(Judy Ramey).

What is the effect of externalizing users’ use of The Arthritis Source? If log file data was
graphically summarized so that users could see the “top pages,” would that affect overall
navigation? Would it affect the “author image” of the site? (Judy Ramey)

How can the medical model of education inform design decisions in web-based medical
information? Some of the pages have a more expert feel to them—how are those pages
interacted with in comparison to lay person information? What does those results say
about the underlying design decisions/audience analysis?

How do you organize, present and write content for a diverse audience? We could do a
case study on web-based medical information for use by different audiences such as
Medical professionals and lay people.

What web-based media types are best suited for different learning goals?
Does a given technology or practice work?
Draft
Page 6 of 7

How effective is the current use of video incorporated into learning materials; in
particular, exercises in The Arthritis Source. One product of such a question could be
guidelines on how to best use video in instructional materials.

How effective are online surveys? A meta-analysis of research supplemented by our own
findings on their effectiveness and representativeness.

How do end users apply heuristics, principles and templates to their own educational
materials: A usability guide example. (design a usability package for instructors and ask
them to use it to develop a usability test for their own site).
How can the development process be improved?

How can developers incorporate subject matter experts, webmasters and researchers into
the updating of an informational web page?

How do we educate instructors on how to develop their own website: designing
principles, heuristics and templates.
What factors affect the implementation of a technology?
Draft

Methods of “advertising” for an online medical resource: Doctor referral, search engine
placement, friends and family, Hub organization link pages (Arthritis Foundation), kiosk

How do patients interact differently with the kiosk-based presentation than they do with a
more conventional “web” presentation? (we could do a log file analysis of both?)
Page 7 of 7