E19.2174 Cognitive Science and Educational Technology I
Visual Learning
Jan L. Plass
New York Univerisity
Center for Research and Evaluation of Advanced Technologies in Education
Overview
Visual Learning
You are asked to design a visualization for educational
purposes, for example, to learn about
•
•
•
•
•
•
The ideal gas laws
The carbon cycle
Bird migration patterns
The system of voting districts in the U.S.
Air traffic control
The history of the Gulf War
Overview
Visual Learning
Cognitive Design
Visual Environment
Emotional Design
Cognitive
Variables
Visual Load
Attitudes,
Motivation
Visual Learning
Visual Learning
Visual Learning
• Learning from primarily visual materials
• Text only as labels or brief statements
• Examples:
– Graphs, charts, maps, networks,
pictures, video, animation
Visual Learning
How do Visual and Verbal Information differ from one another?
Visual Learning
Visual v. Verbal Information
•
Visual information:
•
•
•
•
analogous representations
inherently relational
encoded simultaneously
Verbal information:
•
•
•
discreet units of symbolic information
propositional
processed sequentially
Visual Learning
VERBAL STIMULI
Dual Coding
Theory
NONVERBAL STIMULI
SENSORY SYSTEMS
(Paivio, 1986, 1990)
REPRESENTATIONAL CONNECTIONS
Logogens
Associative
Connections
Imagens
REFERENTIAL
CONNECTION
S
REFERENTIAL
CONNECTION
S
VERBAL SY STEM
VERBAL RESPONSES
Associative
Connections
NONVERBAL SYSTEM
NONVERBAL RESPONSES
Visual Cognitive Load
Cognitive Load Components (Sweller, 1999)
•
Intrinsic Load
Load related to complexity of the information
Element interactivity
•
Extraneous Load
Load pertaining to format and design of the interface
(presentation mode, modality, temporal & spatial arrangement, representation type)
•
Germane Load
Mental effort expended by learner
Intrinsic
Load
Working Memory
Extraneous
Load
Germane
Load
Free
Visual Cognitive Load
Visual Cognitive Load
•
Cognitive Load for Visual Representations:
• Intrinsic Visual Load
Visual element interactivity
• Extraneous Visual Load
Visual format and design of
the interface (presentation mode,
modality, temporal & spatial arrangement,
representation type)
Lee, Plass, & Homer (2006)
Visual Learning Environments
Visual Learning Environments
• Highly visual learning environments
• Examples
– Simulations, virtual worlds, microworlds, games
Introduction
Examples
Ideal Gas Law
(Oklahoma State
University)
Introduction
Examples
Odyssey
Simulation
Package
Introduction
Examples
Gizmo/
ExploreLearning
Introduction
Examples
Molecular
Workbench
Introduction
Examples
Schnotz & Rasch (2005)
Introduction
Examples
Ideal Gas Laws
(NYU
Molecules & Minds
project, IES)
Introduction
Examples
Virtual Patient
(Abdominal
Exam)
NYU School
of Medicine
Overview
Visual Learning
Cognitive Design
Visual Environment
Emotional Design
Cognitive
Variables
Visual Load
Attitudes,
Motivation
Visual Learning
Visual Learning
Group Discussion (3-4 students, 15min)
Discuss Design Principles that increase the effectiveness of visual
representations for learning (Animations and Simulations)?
–List and Discuss principles from the assigned reading
–Find and discuss examples
Cognitive Design Factors
Cognitive Design Factors
•
•
•
•
•
•
•
Representation of information (Information Design)
Instructional Approach (Interaction Design)
Interactivity (Interaction Design)
Function of Visuals (in support of cognitive processes)
Scaffolds
Feedback
Narrative structure
Cognitive Design Factors
Representation of Information (Semiotics)
Which mode of relationship between signs and their referents best
facilitates learning?
•
Icon: Most basic representation, relies on
physical resemblance to convey meaning
•
Symbol: Abstract, arbitrary, relies on social
conventions for meaning (Peirce, 1956)
Question of Interest:
• Comparison of Iconic v. Symbolic representations
Research Materials
Chemistry Simulations
•
Ideal Gas Law
Research Materials
Chemistry Simulations
•
Ideal Gas Law
Results: Representation
Representation of Information (Semiotics)
Does adding icons facilitate learning in chemistry simulations?
•
•
•
Study with 93 11th grade students in a NYC high school:
Adding icons increased recall
Icons especially helped learners with low prior knowledge
(Lee, Plass, & Homer, 2006; Plass et al., 2007)
Visual Design of Simulations
Instructional Approach: Level of Learner Control
Which instructional approach best facilitates learning?
Consider:
• Difficulty of content: Intrinsic Cognitive Load
• Complexity of interactions: Extraneous Load
• Educational goals / Cognitive Function of materials
• Learner characteristics
Option
• Direct instruction v. guided exploration
Visual Design of Simulations
Instructional Approach: Level of Learner Control
Which instructional approach best facilitates learning?
•
Comparison of direct instruction v. guided exploration
In other words:
• Comparison of Worked-out example (Animation) v. Exploration
(Simulation)
Or, in even different terms:
• Kirschner, Sweller, & Clark (2006) v. Everybody Else
Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of
Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching
Research Materials
Chemistry Simulations
•
Kinetic Theory
of Heat
Research Materials
Worked-out
Example
•
Ideal Gas Law
Results: Instructional Format
Results: Simulation (exploratory) vs. Animation (worked-out)
Does ability to manipulate parameters facilitate learning?
•
Study with 93 11th grade students
in a NYC high school:
4.0
3.0
For comprehension:
Simulation
Direct
Exploration > Instruction
2.0
Mean M.C. Post test
•
1.0
0.0
Worked Example
Worked Out/Explore
(Plass et al., 2007)
Explore
Cognitive Design Factors
Example
Level of
Interactivity
Cognitive Design Factors
Example
Level of
Interactivity
Visual Design of Simulations
Cognitive Function of Visual Information
What purpose does the visual information serve in the
construction of mental models? (Plass, 1998)
• Levin, Anglin, & Carney (1987): 5 functions (heuristic)
• Decorative
motivational function; little relation to content
• Representative
depicts content of the instruction
• Organizing
depicts knowledge structures
• Interpreting
visualizes abstract concepts
• Transforming
supports higher-level cognitive processes
Visual Design of Simulations
Cognitive Theory of Multimedia Learning (Mayer, 2001)
Words
Ears
Sounds
Select
Words
Verbal Model
(Verbal Mental
Organize Representation)
Words
Integrate
Select
Images
Pictures
Eyes
Multimedia
Presentation
Sensory
Memory
Organize
Images
Images
Prior
Knowledge
Pictorial Model
(Visual Mental
Representation)
Working
Memory
Long-Term
Memory
Visual Design of Simulations
Cognitive Function of Visual Information
What purpose does the visual information serve in the
construction of mental models? (Plass, 1998)
•
Our approach: Define function based on Mayer’s CTML
• Selecting
• Organizing
• Integrating
•
Different types of visuals support different learning
outcomes (recall, comprehension, transfer)
(Plass, Hamilton, & Wallen, 2004; Wallen, Plass, & Brünken, 2005)
Visual Design of Simulations
Function of Multimedia Aids in Text Comprehension
Visual Design of Simulations
Established Cognitive Design Principles
•
Split Attention Principle
Avoid requiring learners to split their attention between, and mentally
integrate, several sources of physically or temporally disparate
information, where each source of information is essential for
understanding the material.’’ (Ayres & Sweller, 2005)
•
Modality Principle
Present animation with narration rather than with on-screen text
(Mayer, 2001)
•
Contiguity Principle
Present related information near to each other in time and space
(Mayer, 2001)
Visual Design of Simulations
Emerging Visual Design Principles
•
Cueing
Adding design elements that direct learners’ attention to the important
part of a simulation reduces cognitive load and enhances learning
(Dwyer, 1978, Jeung et al., 1997; Tabbers et al., 2004; de Koenig el al., 2007)
•
Representation of Information
Adding iconic representations can enhance learning, especially for
learners with low prior knowledge
(Lee et al., 2006; Plass et al., 2009)
•
Color Coding
Use color to highlight important features and attributes of the visual
display (Dwyer and Moore, 1991; Keller et al., 2006)
•
Multiple Dynamic Visual Representations
Multiple dynamic representations should be integrated and linked
(van der Meij & de Jong, 2006)
Visual Design of Simulations
Established Interaction Design Principles
•
Learner Control of Segmenting
Learner control over the advancement from one segment of visual
materials to the next improves learning
(Mayer & Chandler, 2001; Mayer et al., 2003; Moreno, 2007)
•
Guided Discovery Principle
Provide guidance in discovery-based learning environments
(de Jong, 2006; de Jong & van Joolingen, 1998; Kirschner et al., 2006; Mayer, 2004)
Visual Design of Simulations
Established Interaction Design Principles
•
Learner Control of Pacing
Learner control over the pace of the presentation of visual materials
improves learning
(Hasler et al., 2007; Schwan & Riempp, 2004; Tabbers et al., 2004)
•
Task-Appropriate Representations
–Simulations need to prepare learners for future tasks to be performed–
Facilitating, Enabling, or Inhibiting Effects
–Cognitive Function of Simulations (Retention, Understanding, Transfer)
(Carney & Levin, 2002; Levin et al., 1987; Plass, Wallen, & Hamilton, 2004)
•
Content-Manipulating Interactivity
Learner control over the content of visual materials improves learning
(Chandler, 2004; Hegarty, 2004; Rieber, 1990, Wouters et al., 2007)
Emotional Design Factors
Emotional Design Factors
•
•
•
•
•
•
Visual Design (Information Design)
Control (Interaction Design)
Feedback
Intrinsic motivation v. Extrinsic motivation
Social interaction
Social Presence/Telepresence
Overview
Visual Learning
Cognitive Design
Visual Environment
Emotional Design
Cognitive
Variables
Visual Load
Attitudes,
Motivation
Visual Learning
Visual Design of Simulations
Other Emotional Design Principles
•
Personalization Principle
Learning more deeply when words in a multimedia presentation are in
conversational rather than formal style
(Mayer, 2005)
•
Social Presence Hypothesis
Learning is facilitated by giving learners a sense of the presence of
others in a learning environment
This effect is expected to be especially strong in self-learning
Visual Learning
Group Activity (3-4 students, 30min)
Apply the Design Principles we discussed to your own projects by
designing a simulation or animation.
–Select which topic to cover
-Discuss which principles apply
–Describe how you will apply the principles for
the information design and interaction design
of the simulation