Session 6
The Unfinished Revolution - II
Networked Improvement
Communities in Education
Roy Pea
Bootstrap Colloquium
Stanford University, Feb 10, 2000
Copyright © 2000. SRI International. All rights reserved.
Center for Technology
in Learning
MISSION:
Improving learning and teaching
through innovation and inquiry in
computing and communications
http://www.sri.com/policy/ctl/
Copyright © 2000. SRI International. All rights reserved.
How does CTL do this?
• Advance theory
and research on
effective learning
and teaching.
• Innovative design,
use, assessment
of interactive
learning
environments.
• In short, learning
sciences and
engineering.
Copyright © 2000. SRI International. All rights reserved.
What’s the need?
• Education in the U.S. is decentralized
and fragmented
• Education improvement efforts are
isolated and incremental
• Technology can leverage the knowledge
“in the system” in revolutionary ways
Copyright © 2000. SRI International. All rights reserved.
A “powerful idea” for CTL...
Networked Improvement Community (NIC)
• …is a coalition of organizations each engaged
in a similar improvement process
• ... networks these organizations and crafts new
mechanisms for improving their isolated
improvement processes
• ...builds value and leverages knowledge for
distributed communities
• Thanks to Douglas Engelbart for fruitful
discussions about NICs
Copyright © 2000. SRI International. All rights reserved.
The Many NICs of CTL
PROJECT
TOPIC
SCOPE
TAPPED IN
On-line teacher professional
development institute
$1.7M (4 yrs),
spurs Minerva spinoff
CILT
Virtual center: K-14 learning
technologies R&D
$6M (4 yrs)
plus industry sponsors
PALS
Performance assessments
for learning science
$1.2M (3 yrs)
OERL
On-line library of education
project evaluation resources
$1.75M (7 yrs)
ESCOT
Interoperable mathematics
educational components
$2M (2 yrs)
Copyright © 2000. SRI International. All rights reserved.
TM
An Education Community-of-Practice Model for
Scalable, Sustainable Teacher Professional Development
http://www.tappedin.org
Leader: Mark Schlager
Contact: schlager@unix.sri.com
Sponsors:
Copyright © 2000. SRI International. All rights reserved.
TAPPED IN Overview
•
•
TM
What? Web-based Virtual Conference Center
Who? A growing community of over 7000 K-16
education professionals, 15 partner organizations and
agencies, and scores of small groups who create online learning experiences from any computer, anytime.
•
Multi-user virtual environment (MUVE)
•
Synchronous and asynchronous communication
•
Storing, sharing of Websites and documents (permanence)
•
Community-wide activities and support services
Copyright © 2000. SRI International. All rights reserved.
Gathering Place: TAPPED IN Campus
TM
TAPPED IN
Building
Pepperdine School
of Education
Student
Activities
Center
Private
Public
Copyright © 2000. SRI International. All rights reserved.
TM
1997
1998
1999
Copyright © 2000. SRI International. All rights reserved.
Jan-00
7000+
Some TAPPED IN users
TM
University Schools of Education




Pepperdine University (PT3)
California State Univ., Sacramento
University of Wisconsin (NSF)
George Mason University
Local and State Education Agencies




New Haven Unified School District, California (PT3, BTSA)
Joint Venture: Silicon Valley 21st Century Education Initiative (TICG)
Kentucky Department of Education
Los Angeles County Office of Education TFL, TEAMS, LAAMP (NSF)
Teacher/Librarian Professional Development Organizations and Websites




American Association of School Librarians
Institute for Research on Learning MMAP program (NSF)
Lawrence Hall of Science, University of California, Berkeley (NSF)
Math Forum Website at Swarthmore College (NSF)
Copyright © 2000. SRI International. All rights reserved.
What can educators do on-line?
Extend capacity to interact online through:
•
•
•
•
•
•
Real-time and e-mail Help Desk
After School Online discussion sessions
Free offices for individuals and small groups
Newsletter, events calendar, and MeetMe list
How-to Guides (e.g., lead a discussion; conduct training)
Consulting for TPD organizations
Educators engage in professionally relevant activities:
•
•
•
•
•
•
•
Planning and conducting projects with colleagues and students
Leading or joining topical discussions
Conducting and attending courses
Finding resources, experts, and new colleagues
Serving as resources for other educators
Trying out new ideas in a safe supportive environment
Using the Internet in new ways
Copyright © 2000. SRI International. All rights reserved.
TM
What are we learning?
•
•
Content is important, but learning occurs through
discourse around content
Membership diversity is essential for innovations to
germinate and spread
–
–
•
Preservice and new teachers learn about the profession and
veterans become valued resources
Members form groups that cross agency, program, project,
stakeholder group, and geographical boundaries
Both private and public places are needed to support
the community
Copyright © 2000. SRI International. All rights reserved.
TM
Challenges?
No single organization can satisfy teachers’ changing TPD needs
• Yet sustainable on-line communities are hard & costly to establish
• TPD organizations must understand the teachers’ challenge begins
where their efforts leave off
• Teachers need ownership of an on-line workplace that will be there
after TPD courses, projects, workshops, or grants end
Teachers are ready for on-line learning; many leaders are not
• Leaders see their projects as unique and view going on-line as a
costly risk, not a strategic cost- and time-saving tool
• Traditional TPD approaches do not transfer well to on-line learning
• TPD pedagogy, staffing, and budgeting must be rethought
Copyright © 2000. SRI International. All rights reserved.
TM
Performance Assessment
Links in Science
• Leader: Edys Quellmalz (quelmalz@unix.sri.com)
http://www.ctl.sri.com/pals
• Focus: An interactive resource bank for
improving standards-based science assessment
• Size: $1.2M for 3 years
• Funder: National Science Foundation
– Elementary, Secondary, Informal Education (EI SE)
– Instructional Materials Development
Copyright © 2000. SRI International. All rights reserved.
Copyright © 2000. SRI International. All rights reserved.
Project Participants
Assessment Consortia
• Council of Chief State School Officers (CCSSO)
• California Systemic Initiatives Assessment Consortium
State Departments of Education
• Connecticut Department of Education
• Illinois Department of Education
• Kentucky Department of Education
School Districts
• Springfield, Illinois
• Vancouver, Washington
Science Curriculum Projects
Teachers
Copyright © 2000. SRI International. All rights reserved.
Performance Assessment
Links in Science
Lessons Learned
• Start by addressing priorities voiced by key
participant groups (state science assessment
programs, professional development groups)
• Partner with representatives of participant groups
• Involve participants, contributors, and funders in
prototype design, seed content, and user testing
Copyright © 2000. SRI International. All rights reserved.
Performance Assessment
Links in Science
Addressing Challenges
• Obtain commitment of partners to content
development and review
• Engage partners in ongoing interactions and
community events
• Design incentives for users to provide feedback
• Build on models for maintenance and growth
• Collect systematic evidence of quality, usability and
impact
• Balance public service and commercial functions
Copyright © 2000. SRI International. All rights reserved.
Online Evaluation Resource Library
• Leader: Edys S. Quellmalz (quelmalz@unix.sri.com)
http://oerl.sri.com
• Focus: Improving Project Evaluation
• Size: $1.75M for 7 years
• Funder: National Science Foundation
– Research, Evaluation, and Communication
Copyright © 2000. SRI International. All rights reserved.
Online Evaluation Resource Library
Project Participants
• NSF Current and Prospective Principal Investigators
and Evaluators
• NSF Program Officers
• Evaluation Professors, Students
• Evaluation Training Programs
• The Evaluation Community
Copyright © 2000. SRI International. All rights reserved.
Online Evaluation Resource Library
Lessons Learned
• Exemplary content may be hard to find
• Develop navigation supports for novice users
• Involve key stakeholders in design and reviews
• Seek multiple dissemination paths early on
Copyright © 2000. SRI International. All rights reserved.
Online Evaluation Resource Library
Addressing Challenges
• Enable participants to take multiple paths to access
information and resources; “no one size fits all”
• Engage participants in active learning experiences
aimed at extending their knowledge base and
strategies
• Support and provide incentives for content
contributions
• Support collaboration and peer review
Copyright © 2000. SRI International. All rights reserved.
Center for Innovative Learning
Technologies (CILT)
• Leaders:
Roy Pea & Barbara Means
SRI International
Marcia Linn
UC Berkeley
John Bransford
Vanderbilt University
Bob Tinker
Concord Consortium
• Mission: To serve as a national resource for
stimulating research on innovative technologyenabled solutions to critical problems in K-14
learning
http://cilt.org
• Funding sources:
– NSF ($6M total for 4 years)
– Industry partners (Intel: $100K/year)
Copyright © 2000. SRI International. All rights reserved.
CILT Activities and Participants
• Annual workshop for harvesting knowledge and leveraging
diverse efforts on “theme teams” of high-priority and
breakthrough opportunity
•
•
•
•
Visualization and Modeling
Ubiquitous Computing
Community Tools
Assessments for Learning
• 400-500 Participants set priorities for partnership projects; CILT
later “seed funds” promising efforts and promotes synergy
efforts
• “CILT Knowledge Network” for People, Places, Projects,
Bibliography, Collaborations, Syllabi
http://kn.cilt.org
• Postdoctoral training program
Copyright © 2000. SRI International. All rights reserved.
CILT as a Multi-level NIC
SRI Vanderbilt
Concord Berkeley
Copyright © 2000. SRI International. All rights reserved.
CILT as a Multi-level NIC
SRI Vanderbilt
Concord Berkeley
Research
Community
Copyright © 2000. SRI International. All rights reserved.
CILT as a Multi-level NIC
Industry
Policy
SRI Vanderbilt
Practice
Concord Berkeley
Research
Community
Copyright © 2000. SRI International. All rights reserved.
Lessons Learned:
Getting Started
• NIC-ing takes a huge amount of coordination plan for it!
– Tasks that are often viewed as “overhead” by
partners become core NIC functions
• Work with all groups of stakeholders early
• Provide a concrete focus or venue
for sharing and collaboration (e.g., improving
learning technology R&D projects; workshops)
Copyright © 2000. SRI International. All rights reserved.
Indicators of CILT Success
• Informal results as important as formal ones
• Formal Industry Alliance partnerships are difficult,
but ad-hoc industry/research interactions are fruitful
• Metrics include:
• CILT-sponsored collaborations that resulted in
sustained outside funding
• Ideas we promoted that took hold in research, policy,
education, product development communities
• Isolated individuals who got the connections/support
they needed to succeed
Copyright © 2000. SRI International. All rights reserved.
Lessons Learned:
CILT Challenges
• Contracting: Universities and other institutions
like to own, not share (public-domain)
• Financing: Under traditional models, the
process of collaboration is rarely funded
• Time and attention: NICs bring together the
best and brightest - and therefore “the busy”
• Design: It’s easier to share curricular resources
when they were designed for adaptability from
the start
Copyright © 2000. SRI International. All rights reserved.
Leaders: Jeremy Roschelle, Chris DiGiano, Roy
Pea (SRI); Jim Kaput (U. Mass., Dartmouth)
http://www.escot.org
Focus: Improving quality, interoperability and
reuse of software components for teaching
middle school mathematics
Size: $2 million for 2 years
Funder: National Science Foundation
Copyright © 2000. SRI International. All rights reserved.
NEED: Without a NIC, disparate math education
efforts have their own isolated improvement
processes, resulting in ‘application islands.’
Math
Teachers
ShowMe
Curricula
MathForum
Resources
Univ. of
CO SW
SimCalc
SW
Key Press
SW
CTL SW
• Redundant
• Non-cumulative
• Unconnected
Copyright © 2000. SRI International. All rights reserved.
ESCOT as a NIC
Evolving knowledge network and interoperable
software library for continuous improvement by
accumulating, integrating, sharing, and testing work.
Math
Teachers
MathForum
Resources
ShowMe
Curricula
ESCOT
Testbed
Univ. of CO
Components
• Cumulative
SimCalc
Components
Key Press
Components
CTL
Components
• Integrated
• Continual testing
Copyright © 2000. SRI International. All rights reserved.
Growing Partnerships and Results
Overall: 8 funded institutional partners, plus 25+ volunteer
institutional partners; 23 software developers
Non-Profits: SRI International; U. Massachusetts; U. Colorado;
Swarthmore College; Stanford University; UC-Berkeley; many
middle schools
Small Companies: Key Curriculum Press, DesignWorlds
International: Centre for Constructive and Experimental
Mathematics (Canada); Computer Technology Institute (Greece)
Results:
• ESCOT now has 37 components, inc. spreadsheet, web browser,
grapher, scripting languages, simulation engines, geometry sketchpad...
• ESCOT database of 5 major math textbooks indexes 3000 places to
use technology components to enhance learning
Copyright © 2000. SRI International. All rights reserved.
Lessons Learned (startup)
• Vet ideas for NIC in extensive planning process
• Pick motivated, complementary, communicative
core partners
• Support extensive face-to-face contact for first 6
months
• Clear tasks / expectations for partner roles
• Create good multi-channel communications
infrastructure (extranet site, e-mail lists, phone
conferences, software configuration
management)
Copyright © 2000. SRI International. All rights reserved.
Lessons Learned (Ongoing)
• Structure IP cross licensing carefully; hard to
both build trust and create the most value from
synergy across developers
• Focus NIC around frequent integration task all
participate in (Problem of the Week)
• Bi-weekly NIC newsletter serves to keep
members motivated, yet hard to keep loosely
coupled and busy partners on-task
• Challenge of finding $$ for project coordination
• Key metric: How easy is it for stakeholders to
adopt techniques coming from the NIC?
Copyright © 2000. SRI International. All rights reserved.
A guiding principle for CTL
• Developing NICs powerfully amplifies your
ability to solve important interdisciplinary
problems in education no one can solve
alone, and...
• Requires and grows champions
• Establishes Centers of Excellence
• Seeds business development (partners and
funders come back to us)
• Grows name and leadership recognition
• Supports recruitment
Copyright © 2000. SRI International. All rights reserved.
Cross-NICing and Meta-NICs?
• Large challenges due to demanding natures of NICs and
domain differences
• NOTE: each of our education NICs had different
mechanisms and solutions:
• TAPPED IN: virtual places; member services
• PALS: web-based digital library and training tools
• OERL: digital library of case-study models, and tools
• CILT: F2F workshops; CILT-KN; partnership seed funding
• ESCOT: interoperability testbed; POW integration teams
• All needed activities for integrating knowledge and
incenting contributions and participation.
Copyright © 2000. SRI International. All rights reserved.