Frameworks for Assessment of Student Learning:
Questions of Concept, Practice, and Detail
Christine Siegel, Ph.D.
Associate Professor of School Psychology
Associate Dean, Graduate School of Education & Allied Professions
Fairfield University
Learning in Action: Innovative Pedagogy and Course Redesign XI
Summer Teaching Conference
June 2, 2011
How do we know what students are
learning?
C. Siegel, May 2011
What is being learned?
C. Siegel, May 2011
How do we know what students are
learning?
CONCEPTUAL
What is learning?
PRACTICAL
DETAIL
Who are learners?
What do we expect students to learn?
How does learning occur?
What is being learned?
How do we know what students are
learning?
How do we know what we know?
What factors influence
learning?
C. Siegel, May 2011
How can learning be
promoted?
CONCEPTUAL
PRACTICAL
What is learning?
Who are learners?
DETAIL
What do we expect students to learn?
How does learning occur?
What is being learned?
How do we know what students are
learning?
How do we know what we know?
What factors influence
learning?
C. Siegel May 2011
How can learning be
promoted?
CONCEPTUAL
PRACTICAL
What is learning?
Who are learners?
DETAIL
What do we expect students to learn?
What is being learned?
How do we know what students are
learning?
How do we know what we know?
C. Siegel May 2011
What is learning?
C. Siegel May 2011
What is learning?
Learning involves change
Learning endures over time
Learning occurs through experience
C. Siegel May 2011
Who are learners?
C. Siegel May 2011
What do we expect students to learn?
C. Siegel May 2011
What do we expect students to learn?
Aspirations
Learning
Outcomes
C. Siegel May 2011
What do we expect students to learn?
Aspirations
Learning
Outcomes
C. Siegel May 2011
Goals
What do we expect students to learn?
Aspirations
Learning
Outcomes
Goals
Objectives
C. Siegel May 2011
What is being learned?
• Knowledge
–
–
–
–
General
Content -specific
Procedural
Content-specific procedural
• Skills
–
–
–
–
Basic
Higher Order
Cross-disciplinary
Content-Specific
• Values
– Perspectives
– World views
– Attitudes
C. Siegel May 2011
How do we know what students are learning?
Inferential
C. Siegel May 2011
How do we know what students learn?
• What students say
• What students do
• What students make
C. Siegel May 2011
How do we know what
students are learning?
Knowledge
general
content
procedural
content-specific procedures
Skills
basic
higher-order
cross-disciplinary
content-specific
Values
attitudes
world views
perspectives
C. Siegel May 2011
What students say
(statements)
What students do
(behaviors)
What students make
(products)
How do we know what
students are learning?
What students say
(statements)
What students do
(behaviors)
Knowledge
• Test/ quiz performance
general
content
procedural
content-specific procedures
• Homework performance
Skills
basic
higher-order
cross-disciplinary
content-specific
Values
attitudes
world views
perspectives
C. Siegel May 2011
• Performance on essays /
papers
What students make
(products)
How do we know what
students are learning?
What students say
(statements)
What students do
(behaviors)
What students make
(products)
Knowledge
•Survey/ questionnaire
• Test/ quiz performance
• Portfolios
general
content
procedural
content-specific procedures
•Interviews
• Homework performance
•Capstone project
•Reflective writings
• Performance on essays /
papers
• Works of art
•Course Evaluations
•Research projects
•Professional work products
•Survey / questionnaire
Skills
basic
higher-order
cross-disciplinary
content-specific
• Theatrical/ Artistic
Performance
•Interviews
• Portfolios
•Capstone project
•Demonstration
•Reflective writings
•Works of art
•Class presentation
•Think-aloud
•Research projects
•Observations by others
•Plans / proposals
•Professional work products
•Course Evaluations
•Survey / self-report
• Anecdotal Records
•Portfolios
Values
•Interviews
• Engaged time
•Capstone project
attitudes
world views
perspectives
•Self-report
• Money spent
•Works of art
C. Siegel May 2011
•Reflective writings
•Research project
•Course Evaluations
•Professional work products
GSEAP
Psychological &
Educational
Consultation
School Psychology
PY 538
Example 1
PY 538
PSYCHO-EDUCATIONAL ASSESSMENT II: COGNITION
C. Siegel May 2011
What is learning?
Gaining
proficiency in the
use of
intelligence tests
for education
decision making,
through exposure
to and practice
with a variety of
measures of
intelligence
Who are learners?
Advanced degree
candidates in
school psychology
What do we expect students to learn?
ASPIRATION: Employability as a School
Psychologist
GOAL: Proficiency in data-based decision
making
What is being learned?
OBJECTIVES: Acquiring knowledge, skills,
and ethics associated with the professional
administration and interpretation of
standardized intelligence tests
How do we know what students learn?
C. Siegel May 2011
How do I know what
students are learning
in PY 538?
Knowledge
Current intelligence theory
Intelligence test administration
Intelligence test interpretation
Cross-battery testing
What students say
(statements)
What students do
(behaviors)
What students make
(products)
•Survey/ questionnaire
•“Pre-test” of familiarity
with individually
administered
standardized intelligence
tests
• Test/ quiz performance
•Quizzes on test
development,
psychometric properties,
administration, and
interpretation of tests
• Professional work products
•Test protocols
•Course evaluations
•Written assessment
reports
•Tests on current
intelligence theory
Report writing
Skills
Intelligence test administration
Intelligence test interpretation
Cross-battery testing
Report writing
Values
Awareness of the limitations
intelligence tests in educational
decision making
Awareness of issues related to the use
of intelligence test with special
populations
C. Siegel May 2011
•Think-aloud
•In class demonstrations
of test administration
•In class exercises in test
score interpretations
• Demonstration
•Videos of test
administration
•Observations by others
•Peer observation of inclass demonstration
• Professional work products
•Test protocols
•Written assessment
reports
•Course evaluations
•Survey/ questionnaire
•“Pre-test” of familiarity
with individually
administered
standardized intelligence
tests
•Course evaluations
• Anecdotal Records
•Informal observation of
student comments about
tests and practice
subjects
• Engaged time
•Informal reports of time
spent with practice
subjects
•Professional work products
•Consent Forms
•Test protocols
•Written assessment
reports
How do I know what
students are learning
in PY 538?
Knowledge
Current intelligence theory
Intelligence test administration
Intelligence test interpretation
What students say
(statements)
•Survey/ questionnaire
•“Pre-test” of familiarity
with individually
administered standardized
intelligence tests
Intelligence test interpretation
Cross-battery testing
Report writing
Values
Awareness of the limitations
intelligence tests in educational
decision making
Awareness of issues related to the use
of intelligence test with ethnic
minority populations
C. Siegel May 2011
• Professional work
products
•Test protocols
•Written
assessment reports
•Quizzes on cross-battery
assessment
Report writing
Intelligence test administration
• Test/ quiz performance
•Quizzes on test
development,
psychometric properties,
administration, and
interpretation of tests
What students make
(products)
•Course evaluations
Cross-battery testing
Skills
What students do
(behaviors)
•Tests on current
intelligence theory
•Think-aloud
•In class demonstrations
of test administration
•In class exercises in test
score interpretations
• Demonstration
•Videos of test
administration
•Observations by others
•Peer observation on inclass demonstration
• Professional work
products
•Test protocols
•Written
assessment reports
•Course evaluations
•Survey/ questionnaire
•“Pre-test” of familiarity
with individually
administered standardized
intelligence tests
•Course evaluations
• Anecdotal Records
•Informal observation of
student comments about
tests and practice subjects
• Engaged time
•Informal reports of time
spent practicing, and
administering tests
•Professional work products
•Consent Forms
•Test protocols
•Written
assessment reports
How do we know what School Psychology
candidates are learning?
What students
say
(statements)
Knowledge
•Admissions
Interviews
•Data-based decision making
•Interpersonal communication , collaboration and consultation
•Effective instruction & development of cognitive academic skills
•Socialization and development of behavioral competencies
•Student diversity in learning and development
•School structure, organization, and climate
•Prevention, wellness promotion, and crisis intervention
•Home/school community collaboration
•Research and program evaluation
•Legal issues, ethical practice and professional development
•Information technology
Skills
•Data-based decision making
•Interpersonal communication , collaboration and consultation
•Effective instruction & development of cognitive academic skills
•Socialization and development of behavioral competencies
•Student diversity in learning and development
•School structure, organization, and climate
•Prevention, wellness promotion, and crisis intervention
•Home/school community collaboration
•Research and program evaluation
•Legal issues, ethical practice and professional development
•Information technology
•Philosophy of School
Psychology
What students
do
(behaviors)
• Test/ quiz
performance
What students
make
(products)
• e - portfolios
• MA-level
comprehensive
exam
•Course Evaluations
•Alumni Surveys
•PRAXIS II in
School
Psychology
•Employer Surveys
•Plans / proposals
•Treatment
Plans
•Class presentations
• e -portfolios
•Observations by others
•Research
Proposals
•Field work
Goals
•Internship
Goals
•Field Work
Evaluations
•Internship
Evaluations
•Employer Surveys
•Alumni Surveys
Values
•Legal issues, ethical practice and professional development
•Reflective writings
across courses
• Anecdotal Records
• Engaged time
•Course Evaluations
•e-portfolios
How do we know what School Psychology
candidates are learning?
What students
say
(statements)
Knowledge
•Admissions
Interviews
•Data-based decision making
•Interpersonal communication , collaboration and consultation
•Effective instruction & development of cognitive academic skills
•Socialization and development of behavioral competencies
•Student diversity in learning and development
•School structure, organization, and climate
•Prevention, wellness promotion, and crisis intervention
•Home/school community collaboration
•Research and program evaluation
•Legal issues, ethical practice and professional development
•Information technology
Skills
•Data-based decision making
•Interpersonal communication , collaboration and consultation
•Effective instruction & development of cognitive academic skills
•Socialization and development of behavioral competencies
•Student diversity in learning and development
•School structure, organization, and climate
•Prevention, wellness promotion, and crisis intervention
•Home/school community collaboration
•Research and program evaluation
•Legal issues, ethical practice and professional development
•Information technology
•Philosophy of School
Psychology
What students
do
(behaviors)
• Test/ quiz
performance
What students
make
(products)
• e - portfolios
• MA-level
comprehensive
exam
•Course Evaluations
•Alumni Surveys
•PRAXIS II in
School
Psychology
•Employer Surveys
•Plans / proposals
•Treatment
Plans
•Research
Proposals
•Field work
Goals
•Internship
Goals
•Class presentations
• e -portfolios
•Observations by
others
•Field Work
Evaluations
•Internship
Evaluations
•Employer Surveys
•Alumni Surveys
Values
•Legal issues, ethical practice and professional development
•Reflective writings
across courses
• Anecdotal Records
• Engaged time
•Course Evaluations
•e-portfolios
GSEAP
PEC
School
Psychology
Applied
Psychology
ESTP
Special
Education
TESOL/
Bilingual
Education
Example 2
PY – ED 534
THEORIES OF LEARNING
C. Siegel May 2011
PY-ED 534
Secondary
Education
MFT
TEFO
Schoolbased
MFT
What is learning?
Making explicit
the implicit
assumptions
professionals
have about
learning that
guide their
practice, through
the critical
examination of
and reflection on
a variety of
learning theories
across multiple
fields of study
Who are
learners?
Advanced degree
candidates in
variety of
programs across
GSEAP
What is being learned?
What do we expect students to learn?
ASPIRATION: The GSEAP aims to prepare
scholar practitioners who engage in
reflective practice, promote human
development and learning, and advocate
for children and families.
GOALS: Develop a personal theory of
learning to guide their professional practice
PROFICIENCIES / OBJECTIVES: Acquiring
foundational knowledge in education, skills
for designing effective instruction and
interventions, dispositions of reflective
practice and advocacy
How do we know what students learn?
C. Siegel May 2011
How do I know what students
are learning in PY-ED 534?
Knowledge
•Historical and contemporary theories
of learning
•Psychological, social, and cultural
influences on learning
What students say
(statements)
•Survey/ questionnaire
•Pre and post group
discussions around central
questions
What students do
(behaviors)
What students make
(products)
• Participation in class
discussions and group work
• Professional work products
•Personal Theory of
Learning
•Class Presentations
•Participation in Class Discussions
•What do you know about…?
•What did you think about…?
•Reflective Writings
•Course evaluations
•Reflective Writings
• Observations during group
work
•Course evaluations
•Class Presentations
Values
•Reflective Writings
•Respect for a variety of opinions
regarding the process of learning
•CL group evaluation form
•Anecdotal Records
Informal observations
during group work
•Course evaluations
•CL group evaluation form
Skills
•Critical analysis
•Personal Reflection
• Professional work products
•Personal Theory of
Learning
•Communicating about learning across
a range of perspectives
(Developing effective interventions
and instruction)
•Respect for open dialogue across
opinions
•Awareness of the impact of learning
theory on professional practice
•Engaged Time
Informal observations
during group work
•Professional work products
•Personal Theory of
Leaning
CONCEPTUAL
PRACTICAL
What is learning?
Who are learners?
DETAIL
What do we expect students to learn?
What is being learned?
How do we know what students learn?
How do we know what we know?
C. Siegel May 2011
How do we know what we know?
C. Siegel May 2011
How do we know what we know?
Quantitative
Qualitative
• Reliability
• Credibility
–
–
–
–
Stability
Equivalence
Internal consistency
Inter-rater
• Validity
– Content
– Internal
– External
• Predictive
• concurrent
C. Siegel May 2011
• Transferability
• Dependability
• Confirmability
Special Thanks
C. Siegel May 2011