Developing and Using
Rubrics to Assess Vision and
Change Outcomes
GCAT-SEEK Workshop, 2013
Dr. Tammy Tobin, Susquehanna University
Adapted from Mary J. Allen, SACS-COC Summer Institute
Designing and Assessing Student
Activities for Vision and Change
Learning Goals - Overview
• Use “Backwards” Course Design
 Identify student learning goal(s) first....
 THEN decide on activities/topics that best allow
students to gain the relevant skills, knowledge and/or
abilities
 THEN decide on a rubric (if needed) and make sure
that it actually addresses the learning goal in ways
that are appropriate for the activity
Identifying Student
Learning Goals: V&C
Core Concepts
• Evolution
• Structure and Function
• Information Flow, Exchange and Storage
• Pathways and Transformations of Energy and Matter
• Systems
Identifying Student Learning
Goals: V&C Core
Competencies
• Ability to Apply the Process of Science
• Ability to Use Modeling and Simulation
• Ability to Use Quantitative Reasoning
• Ability to Tap Into the Interdisciplinary Nature of Science
• Ability to Communicate and Collaborate with Other Disciplines
• Ability to Understand the Relationship Between Science and
Society
Decide How to Best
Assess Student Learning
• Direct vs. Indirect Assessment
 Authentic Assessment
• Formative vs. Summative Assessment
• Imbedded vs. External Assessment
The GCAT SEEK Assessment
Instrument
The Challenge: Develop an assessment instrument that
is broad enough to assess the outcomes of ALL GCATSEEK projects for ALL V&C concepts and
competencies…
Stage One: Develop broad learning goals for GCAT
SEEK – Using V&C categories as guide
Stage Two: Develop and implement a survey/quiz in
Google Forms to assess participant progress on (some
of) these learning goals
Stage Three: Refine the instrument/identify gaps in V&C
coverage
Stage Four: Use data to improve GCAT-SEEK activities
Rubrics
• Classify continuous behaviors (V&C
Competencies!) into categories
• Can be developed to classify any student
product
• Can be used to provide formative feedback to
students, to grade students, and/or to assess
programs
• Are most effective when they are generated at
the same time as the assignment
Creating Rubrics
• Adapt a pre-existing rubric to your needs
Loyola Marymount University has a good
database of samples, including those from the
AAC&U VALUE project.
•http://www.lmu.edu/about/services/academicplanning/assessment/
Assessment_Resources/Rubrics/Example_Rubrics.htm
Steps for Creating Your Own
Rubrics
1. Write down the learning outcome(s) to be assessed:
Vision and Change Core Competencies:
• (STUDENTS WILL BE ABLE TO) TAP INTO THE INTERDISCIPLINARY
NATURE OF SCIENCE.
• (STUDENTS WILL BE ABLE TO) COMMUNICATE AND COLLABORATE
WITH OTHER DISCIPLINES.
• (STUDENTS WILL) UNDERSTAND THE RELATIONSHIP BETWEEN
SCIENCE AND SOCIETY.
Steps for Creating Your Own
Rubrics
2. Identify measurable central components or skills for
the learning goals, using the Vision and Change
descriptors as as your guide.
Ability to tap into the interdisciplinary nature of science
Interdisciplinary science practice may be achieved in several ways. For future biologists, one way is through
developing expertise not just in an area of biology, but also in a related discipline. That way, students will
develop the vocabulary of both disciplines and an ability to think independently and creatively in each as
well. A second, less intensive approach is to develop deep expertise in one area and fluency in related
disciplines. A third option is to serve as a biologist on a multidisciplinary team. All of these routes
develop a student’s facility to apply concepts and knowledge across traditional boundaries. For those not
majoring in biology, the inherent interdisciplinary nature of biology practice lends itself to forming
connections between biology and other sciences and, in so doing, can help all students understand the
way science disciplines inform and reinforce each other.
Steps for Creating Your Own
Rubrics
3. Develop an assessment instrument:
GCAT-SEEK Essay:
Science is an interdisciplinary, collaborative process that can be of great use to society, but only if
it is conveyed in a manner that makes its importance clear to both scientists and non-scientists.
We are interested to see if you have this ability. Write a short (2-3 paragraph) essay in which
you try to convince a charitable foundation to fund an interdisciplinary next generation
sequencing project. In your essay you should briefly explain the project that you will be
performing, why it is important to society, and how next generation sequence data will help you
complete the project. Also convince the foundation to fund the work of at least one individual
who is not a molecular biologist, but without whom your research project would be less
meaningful or effective.”
Steps for Creating Your Own
Rubrics
4.Decide which rubric type to use (holistic or analytic).
Holistic Scoring - One global, holistic score for a
product or behavior.
• See rubric examples 1-2 at end of packet
Analytic Rubrics - Separate, holistic scoring of
specified characteristics of a product or behavior
• See rubrics 3-4 at end of packet
Steps for Creating Your Own
Rubrics
5. Create a descriptor for the highest student performance
level. Keep it short, simple and measurable. Make sure the
rubric asks the student to perform the task!
Science is an interdisciplinary, collaborative process that can be of great use to society, but only if it is
conveyed in a manner that makes its importance clear to both scientists and non-scientists. We are
interested to see if you have this ability. Write a short (2-3 paragraph) essay in which you try to convince a
charitable foundation to fund an interdisciplinary next generation sequencing project. In your essay you
should briefly explain the project that you will be performing, why it is important to society, and how next
generation sequence data will help you complete the project. Also convince the foundation to fund the work
of at least one individual who is not a molecular biologist, but without whom your research project would be
less meaningful or effective.”
ABILITY TO TAP INTO THE INTERDISCIPLINARY NATURE OF
SCIENCE.
5
Accurately and completely describes how both next generation
sequencing and at least one other discipline are necessary to make
the research project chosen more meaningful or effective.
Creating Your Own
Rubrics
6. Identify the word(s) in the descriptor that can vary to
convey different levels of performance
Accurately and completely describes how both next generation
sequencing and at least one other discipline are necessary to
make the research project chosen more meaningful or effective.
• Vary these words when writing your lower performance
level descriptions.
Creating Your Own
Rubrics
7. Create a descriptor for the lowest student performance
level.
Accurately and completely describes how both next generation
sequencing and at least one other discipline are necessary to
make the research project chosen more meaningful or effective
Does not accurately describe how either of the chosen techniques
are necessary to make the research project chosen more
meaningful or effective
8. Decide how many levels to use, and write descriptors
for the intermediate levels.
ABILITY TO TAP INTO THE INTERDISCIPLINARY NATURE OF
SCIENCE
5
Accurately and completely describes how both next generation sequencing and at
least one other discipline are necessary to make the research project chosen more
meaningful or effective
4
Accurately and completely describes how either next generation sequencing or at
least one other discipline is necessary to make the research project chosen more
meaningful or effective. However, the description of the role of the other technique,
while accurate, is superficial
3
Accurately but superficially describes how both next generation sequencing and at
least one other discipline are necessary to make the research project chosen more
meaningful or effective
2
Accurately describes how either next generation sequencing or at least one other
discipline is necessary to make the research project chosen more meaningful or
effective. However, the description of the role of the other technique is inaccurate
1
Does not accurately describe how either of the chosen techniques are necessary
to make the research project chosen more meaningful or effective
Practice, Practice, Practice …
In teams, identify the central components or skills for the
learning goals below, using the Vision and Change
descriptors as as your guide (V&C page 15).
• (STUDENTS WILL BE ABLE TO) COMMUNICATE AND COLLABORATE
WITH OTHER DISCIPLINES.
• (STUDENTS WILL) UNDERSTAND THE RELATIONSHIP BETWEEN
SCIENCE AND SOCIETY:
Practice, Practice, Practice…
Next, create a descriptor for the highest and the lowest
student performance levels. Make sure they are
measurable, and that they measure something the student
was actually asked to do.
Varying the appropriate terms, write descriptors for all of
the levels of achievement in the chart below
5
4
3
2
1
Using Rubrics
Pilot Testing the Rubric
Managing Group Readings
One reader per document
Requires careful reader calibration!
Two independent readers per document
Paired readers
Analyzing the Data
• Which level of performance
is "good enough“?
• Consider using a preformed
computer spreadsheet to
quickly enter and process
your data as you go.
• Modify the rubric/prompt for
the next cycle, as needed.
• Reward yourselves!