Open Pathways:
Testing the Degree Profile
Activity 3
Marshall University
October 5 2012 – February 1, 2013
Open Pathways Project Deadlines
• December 15, 2012
o Complete and submit assessment rubrics for remaining outcomes in
program or major.
• February 1, 2013
o Submit fall 2012 Assessment report using Activity 2 program outcome
rubrics.
• May 17, 2013
o Submit spring 2013 assessment report using new program outcome
rubrics.
• June 30, 2013
o Quality Initiative: Testing the Lumina Foundation’s Degree Qualifications
Profile Learning Outcomes Final Report sent to Higher Learning
Commission and Lumina Foundation.
Activity 3
(in a nutshell)
Step 1:
Review Activity 2 Rubrics.
Step 2:
Set benchmarks for each assessment point.
Step 3:
Review courses and assessments identified for each assessment point.
Step 4:
Use Activity 2 rubrics to assess artifacts at each assessment point.
Step 5:
Collect data
Step 6:
Analyze data. Identify areas in need of improvement.
Step 7:
Based on data analysis, determine an action plan to improve student learning
in program.
Step 8:
Report data using data reporting submission form available at
www.marshall.edu/hlcopenpathways
Activity 3
Step 1: Review Activity 2 rubrics
Step 2: Set benchmarks.
Review Activity 2 rubrics and set benchmarks: Why?
• To revise Performance level names and set benchmarks as
follows:
o
Introductory Benchmark = benchmark expected during first year
of study for Baccalaureate students (benchmark for first
assessment point for Associate’s Degrees)
o
Milestone Benchmark = first assessment point benchmark for
Baccalaureate Degrees (exiting benchmark for Associate’s
Degrees)
o
Capstone Benchmark = exiting benchmark for Baccalaureate
Degrees and first assessment point for Master’s Degrees
o
Advanced Benchmark = exiting benchmark for Master’s Degrees
Activity 3
Step 3: Review courses and assessments identified for each
assessment point.
Review courses and assessments selected for
•
•
•
•
Introductory benchmark assessments
Milestone benchmark assessments
Capstone benchmark assessments
Advanced benchmark assessments
Considerations during review
• Have courses and embedded assessments been identified for each
assessment point?
• Are courses and assessments appropriate
o Correct level of instruction for performance benchmark level?
o Controlled by your program/department?
• Have all courses used for assessment points been mapped to
program and DQP outcomes? If not
• Map course outcomes to program outcomes and to the broad areas of
learning and degree appropriate outcomes in the DQP.
o Use Activity 1 form to do this.
Activity 3,
Step 4: Use Activity 2 rubrics to assess artifacts at each assessment
point.
Use Activity 2 rubrics to
•
•
Assess course embedded (or other appropriate) artifacts
Assess at a minimum of two assessment points
o For Associate’s Programs: Introductory and Milestone
o For Bachelor’s Programs: Milestone and Capstone
o For Master’s Programs: Capstone and Advanced
Activity 3
Step 5: Collect data
• Collect data for each outcome
o Collect data separately for each outcome
o Enter data into Excel, SPSS, or other statistical package.
• SPSS Example
o On bottom of data entry sheet, select “variable view” tab.
o Enter the following variable names in the “name” column
o Student
o Each rubric trait in a separate row (One word – no spaces – if you want to
describe in more detail, add information under the “label” column
o Set performance level value labels for each rubric trait
Activity 3, Step 5 Continued
• SPSS Example Continued
o On bottom of data entry sheet, select “data view” tab.
o The variable names that you entered earlier will appear at the top
of each column.
o Enter data for each variable in the appropriate column
o Student – can use ID, code or name (If you student name, must make variable
“String” rather than “Numeric”
o Enter number that corresponds to rating for each rubric trait for each student
o Run frequency analysis
o
o
o
o
At top of sheet, select “analyze,” then “frequencies”
Move trait variables to the variables box
Click on “OK” to run frequency analysis
Results will be reported in frequency tables
Activity 3
Step 6: Analyze data.
Step 7: Determine an action plan.
Step 8: Report data.
• Step 6
o Analyze data. Identify areas in need of improvement (i.e. relative
strengths and weaknesses).
• Step 7
o Determine an action plan to address weaknesses or make other
modifications to improve student learning.
• Step 8
o Report all information using reporting form found at
o www.marshall.edu/hlcopenpathways
Benchmark = Milestone Performance Level
Areas in need of improvement are method of inquiry and
data analysis and conclusions
0
100%
90%
0
10
20
35
80%
60
70%
60%
Advanced
Capstone
50%
40%
90
Milestone
80
Introductory
65
30%
40
20%
10%
0%
0
Problem
0
Research
Method
Analysis
Due Date Reminders
• December 15, 2012
o Complete mapping of any course used for program level assessment
during fall 2012 that was not previously mapped in Activity 1. Use
Activity 1 submission process.
o Revise Activity 2 rubrics as appropriate.
o Complete and submit assessment rubrics for remaining outcomes in
program or major. Submission form will be available at
www.marshall.edu/hlcopenpathways no later than November 1.
• February 1, 2013
o Submit fall 2012 Assessment reports using Activity 2 program outcome
rubrics. Form for this process will be available no later than November 1.
Course mapping form
Preview of Activity 4
Spring 2013
• Use program outcome rubrics completed in Activity 3 to assess
remaining program outcomes.
• Submit report of spring 2013 assessment by May 17, 2013,
using reporting template used for Activity 3.
Final Goal of Project
• To provide feedback to the HLC and to the Lumina Foundation
for the purpose of improving the DQP.
• To develop a degree profile unique to Marshall University.
o Watch for proposal for a Marshall University Baccalaureate
Degree Profile soon!