• By the end of today’s session, you should:
– Understand basic assessment of student learning;
– Tentatively identify when and where assessment of the six ITF LOs will take place in your program;
– Develop a plan for assessment measures to be used
(ITF rubric, major field test, standardized final, etc.);
– Develop a plan for collecting baseline data (preferably during Spring 2014);
– Decide when you and your colleagues will discuss assessment results ;
– Know what your responsibilities are for reporting ; and
– Be familiar with the ITF assessment calendar .


http://img4.wikia.nocookie.net/__cb20100313135124/simpsons/images/4/47/Dabf20.gif

From Linda Suskie, courtesy of Jane Wolfson, Towson University

1)
4)
2)
3)
Excerpted from Assessing Student Learning by Linda Suskie, Jossey-Bass, 2009

• Outcomes : Your six discipline-specific ITF LOs
• Opportunities : Where will students practice ITF skills?
• Measurement Sites : Which courses and assignments?
• Measures : Projects, papers, exam questions, clinicals
• Sampling (if used)
• Baseline Data : When, where, and how collected
• Sharing : Who can act on results to improve student learning?

QUESTIONING
SEEKING
EVALUATING
USING
COMMUNICATING
RECOGNIZING
• Six INtopFORM Learning Outcomes
• What do they mean in YOUR discipline?
– See Parts 1 and 2 of your planning worksheets
Record draft ITF learning outcomes for your program in Column A of the Assessment Plan Template

• Must allow students to practice and develop skill in each of the six INtopFORM Learning Outcomes
• Where will this take place in your curriculum?
– See results of Curricular Mapping exercise
• What methods might you use?
– See Teaching Ideas (Part 3 of planning worksheets) for each ITF LO
Record possible learning opportunities for your students in Column B of the Assessment Plan Template; please be specific (course + assignment or activity)

• Where will you measure student learning?
• Must be measurable or observable
• Goal
– Collect “reasonably accurate, truthful evidence”
• Doesn’t have to be worthy of publication
– Good enough to use with confidence
• Possibilities include single assignments, capstone courses, projects, final exam items

• Two options:
– Modify the ITF assessment rubric to evaluate papers or projects (preferred);
– Study students’ performance on designated items on major field tests, standardized final exams, or locally designed tests of information fluency
Record where assessment might take place (course, assignment) in your curriculum in Column C of the
Assessment Plan Template. Include possible measures
(exam items, rubrics) for your program in Column D.

QUESTIONING: Students ask questions that facilitate the solution of problems and the pursuit of opportunities.
Questions define the scope of the area of interest.
Questions identify key issues to be resolved.
Questions demonstrate understanding of the problem, task or topic.
Questions encourage a broad search for information.
Questions are relevant to the problem, task or topic.
Questions bring needed focus or structure to the problem, task or topic.
Questions examine diverse perspectives on the problem, task or topic.
Questions reflect attitudes of openness and curiosity.
Overall Rating (required item)
Other items (subject to approval):
SEEKING: Students locate or create information needed to accomplish their purposes.
Sources used are credible.
Sources used are appropriate for the purpose.
The student uses several sources.
The student uses different kinds of sources.
Sources and information used indicate a thorough search process.
Sources and information used indicate persistence and resourcefulness in seeking information.
The student insightfully evaluates the search process.
The student conducted an iterative search process.
The student conducted an efficient search process.
The student correctly identifies limitations of research he or she conducted.
Overall Rating (required item)
Other items (subject to approval):
EVALUATING: Students apply critical thinking skills in evaluating sources, information and search processes.
The student correctly analyzes the reliability of sources or information.
The student correctly analyzes the potential bias of sources.
The student insightfully discusses the implications of information.
The student demonstrates comprehension of information.
The student correctly identifies assumptions underlying information.
The student identifies questions raised by information.
The student analyzes his or her own and others’ assumptions.
Overall Rating (required item)
Other items (subject to approval):

USING: Students assemble and synthesize information to accomplish their purposes.
Information used is relevant to the purpose.
The amount of information provided is sufficient for the purpose.
The student draws sound conclusions from information.
The student skillfully synthesizes information from multiple sources.
The student draws non-obvious insights from information.
The analysis of information flows logically from point to point.
The student does not oversimplify complex information.
The student clearly relates information to the larger purpose.
The student draws upon diverse perspectives.
Overall Rating (required item)
Other items (subject to approval):
COMMUNICATING: Students communicate information effectively.
The student aids others’ understanding of information.
The student provides useful explanations of information.
The student presents information accurately.
The student presents information completely.
The presentation of information helps others see connections or larger themes.
The presentation of information arouses others’ interest.
The presentation of information evokes questions or further study by others.
The student openly acknowledges when information is inconsistent or inconclusive.
Overall Rating (required item)
Other items (subject to approval):
RECOGNIZING: Students recognize the responsibilities and consequences related to information ethics and intellectual property.
The student clearly identifies sources of information used.
The student respects confidentiality rights associated with information.
The student shows fair use in the quotation of sources.
Overall Rating (required item)
Other items (subject to approval):

• Must evaluate all six ITF LOs
• Must occur annually
• Must provide data on the culminating skills and abilities of students in a program of study
– Ordinarily in the senior year / 4000-level courses
• Must yield findings applicable to all majors
– Can evaluate all majors’ work or that of a sufficient representative sample

• More evidence collected means higher confidence in conclusions, but be practical
• If you use a sample, it should adequately represent your student population
– Online vs. on ground
– Adult vs. traditional
Record from whom you will collect assessment information in Column E of the Assessment Plan
Template; describe sampling (if applicable)

• Must be collected during Year 1
– Spring 2015 preferred
• Use standard reporting template (provided) to include:
– Overall assessment for each of the six outcomes
• Mathematical average (can be weighted) or separate rating
– Number and % of students rated as:
3 = Excellent
2 = Satisfactory
1 = Unsatisfactory
– Whether each student was rated satisfactory on all outcomes
Record ideas of how your program might collect baseline data in Column F of the Assessment Plan Template.

Example Program of Study QEP Rubric
Student ID A
Questioning: Students ask questions that facilitate the solution of problems and the pursuit of opportunites
Students raise appropriate questions regarding a particular issue
The line of argument anticipates and adequately responds to relevant objections
Overall assessment of Questioning
Seeking: Students locate or create information needed to accomplish their purposes
Uses evidence-based sources and peer reviewed literature to support all aspects of the paper
Overall assessment of Seeking
Evaluating: Students apply critical thinking skills in evaluating sources, information, and search processes
Identifies appropriate problem for case presented
Demonstrates clear understanding of response to treatment
Correctly determines next steps in process
Overall assessment of Evaluating
Using: Students assemble and synthesize their information to accomplish their purposes
Adequately describes problem using professional terminology
Demonstrates logical progression of relevant data and facts
Develops accurate and appropriate AQI statements
Plans appropriate strategies for monitoring interventions
Overall assessment of Using
Communicating: Students communicate information effectively
Addresses significance of the issue and facts pertinent to the issue
Strong conclusion summarizing data, facts, and information related to issue
Uses appropriate fonts, font sizes, spacing, cover page, bibliography and headings for each section
Uses appropriate word choice, free from grammatical and spelling errors
Overall assessment of Communicating
Recognizing: Students recognize the responsibilities and consequences related to information ethics and intellectual property
Sources cited appropriately using APA citation style
Gives credit when paraphrasing or rewording
Overall assessment of Recognizing
3.00
3.00
3.00
3.00
3.00
3.00
3.00
2.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
2.00
2.00
2.00
2.00
3.00
2.00
2.50
3.00
3.00
2.00
3.00
2.75
3.00
3.00
3.00
2.00
2.75
1.00
2.00
1.50
B
3.00
2.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
2.00
2.00
3.00
2.50
3.00
3.00
3.00
C Dept. Totals
3.00
2.67
2.67
2.67
2.67
2.33
2.67
2.50
2.67
3.00
2.33
2.83
3.00
3.00
2.67
3.00
2.89
3.00
2.67
2.67
2.67
2.83

• Assessment is worthless if it isn’t useful
– How will results be analyzed? By whom?
– How and when will results be shared?
– Who can act on your assessment data to make improvements?
Discuss how your program might use the assessment information you collect and record in
Column G of the Assessment Plan Template.

• INtopFORM provides support during Faculty
Learning Communities
• INtopFORM staff will hold meetings with each program to discuss assessment and implementation plans
• The Director of Assessment (with assistance from the Director of the QEP) will
– Assist faculty in designing assessments
– Approve each programs’ assessment protocols

For each program:
• Improvement in Years 2-5 compared to baseline data in Year 1
• >30% of the rubric ratings or test scores on each
INtopFORM learning outcome are “excellent”
• >90% of the rubric ratings or test scores on each
INtopFORM learning outcome are at least
“satisfactory”
• >90% of program majors achieve rubric ratings or test scores demonstrating at least “satisfactory” performance on all INtopFORM learning outcomes

• ETSU’s Senior Exit Exam, administered since 2005 for general education assessment
• 34 questions, 45 minutes in length
• Five areas: Analysis and Interpretation, Inference,
Evaluation and Explanation, Deductive Reasoning, and Inductive Reasoning
• CCTST maps well to EVALUATING and USING
INtopFORM learning outcomes
• Office of Assessment will collect CCTST data annually and provide reports to programs of study

CCTST Area Description
Score Interpretation
Weak Moderate Strong
Analysis (maps to
EVALUATING) used when pulling apart arguments and points of view
Inference (maps to USING) used whenever drawing conclusions based on reasons and evidence
Evaluation (maps to EVALUATING) used when deciding believability of a given statement
Induction
Deduction used when deciding that evidence supports truth in given conclusion used when deciding a false conclusion is impossible considering all arguments are true
0 – 2
0 – 3
3 – 4
0 – 5 6 – 11
4 – 7
0 – 5 6 – 11
0 – 5 6 – 11
5 or more
12 or more
8 or more
12 or more
12 or more
CCTST Total
Score
• 8-12
(Weak)
• 13-18
(Moderate)
• 19-23
(Strong)
• ≥24
(Superior)

18
17,8
17,6
17,4
17,2
17
16,8
16,6
Year N Mean score Percentile Rank
16,2
2005-06 1135 17.5
61 st
16
– 68 th
2006-07 2677 17.4
54 th – 61 st
2007-08 1815 17.7
2008-09 1593 17.2
2009-10 1512 17.3
2010-11 1672 17.5
54 th – 61
55
56
59 th th th st
2011-12 1859 17.6
2012-13 1723 16.5
2013-14 2016 17.0
57 th
49 th
51 st
Academic Year

For each program:
• Mean CCTST Total Scores and subtest * scores should exceed baseline in Years 2-5
• >30% of scores on each CCTST subtest * at or above
“strong”
• >90% of scores on each CCTST subtest * at or above
“moderate”
• >30% of CCTST Total Scores are at least 19 (“strong”)
• >90% of CCTST Total Scores are at least 15
(“moderate”)
*mapped to an INtopFORM learning outcome

• Each fall in Years 3-5
• Includes Director of Assessment, QEP Director, program leaders, and other interested faculty
• Purpose
– Review previous year’s assessment data
– Use results to refine plan for enhancing information fluency
– As needed, define improvement actions, responsible parties, and calendars
• Annual assessment and improvement report drafted by
Directors and approved by program faculty

Develop program assessment plan X
Collect baseline data
Collect and analyze results, refine plan, make improvements
Year 1: 2014-15
Su Fa Sp
X X (refine)
X
Year 2: 2015-16
Su Fa Sp
X X X
Develop program assessment plan
Collect baseline data
Collect and analyze results, refine plan, make improvements
Year 3: 2016-17 Year 4: 2017-18
Su Fa Sp Su Fa Sp
X X X X X X

When
Fall
Spring
First Year of Participation
What
Create a plan to help all students in the program attain the six INtopFORM learning outcomes
Who
Program of Study Faculty
Design assessment of ITF learning outcomes
Program of Study Faculty
Collect and report baseline assessment data Program of Study Faculty
Fall, Spring, and Summer
Summer
Collect and report baseline CCTST data
Send participating programs of study analysis of baseline data, including CCTST
Office of Assessment
Office of Assessment

Second Year of Participation
When What
Fall Meet to review baseline data and discuss implications for planned improvements in program
Fall and Spring Implement plans to help students attain learning outcomes
Collect and report data on ITF learning outcomes
Collect and report CCTST data Fall, Spring, and Summer
Summer Send participating programs of study analysis of data, including CCTST
Who
Director, QEP; Director,
Assessment; Program of
Study Faculty
Program of Study Faculty
Program of Study Faculty
Office of Assessment
Office of Assessment

When
Early Fall
What
Third, Fourth, and Fifth Year of Participation
Who
Meet to evaluate prior year assessment data and refine program’s INtopFORM plan
Program of Study Faculty;
Director, Assessment;
Director, QEP
Program of Study Faculty Fall and Spring Collect and report data on ITF learning outcomes
Collect and report CCTST data Fall, Spring, and Summer
Summer Send participating programs of study analysis of data, including CCTST
Office of Assessment
Office of Assessment




Director of Assessment clavier@etsu.edu
439-7483
136 Sherrod