Creating a
Teacher Education
Assessment System
David C. Smith, Dean Emeritus
College of Education
University of Florida
Email: smithdc@aol.com
Why We Need an Assessment
System
We need to do a better job of assessing what
we are doing and how it can be improved.
Our assessment systems need to
accommodate a paradigm shift from
assessing teaching to assessing learning.
We need to move from collecting data to
implementing an assessment system.
David C. Smith
Creating a Teacher Education
Assessment System
2
We need to be seen as the sources of the best
information available about the preparation of
professional educators.
Each SCDE needs an assessment system.
One assessment system should be capable of
addressing multiple demands for data.
An effective assessment system can help to increase
confidence in the preparation of professional
educators.
David C. Smith
Creating a Teacher Education
Assessment System
3
There are Multiple Demands for
Information
Internal - advocacy, analysis, and
planning.

Making a case for resources.
How do you compare with other units in your
institution?
 Analyzing units within your SCDE.
 How do you compare with “significant others”?



Who are “significant others”?
Annual Reports.
David C. Smith
Creating a Teacher Education
Assessment System
4
External - Accreditation and evaluation.
Accreditation - NCATE and regional.
 Reports to the State Department of
Education.
 Reports to the State Higher Education
Board.
 Title II information.

David C. Smith
Creating a Teacher Education
Assessment System
5
The pressure is very real,
and
it is a high stakes
environment.
David C. Smith
Creating a Teacher Education
Assessment System
6
Considerations in Building an
Assessment System.
The baseline is very important. That may
become your standard. (Neither easy or trivial. High tide and low
tide.)
Historical data are important insofar as they lead
to an understanding of your current needs and
direction. (The Past is Prologue. National Archives.)
Collect information that is likely to be valuable
and relevant avoid the temptation to collect data
which are not meaningful but readily available.


Trivial information may be readily available.
Have a reason for each item in your assessment system.
David C. Smith
Creating a Teacher Education
Assessment System
7
Further
Considerations.
Consider your audience. (Bob and Gene show.)
A system that is not current is not of much
value. (It must be kept up to date.)
It is imperative that your data be accurate.
(The
price is lost credibility.)
You get big numbers by adding little numbers.
(Institutional research.)
Collect data elements at least one level lower
than the unit. (College down 10%.)
David C. Smith
Creating a Teacher Education
Assessment System
8
Considerations.
-
(Still more.)
Data are inextricably related to resource
allocation and the demonstration of
productivity. (Without data – what did you do with what you got?
Summer budget allocation.)
The first line of defense is to question the
accuracy of the data or to ignore dissonance
producing data. (Use mine until you have better numbers.)
Try to use data that are difficult for others to
reject. (I got the information from your office.)
David C. Smith
Creating a Teacher Education
Assessment System
9
Considerations.
-
(Even more.)
Avoid asking for data which require
calculations. (Avoid introducing errors.)
Precise measures are better than crude
measures; crude measures are better than
approximations; and approximations are better
than no data at all. (Important variables may be difficult to measure.)
You can make better approximations for your
unit than anyone else. (Yo0u need not apologize for reasonable
approximations.)
David C. Smith
Creating a Teacher Education
Assessment System
10
Considerations.
-
(Still more.)
Do not expect sharp changes over a short time
interval. (You can’t drive an 18-wheeler like a sports car. Complex
organizations change slowly.)
An assessment system faces conflicting
demands; the ability to compare and the need
to tailor. (Data from comparable institutions are powerful. Yet your system
must fit your needs.)
The system must be capable of easy
modification.
The assessment system must be capable of
supporting a rapid response.
David C. Smith
Creating a Teacher Education
Assessment System
11
Considerations.
-
(Even more.)
People will attend to those things that are
measured, rather than those things which are
said to be important. (Achievement tests.)
It doesn’t make sense to think you need to
measure with a micrometer if you are going to
mark with a piece of chalk and cut with an axe. (
If it is important. Quantitative economics.)
Use data to make your problem important to
others. (Abolish elementary education.)
David C. Smith
Creating a Teacher Education
Assessment System
12
Principles. (Enough – at last.)
A single system is much more powerful and
useful than multiple data sources.
Information is power. You need to be the best
informed persons on the campus with respect
to matters relating to the preparation of
professional educators. (An assessment system can help
you accomplish that goal.)
David C. Smith
Creating a Teacher Education
Assessment System
13
Data can help us get our act
together.
David C. Smith
Creating a Teacher Education
Assessment System
14
The institution almost assuredly
collects data.
 Those data signal what the institution




values.
The institution can hardly question the
accuracy of the data it collects and reports.
Those data can be studied and interpreted.
What do these data suggest about your
SCDE?
Some of these data can be imported into
your assessment system.
David C. Smith
Creating a Teacher Education
Assessment System
15
What do we mean by the
term “assessment
system”?
A carefully designed and functional
system for collecting, organizing,
maintaining, analyzing and utilizing
meaningful information.
A system is made up of sub-systems.
To be effective, the system must be
incorporated into the regular operation
of the SCDE.
David C. Smith
Creating a Teacher Education
Assessment System
16
A Performance Assessment
System involves utilizing
information to support vital
activities of the SCDE.
Assessment
David C. Smith
Analysis
Creating a Teacher Education
Assessment System
Implementation
17
What kinds of data might
be included in a Teacher
Education Performance
Assessment System©?
(Establishing a conceptual
framework.)
Resource Data
Productivity Data
Candidate Performance Data
Faculty Performance Data
David C. Smith
Creating a Teacher Education
Assessment System
18
Structure of a Teacher Education
Assessment System©
 First Symbol - Classification
A = Resource Data
√ B = Productivity Data
√ C = Candidate Performance Data
√ D= Faculty Performance Data
 Second Symbol - Level
√ 1= Unit Data
√ 2= Institutional Data
√ 3= Calculated Data
 Third Symbol - Category
√ Sub-systems
 Fourth Symbol - Number
√ Element Number
√
David C. Smith
Creating a Teacher Education
Assessment System
19
TEAS© System
and
Sub-systems
A. Resources
1. Unit
1. Human Resources
2. Fiscal Resources
3. Demographic Data
2. Institution
1. Human Resources
2. Fiscal Resources
3. Demographic Data
3. Derived
David C. Smith
B. Productivity
1. Unit
1. Enrollment
2. Loads and Assignments
3. Program
4. Title II Data
5. External $ Generated
6. Student Demographic Data
2. Institution
1. .Enrollment
2. Loads and Assignments
3. External $ Generated
4. Student Demographic Data
3. Derived
1. Enrollment
2. Loads and Assignments
3. External $ Generated
4. Student Demographic Data
Creating a Teacher Education
Assessment System
20
TEAS© System and Sub-systems
C. Candidate Performance
1. Unit
1. Initial Program Admission
2. Early in Program
3. Mid-program
4. Summative
5. Graduate Follow-up
6. Advanced Programs Admission
7. Mid-program
8. Summative
9. Graduate Follow-up
2. Institution
1. Undergraduate
2. Graduate
3. Derived
1. Undergraduate
2. Advanced Programs
David C. Smith
D.Faculty
1. Unit
1. Teaching Evaluations
2. Scholarly Productivity
3. Professional Service
4. Institutional Service
2. Institution
1. Teaching Evaluations
2. Scholarly Productivity
3. Professional Service
4. Institutional Service
3. Derived
1. Scholarly Productivity
2. Institutional Service
Creating a Teacher Education
Assessment System
21
Kinds of Information in TEAS.
AACTE – NCATE Annual Report data.
Demographic information.











A.1.3.1 Number White
A.1.3.2 Number Black
A.1.3.3 Number Hispanic
A.1.3.4 Number Asian
A.1.3.5 Number Native American
A.1.3.6 Number Pacific Islander
A.1.3.7 Number Eskimo
A.1.3.8 Number Other
A.1.3.9 Total Minority
A.1.3.10 Total Faculty
A.1.3.11 % Minority Faculty
David C. Smith
0
0
#DIV/0!
Creating a Teacher Education
Assessment System
22
Examples of Use - Resource
Information.
Section A - Resource Data -- TEAS
Unit Data – Resource
2000
Human Resources
A.1.1.1 FTE Faculty - Full Time
A.1.1.2 FTE Faculty - Part Time
A.1.1.3 Summer FTE Faculty
A.1.1.4 Total FTE Faculty
A.1.1.5 FTE Administration
A.1.1.6 FTE Support Personnel
David C. Smith
Creating a Teacher Education
Assessment System
1999
0
23
Comparative Salary
Information.
Section A - Resource Data -- Derived
A.3.1 Instr. $ Diff. Inst. – Unit
#DIV/0!
A.3.2 Asst. Prof. $ Diff. Inst. – Unit
#DIV/0!
A.3.3 Assoc. Prof. $ Diff. Inst. – Unit
#DIV/0!
A.3.4 Prof. $ Diff. Inst. – Unit
#DIV/0!
A.3.5 All Ranks $Creating
Diff.a Teacher
Inst.Education
– Unit
David#DIV/0!
C. Smith
Assessment System
24
Productivity Data.
Section B - Productivity Data -- Unit
Enrollment
B.1.1.1 FTE Enroll. Undergrad.
B.1.1.2 FTE Enroll. Grad.
B.1.1.3 Total FTE Enroll.
B.1.1.4 Headcount Enroll. Undergrad
B.1.1.5 Headcount Enroll. Grad
B.1.1.6 Total Headcount Enroll.
B.1.1.7 Total Undergrad. SCH
B.1.1.8 Total Grad. SCH
B.1.1.9 Total SCH
B.1.1.10 Summer SCH Undergrad.
B.1.1.11 Summer SCH Grad.
B.1.1.12 Total Summer SCH
David C. Smith
Creating a Teacher Education
Assessment System
0
0
0
0
25
Data for Staff Planning.
A.1.2.18 % Instructors
A.1.2.19 % Asst. Prof.
A.1.2.20 % Assoc. Prof.
A.1.2.21 % Prof.
A.1.2.22 Avg. Instructor
A.1.2.23 Avg. Asst. Prof.
A.1.2.24 Avg. Assoc. Prof.
A.1.2.25 Avg. Prof.
A.1.2.26 Avg. All Ranks
David C. Smith
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
Creating a Teacher Education
Assessment System
26
Budget Analysis.
A.1.2.27 Consumable Expenses
A.1.2.28 Equipment
A.1.2.29 Library Budget
A.1.2.30 Travel Budget
A.1.2.31 Graduate Assistants
A.1.2.32 Other
A.1.2.33 Total Budget
A.1.2.34 Faculty Support Factor
David C. Smith
Creating a Teacher Education
Assessment System
$0
#DIV/0!
27
Equity in the System –
Over Time.
A.3.10 Unit % Faculty
#DIV/0!
A.3.11 Unit % Summer Faculty
#DIV/0!
A .3.12 Unit % FTE Support Personnel
#DIV/0!
A.3.13 Unit % Part-time Faculty
#DIV/0!
A.3.14 Unit % Consumable Expenses #DIV/0!
A.3.15 Unit % Equipment
#DIV/0!
A.3.16 Unit % Administration
#DIV/0!
A.3.17 Unit % Library
#DIV/0!
A.3.18 Unit % Travel
#DIV/0!
A.3.19 Unit % Budget
#DIV/0!
David C. Smith
Creating a Teacher Education
Assessment System
28
Monitoring the System.
A.3.20 Unit % Faculty Change
A.3.21 Inst. % Faculty Change
A.3.22 Unit Total Budget % Change
A.3.23 Inst. Total Budget % Change
B.3.2.9 Unit % Change Total SCH
#DIV/0!
B.3.2.10 Inst. % Change Total SCH
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
Patterns become apparent over time!
David C. Smith
Creating a Teacher Education
Assessment System
29
Using Productivity Data Examples.
B.1.2.7 Hours Taught/FTE Faculty
#DIV/0!
B.2.2.4 Hours Taught/FTE Faculty
#DIV/0!
B.1.2.8 Sch/FTE Faculty
#DIV/0!
B.2.1.10 Sch/FTE Faculty
#DIV/0!
B.1.3.1 Cost/SCH
#DIV/0!
B.2.2.5 Cost/SCH
#DIV/0!
Compare productivity over time!
David C. Smith
Creating a Teacher Education
Assessment System
30
Candidate Performance
Data.
Section C - Unit -- Candidate Performance Data
Basic Programs
Admission
C.1.1.1 SAT/ACT
C.1.1.2 Writing Sample
C.1.1.3 GPA
C.1.1.4 Interview
David C. Smith
Creating a Teacher Education
Assessment System
31
Candidate Performance
Data.
Multiple Measures
Early in Program
C.1.2.1 Praxis I
C.1.2.2 Educ. GPA
C.1.2.3 Classroom Obser. - Knowledge.
C.1.2.4 Classroom Obser. – Application
Clear Expectations
David C. Smith
Creating a Teacher Education
Assessment System
32
Candidate Performance
Data.
Mid-Program
C.1.3.1 Total GPA
C.1.3.2 Tutorial Knowledge
C.1.3.3 Tutorial ESL
C.1.3.4 Small Group Knowledge
C.1.3.5 Small Group ESL
ESL = Evidence of Student Learning.
David C. Smith
Creating a Teacher Education
Assessment System
33
Candidate Performance Data.
Summative
C.1.4.1 Final GPA
C.1.4.2 GPA in Non-Educ. Courses
C.1.4.5 Portfolio
C.1.4.6 Student Teaching Grade
C.1.4.7 No. Taking Praxis II
C.1.4.8 No. Passing Praxis II
C.1.4.9 Praxis II Success Rate
C.1.4.10 No. Taking State License Exam.
C.1.4.11 No. Passing State License Exam.
C.1.4.12 State License Exam. Rate
C.1.4.13 No. Candidates Rec. Elem. Lisc.
C.1.4.14 No. Candidates Rec. Sec. Lisc.
C.1.4.15 Total No. Candidates Rec. Lisc.
Creating a Teacher Education
0
David C. Smith
Assessment System
#DIV/0!
#DIV/0!
34
Candidate Impact
Data.
How important are schools, colleges and
departments of education?
C.1.4.16 Potential Candidate Impact
=(D345*25*25)+(D346*125*25)
Movers and shakers. One years’ graduating class.
David C. Smith
Creating a Teacher Education
Assessment System
35
Candidate Performance
Data.
Graduate Follow-Up
C.1.5.1 Prog. Assessment - Graduates
C.1.5.2 Graduate Assess. - Principals
David C. Smith
Creating a Teacher Education
Assessment System
36
Faculty Performance Data.
Professional Service
D.1.3.1 No.State Wrkshps, Elem
D.1.3.2 No. State Wrkshps, Sec
D.1.3.3 No. State Wrkshps, Total
D.1.3.4 Elem. Tchrs. @ State Conf.
D.1.3.5 Sec. Tchrs. @ State Conf.
D.1.3.6 Total Tchrs. @ State conf. (PS)
D.1.3.7 Potential Student Impact
D.1.3.8 No. State Wrkshps/Faculty
David C. Smith
Creating a Teacher Education
Assessment System
0
0
0
#DIV/0!
37
Much of this information is
important only to deans.
NOT SO!
Without information of this kind the dean
cannot effectively serve as an advocate for
the unit.
Much of the information can be generated
only by the faculty and program heads.
Often other units are able to make their data
more compelling.
David C. Smith
Creating a Teacher Education
Assessment System
38
Relationships Among Assessment
Systems
Institution
Department
Unit
Program
Remember, you get big numbers by adding little numbers.
So you may need to have sub-unit assessment systems.
David C. Smith
Creating a Teacher Education
Assessment System
39
Kinds of Data
Bottom Up Data
Top Down Data
SCH
Budget Data
Enrollment Data
FTE Faculty
FTE Students
Student Demographics
Most of these data
can be acquired.
David C. Smith
Faculty performance data.
Candidate performance
data.
Student performance data.
Most of these data
can only be
generated at the
program level and
aggregated to the
unit level.
Creating a Teacher Education
Assessment System
40
Focus on Faculty Performance
Critical Check Points
Admission (GPA – Essay – Interview)
Knowledge
Dispositions
N.A.
Skills
Evidence of student learning –
Early Program Assessment (Classroom
Observation – Participation)
Knowledge
Skills
Dispositions
Evidence of student learning –
N.A.
Mid-Program Assessment (Diagnosis - Tutorial)
Knowledge
Dispositions
David C. Smith
Skills
Evidence of student learning
Creating a Teacher Education
Assessment System
41
Critical Check Points
Exit Assessment (Student Teaching
Evaluation – Tests)
Knowledge - Skills - Dispositions
Evidence of student learning
Follow-Up Assessment (Principal
Evaluation – Graduate Survey)
Knowledge - Skills - Dispositions
Evidence of student learning
David C. Smith
Creating a Teacher Education
Assessment System
42
Critical Check Points
KnowledgeSkill
Admission
GPA
Essay
Interview
Disposition Student
Learning
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
Early Program Assessment
Classroom Observation
Classroom Participation
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
Mid-Program Assessment
Diagnosis
Tutorial
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
Exit Assessment
Portfolio
Test
Student Teaching Eval.
Follow-Up Assessment
Principal Evaluation
Survey Data
Continuing Contract
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
Consider: Comparability of data among programs.
Aggregation of candidate performance data.
The need for explicit criteria.
The need for rubrics and scaling.
David C. Smith
Creating a Teacher Education
Assessment System
43
Attributes of this system.
(Exclusive of Title II Data)
366 pieces of information are contained in the
system.
216 pieces of information (59%) are entered.


Many of the data are already routinely collected.
It is very likely that the institution and unit currently
collects some of the data.
150 pieces of information (41%) are generated.
Consider the kinds of information generated.




Comparative data within the system.
Comparative data over time – multiple years.
Evaluative data.
Data for planning.
Graphs and tables can be developed from the data
in this system. (Enrollment and Budget, % of Faculty and
Budget, % of Grad. SCH and Budget)
David C. Smith
Creating a Teacher Education
Assessment System
44
What You Have and What You
Need
180
160
140
120
100
Serie s1
80
60
40
20
0
Prob. Have
David C. Smith
Prob. Need
Derived
Creating a Teacher Education
Assessment System
45
Wha t Yo u Have a nd What You Ne ed (v1.0)©
Section
Prob. Have
Prob. Need
Derived
A.1
35
1
15
A.2
34
1
15
A.3
0
0
23
B.1
32
4
20
B.2
26
4
16
B.3
0
0
25
C.1
18
19
4
C.2
4
1
0
C.3
0
0
5
D.1
13
5
11
D.2
0
18
11
D.3
0
0
6
Total
162
53
151
Percent
44.3%
14.5%
41.3%
David C. Smith
Creating a Teacher Education
Assessment System
Total
51
50
23
56
46
25
41
5
5
29
29
6
366
100 .0%
46
This System and Your System
The data in this performance assessment system can
be used for:




Unit planning, advocacy,annual reports and budgeting.
Institutional planning and evaluation.
External advocacy and information.
Accreditation, program approval and review.
This system is easily modified.
Remember, you do not need to create a performance
assessment system annually.
The entire assessment system does not need to be
put into operation simultaneously.
David C. Smith
Creating a Teacher Education
Assessment System
47
Your Assessment System.
Think about the framework for your system.
All data elements do not need the same
baseline.
You can implement your assessment system
systematically and gradually.
An assessment system can help make
informed decisions about improving your
programs.
There is a relationship between resources
and productivity. Make unconventional
comparisons. (Compare SCH/FTE with candidate
evaluations of programs over time.)
David C. Smith
Creating a Teacher Education
Assessment System
48
Your System
This system is available to you.
Email: smithdc@aol.com
But you need your own system.
This system is easily modifiable.
There are advantages in having some
commonality in data sets - within the state,
region and nationally.
David C. Smith
Creating a Teacher Education
Assessment System
49
Remember
The burden of proof rests with the unit.
You do not need to measure everything.
You need to present sufficient information
for the BOE to make a reasoned and
reasonable judgment.
David C. Smith
Creating a Teacher Education
Assessment System
50
The Real Question is.....
Does your assessment system
provide sufficient information to
permit you and others (in this case
NCATE) to draw accurate and
reasoned judgments about the
quality and performance of your
teacher candidates, faculty, unit and
the context in which you function?
David C. Smith
Creating a Teacher Education
Assessment System
51
Closing thoughts.
Our ability to serve as effective advocates is
essential.
No effective case can be made without data
for support.
We must document the level of our
effectiveness in a much more sophisticated
manner. That requires an assessment
system.
We need to become much more proactive in
portraying the teacher education enterprise.
David C. Smith
Creating a Teacher Education
Assessment System
52