EDUC 7705
I.
Course Number:
EDUC 7710
Course Title: Principles, Trends, and Issues in Standardized Educational Testing
College: Bagwell College of Education
Semester
II.
Instructor:
III.
Class Meetings:
IV.
Required Texts: Koretz, Daniel (2008). Measuring Up: What Educational Testing Really Tells Us, Cambridge, MA: First
Harvard Press.
V.
Catalog Course Description:
This graduate course for educators focuses on the critical analysis of national and global large-scale educational testing, emphasizing
the core principles, trends and issues surrounding the testing and measurement of achievement. This course is designed for master
level students without extensive mathematical training and covers topics such as the evolution of testing in the US and globally, issues
surrounding testing of students with disabilities or English language learners, item analysis with statistics, test domains, sampling,
population, measurement error, reliability, validity, score inflation, factors influencing scale scores, scaling, test statistics,
performance-based statistics, and testing bias. Graduate candidates will explore these topics within the frameworks of common large
scale tests.
VI.
Professional Portfolio Narrative:
A required element in each portfolio for the Graduate Program is the portfolio narrative. The purpose of the portfolio narrative is to
ensure that every candidate reflects on each of the proficiencies on the CPI with regard to what evidence the candidate has selected for
his/her portfolio. In your portfolio, you need to include a narrative which includes descriptive, analytic and reflective writing in which
1
EDUC 7705
you reflect on each proficiency and how you make the case that the evidence you have selected in your portfolio supports a particular
proficiency, using the Portfolio Narrative Rubric as a guide. The narrative should be comprehensive, documenting research-based
best practices.
Each graduate candidate is required to compile both an online portfolio of evidence that documents each candidate’s proficiencies as
defined by the graduate. An additional required element in each portfolio for the Graduate Program is description, analysis and
reflection on each piece of evidence you place for each of the proficiencies. Using Chalk and Wire technology, this means identifying
the content and role of the evidence, and then describing the importance of each piece of evidence. Something like the following:
Date: EVIDENCE OF TITLE: To what time period, approximately or exactly, does this presentation refer?
Context: EVIDENCE TITLE: "This/these artifacts were developed to" …describe with one or two sentences the condition under
which the artifact(s) were created (part of a course requirement/field placement requirement/purpose related to licensure)
Role: EVIDENCE TITLE: What was your role in the event(s) described? Were you acting as part of a collaborative team? Alone?
Author? Editor? Researcher? Instructor? This field is placed here to allow you to indicate what your contribution to the overall
development of the artifact(s) presented This is ethically a requirement if you collaborated with others who also made contributions.
Reflection/Importance: EVIDENCE TITLE: This is by far the most significant information. You should concisely and clearly
explain:
What is happening in this presentation?
How does this artifact (or artifacts) used at that time, clearly illustrate your capacity to perform the standard you are presenting?
What next? Upon reflection, what has this experience suggested as "next moves" for you as a developing professional?
Here´s a simple example of the phrasing you might use for this section:
"I have included/associated/linked this….NAME OF ARTIFACT(s) with NAME OF A DOMAIN AND COMPONENT. I
feel….NAME OF THE ARTIFACT(S) belongs under this standard because….PROVIDE RATIONALE IN TWO OR FOUR
SENTENCES. This artifact(s) demonstrate(s) my ability/position/emerging skill/competence with regard to NAME A
STANDARD/COMPONENT in that PROVIDE AN EXPLANATION OF HOW THE EXPERIENCE YOU HAVE HAD CLEARLY
SHOWS YOUR CONFIDENCE/SKILL/CAPACITY RELATIVE TO THE DOMAIN OR COMPONENT. Given my experience, I
am determined/intend/will/plan….IF APPROPRIATE, DESCRIBE SPECIFIC ACTIONS YOU WILL TAKE TO FURTHER
DEVELOP YOUR PROFESSIONAL SKILLS IN THIS AREA."
2
EDUC 7705
VII.
Purpose and Rationale:
KENNESAW STATE UNIVERSITY’S CONCEPTUAL FRAMEWORK:
Collaborative development of expertise in teaching and learning
The Professional Teacher Education Unit (PTEU) at Kennesaw State University is committed to developing expertise among
candidates in initial and advanced programs as teachers and leaders who possess the capability, intent and expertise to facilitate high
levels of learning in all of their students through effective, research-based practices in classroom instruction, and who enhance the
structures that support all learning. To that end, the PTEU fosters the development of candidates as they progress through stages of
growth from novice to proficient to expert and leader. Within the PTEU conceptual framework, expertise is viewed as a process of
continued development, not an end-state. To be effective, teachers and educational leaders must embrace the notion that teaching and
learning are entwined and that only through the implementation of validated practices can all students construct meaning and reach
high levels of learning. In that way, candidates are facilitators of the teaching and learning process. Finally, the PTEU recognizes,
values and demonstrates collaborative practices across the college and university and extends collaboration to the community-at-large.
Through this collaboration with professionals in the university, the public and private schools, parents and other professional partners,
the PTEU meets the ultimate goal of assisting Georgia schools in bringing all students to high levels of learning.
Knowledge Base:
Teacher development is generally recognized as a continuum that includes four phases: preservice, induction, in-service, renewal
(Odell, Huling, and Sweeny, 2000). Just as Sternberg (1996) believes that the concept of expertise is central to analyzing the teachinglearning process, the teacher education faculty at KSU believe that the concept of expertise is central to preparing effective classroom
teachers and teacher leaders. Researchers describe how during the continuum phases teachers progress from being Novices learning to
survive in classrooms toward becoming Experts who have achieved elegance in their teaching. We, like Sternberg (1998), believe that
expertise is not an end-state but a process of continued development.
Use of Technology:
Technology Standards for Educators are required by the Professional Standards Commission. Telecommunication and information
technologies will be integrated throughout the master teacher preparation program, and all candidates must be able to use technology
to improve student learning and meet Georgia Technology Standards for Educators. During the courses, candidates will be provided
with opportunities to explore and use instructional media. They will master use of productivity tools, such as multimedia facilities,
3
EDUC 7705
local-net and Internet, and feel confident to design multimedia instructional materials, and create WWW resources.
Field Based Activities
While completing your graduate program at Kennesaw State University, you are required to be involved in a variety of leadership and
school-based activities directed at the improvement of teaching and learning. Appropriate activities may include, but are not limited
to, attending and presenting at professional conferences, actively serving on or chairing school-based committees, attending
PTA/school board meetings, leading or presenting professional development activities at the school or district level, and participating
in education-related community events. As you continue your educational experiences, you are encouraged to explore every
opportunity to learn by doing.
VIII: Goals and Objectives:
As a result of satisfactory fulfillment of the requirements of this course, the candidate will accomplish the objectives listed in the table
below.
Course Goals and Objectives
KSU M.Ed CPI
Assessment
General Objective: 2. The graduate teacher candidate will
Outcome 2,3
demonstrate an understanding of the most important factor in testing,
validity, and be able to explain what is meant by the statement, “ a test
is not valid or invalid, it is the inference drawn from the score that is
valid or invalid.
Assessment: Reflective Short Paper
General Objective #1: The graduate candidate will be able to outline Outcome 2
major shifts in American testing requirements beginning in the 1960s.
The candidate will be able to link these differing tests and the political
influence brought to bear on these tests.
Assessment:
Formal Examination and Short Paper (5-page)
General Objective #3: The graduate teacher candidate will be able to Outcome 1
recognize and utilize fundamental measurement concepts and
procedures.
Specific Objectives: The graduate teacher candidate will be able
NBPTS Link
PSC/NCATE Link
Core 3
Professional and
pedagogical skills
and knowledge
Core 3
Professional and
pedagogical skills
and knowledge
Core 3
Professional and
pedagogical skills
and knowledge
4
EDUC 7705
to
a.
recognize the characteristics of a frequency distribution
and frequency polygons.
b. recognize the characteristics of percentiles.
d. interpret percentiles.
e. recognize the characteristics of percentile ranks.
f. interpret percentile ranks.
g. calculate percentile ranks.
h. find ranks for a given set of scores.
j. recognize the characteristics of a mean.
o. recognize the characteristics of the standard deviation
and its square, the variance.
p. interpret standard deviations as measures of dispersion.
q. recognize the characteristics of the normal curve.
r. recognize the characteristics of derived scores.
s. recognize the characteristics of standard scores.
u. interpret z-scores.
v. calculate standard scores that have a mean of 50 and a
standard deviation of 10.
w. interpret standard scores having a mean of 50 and a
standard deviation of 10.
x. recognize the characteristics of a correlation coefficient.
y. interpret a correlation coefficient.,
z. develop a basic understanding of the concepts of
inference, estimation and prediction,
interpreting/designing comparative studies, interpreting
computer generated or published reports and statistical
summaries
Assessment:
Formal Examination
Homework Practice Activities
General Objective #5: The graduate candidate will be able compare
and contrast the CRCT and national standardized achievement test
5
EDUC 7705
specifically
1. What subjects are tested, and what kinds of content and skills are
covered in each?
2. How detailed is the specification of test content?
3. How is the test constructed (number of items per subtest and topic,
types of items,
time limits)?
4. Is there alignment between either test and national standards in your
content area?
5. What types of scores are provided, and what interpretations are they
intended to
support? This includes, for example, the types of scales that are used
to report
performance; the number of subscales reported, if any; the type of
information
provided at the level of both students and schools; and the variety and
detail of score
reports.
6. What statistical concepts are necessary to understand the results?
7. Locate instances of (in the two tests): consequential validity,
content validity, convergent validity, discriminate validity, score
inflation, and bias.
Assessment:
Group Project and Presentation
General Objectives #6: The graduate candidate will explore the
various models of measuring student growth across time and teacher
accountability. Specific topics may include growth and value-added
models, item analysis with statistics: answer distribution, omit rate, pvalue, point-biserial, average item difficulty, reliability and standard
error of measurement.
Assessment:
Formal Examination
Applications
6
EDUC 7705
General Objective #7: The graduate teacher candidate will
demonstrate critical reflection on the use and misuses of today’s highstakes tests and be able to communicate this knowledge to others,
including colleagues, parents and graduate teacher candidates.
Assessment:
Formal Examination
Presentation
General Objective #8: The graduate teacher candidate will recognize,
detect, and control measurement bias in testing and become familiar
with techniques to ensure multicultural validity, also issues
surrounding testing of students with disabilities or English language
learners Particular attention will be paid to test bias and student
achievement in low achieving urban schools.
Assessment:
Formal Examination
General Objective 11: The graduate teacher candidate will write
descriptively, analytically, and reflectively.
Assessment:
All written assignments
General Objective #12: The graduate teacher candidate will work
collaboratively and provide feedback to peers.
Assessment:
Professionalism Evaluation
General Objective #13: The graduate teacher candidate will follow
institutional policies and professional guidelines of academic honesty,
and exhibits professional behavior in interactions with professors and
colleagues.
Assessment
Peer and Professor Feedback
IX.
Outcome 3,
Outcome 1
Core 5, 4
Disposition
Professional and
pedagogical skills
and knowledge
Outcome 2
Core 1, 4
Professional and
pedagogical
knowledge and skills.
Dispositions
Outcome 3
Core
propositions 4
and 5
Outcome 3
Core
propositions 4
and 5
Outcome 3
Core
propositions 4
and 5
Professional and
pedagogical
knowledge and skills.
Dispositions
Dispositions
Dispositions
Requirements/Assignments:
A and B: There will be two formal open book examinations worth 25 points each. (50 points total)
7
EDUC 7705
C. There will be five short assignments each worth 5 points. Only your top four grades will be counted. (20 points)
1. A short-one page paper responding to the statement “a test is not valid or invalid, it is the inference drawn from the score that is
valid or invalid.
2-5. Short applied psychometric exercises responding to the various terms cited in objective #3.
D. A group presentation focusing on a comparison of the NAEP CRCT, and ITBS answering (15 points)
1. What subjects are tested, and what kinds of content and skills are covered in each?
2. How detailed is the specification of test content?
3. How is the test constructed (number of items per subtest and topic, types of items,
time limits)?
4. Is there alignment between either test and national standards in your content area?
5. What types of scores are provided, and what interpretations are they intended to
support? This includes, for example, the types of scales that are used to report
performance; the number of subscales reported, if any; the type of information
provided at the level of both students and schools; and the variety and detail of score
reports.
6. What statistical concepts are necessary to understand the results?
7. Locate instances of (in the two tests): consequential validity, content validity, convergent validity, discriminate validity, score
inflation, and bias.
E. A Group presentation describing a longitudinal model of teacher accountability for student learning. Include the principals and
major components of the model, discuss shortcomings of the model, and reflect on this model may have in education. (15 points)
E. Assignment #5 : Professionalism: Behaviors that indicate professional skill may be demonstrated in a graduate teacher candidate’s
approach to participating in and completing the requirements for any particular course, such as this one. Professional behavior will be
monitored in this course. Should concerns arise regarding an individual teacher candidate; the instructor of this course will
communicate these concerns to the graduate teacher candidate and to the program coordinator of the teacher candidate’s major.
Does the teacher candidate:
 Model high standards and expectations for him or herself?
 Display a commitment to the profession of helping students learn?
 Enjoy learning and indicate enthusiasm toward working with students to facilitate their learning?
8
EDUC 7705








Regularly reflect on and assess his or her performance and effectiveness for self-improvement?
Learn from experiences and show improvement over time?
Manage interpersonal relationships effectively?
Demonstrate courtesy, respect, and civility in interactions with others?
Work collaboratively with professional colleagues and faculty?
Accept responsibility for actions and non-actions, placing the locus of control upon him or herself rather than shifting
blame or claiming inability to control outside factors?
Maintain appropriate attire and appearance?
Promote and model standards of academic honesty?
Tardiness, and leaving class early are also issues of professionalism.
Finally, one final issue regarding professionalism-please turn off all cell phones and pagers during class. Disturbances by these
devices are disrespectful, disrupt the flow of ideas during discussions, and are nuisances that can be easily avoided. Seldom is there a
reason to speak on the phone that could not wait until the end of class.
All written assignments should be typed using 12-point font, double-spaced, on white 8 ½ X 11
paper. They should represent quality, college level work, which includes correct spelling,
grammar and punctuation - utilizing APA ( 5th ed.) style formatting.
X:
Evaluation and Grading:
XI.
Policies
90 – 100 = A
80 – 89 = B
70 – 79 = C
60 – 69 = D……………below 60 = F
Diversity: A variety of materials and instructional strategies will be employed to meet the needs of the different learning styles of
diverse learners in class. Candidates will gain knowledge as well as an understanding of differentiated strategies and curricula for
providing effective instruction and assessment within multicultural classrooms. One element of course work is raising candidate
awareness of critical multicultural issues. A second element is to cause candidates to explore how multiple attributes of multicultural
populations influence decisions in employing specific methods and materials for every graduate teacher candidate. Among these
9
EDUC 7705
attributes are age, disability, ethnicity, family structure, gender, geographic region, giftedness, language, race, religion, sexual
orientation, and socioeconomic status. An emphasis on cognitive style differences provides a background for the consideration of
cultural context.
Kennesaw State University provides program accessibility and accommodations for persons defined as disabled under Section 504 of
the Rehabilitation Act of 1973 or the Americans with Disabilities Act of 1990. A number of services are available to support students
with disabilities within their academic program. In order to make arrangements for special services, students must visit the Office of
Disabled Student Support Services (ext. 6443) and develop an individual assistance plan. In some cases, certification of disability is
required.
Please be aware there are other support/mentor groups on the campus of Kennesaw State University that address each of the
multicultural variables outlined above.
Academic Honesty: KSU expects that graduate students will pursue their academic programs in an ethical, professional manner. Any
work that students present in fulfillment of program or course requirements should represent their own efforts, achieved without
giving or receiving any unauthorized assistance. Any student who is found to have violated these expectations will be subject to
disciplinary action.
Course Outline
10
EDUC 7705
Week (s)
Topic (please refer to specific objectives for details)
1
Major shifts in American testing requirements beginning in the 1960s and reasons.
2, 3, 4
Fundamental measurement concepts and procedures
5,6,7
Application of measurement concepts to instruments and performance standards employed in Georgia
8, 9
Models of measuring student growth across time and teacher accountability
10, 11
Bias in testing
12, 13
Critical reflection on the uses of today’s high-stakes tests
14,15
Communication and application of knowledge
11
EDUC 7705
12
EDUC 7705
XIV. References/Bibliography
American Educational Research Association, American Psychological Association, andNational Council on Measurement in
Education
(1999). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research
Koretz, D., and Hamilton, L. S. (2006). Testing for accountability in K-12.In R. L. Brennan (Ed.), Educational measurement (4th ed.),
Westport, CT: American Council on Education/Praeger.
Linn, R.L., and Miller, M.D. (2004). Measurement and Assessment in Teaching, Ninth Edition.Upper Saddle River, N.J.: Pearson Prentice Hall.
Airasian, P.W. (1997). Classroom assessment. (3rd ed.).New York : McGraw-Hill.
Banks, J. A. & Banks, C. A. M. (Eds.). (1995). Handbook of research on multicultural education. NY: Macmillan.
Dana, R. H. (1993). Multicultural assessment perspectives for professional psychology. Boston: Allyn and
Bacon.
Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart, &
Wilson.
Haney, W. (1989). Testing reasoning and reasoning about testing. Review of Educational Research, 54(9), 557-654.
Haney, W. M., Maduaus, G. F., Lyons, R. (1993). The fractured marketplace for standardized testing. Boston: Kluwer.
Hibbard, K. M. and others. (1996). A teacher's guide to performance-based learning and assessment. Alexandria, VA:
Association for Supervision and Curriculum Development.
Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research. Boston: Allyn and Bacon.
13
EDUC 7705
Gronlund, N. E. & Linn, R. L. (1995). Measurement and evaluation in teaching (7th ed.) New York: Macmillan, Chapter 6,
“Constructing Objective Test Items.”
Katz, M. (1961). Improving classroom tests by means of item analysis. Clearing House, 35, 265-269.
Kohn, A. (2000). The case against standardized testing: Raising the scores, ruining the schools. Westport, CT:
Heinemann.
Linn, R. L., & Gronlund, N. (1995). Measurement and evaluation in teaching (7th ed.). New York: Macmillan.
Messick, S. (1981). Evidence and ethics on the evaluation of tests. Educational Researcher, 10, 9-20.
Popham, W. J. (2002). Classroom assessment- What teachers need to know. (3rd ed.). Boston: Allyn and Bacon.
Stiggins, R. J. (2001). Student-involved classroom assessment. (3rd ed.). Upper Saddle River, NJ: Prentice-Hall.
Wiggins,G. (1998). Educative assessment: designing assessments to inform and improve student performance. San
Francisco, Calif.: Jossey-Bass.
14