M.S. in Education Assessment in the Major Report By Dr. Renee Chandler, Program Director
2010 Submitted: October 2011
TableofContents
1. 2. 3. 4. 5. 6. 7. 8. 9. Outcomes of Previous Assessments ............................................................................................................................................................................ 2 Questions To Be Answered From This Year’s Assessment ........................................................................................................................................ 2 Methods Used to Gather Data: .................................................................................................................................................................................... 3 Number of Students Involved. ..................................................................................................................................................................................... 3 Level of Students ......................................................................................................................................................................................................... 4 Data Analysis Techniques ........................................................................................................................................................................................... 4 Analysis of Data .......................................................................................................................................................................................................... 4 How Results Are Shared with Key Instructors .......................................................................................................................................................... 12 Plans for Improvements Based on Results ................................................................................................................................................................ 12
Appendix A: MSE Benchmarks........................................................................................................................................................................................ 13 Appendix B: Scoring Rubric for MSE Program Application .......................................................................................................................................... 14 Appendix C: Dispositions Rating Form ........................................................................................................................................................................... 15 Appendix D: MSE Rubric for Portfolio ............................................................................................................................................................................ 18 EducationAIMReport2010
Page1
1.
OutcomesofPreviousAssessments
The MS Education program is in the process of implementing a program revision and a new assessment system. Prior to this report, data available
was limited to enrollment data, follow-up surveys, and portfolio ratings. Fall 2011 marks the start of the implementation of a revised program, but
many of the aspects of the new assessment system were implemented during 2010.
Three levels of benchmark criteria are outlined in the new assessment plan (see Appendix A). Assessment measures critical to these benchmarks that
are addressed in this report include:
 ratings of application essays
 disposition ratings by recommenders identified by student self-ratings and instructor ratings
 e-Portfolio ratings
In addition to the items listed above, this report also includes 2010 demographic data for the MS Education program.
2.
QuestionsToBeAnsweredFromThisYear’sAssessment
The 2010 assessment report will focus on data which addresses the following questions:
 Demographic data: Who are the students in the MS Education program?
 Ratings of application essays: What do we know about the students’ philosophies, goals, and writing skills as they enter the MS Education
program?
 Disposition ratings: What are the relative dispositional strengths and weaknesses of our students as perceived by themselves and their
instructors?
 Portfolio data: How are students performing relative to the National Board Standards?
EducationAIMReport2010
Page2
3.
MethodsUsedtoGatherData:




Demographic data: The UW-Stout Fact Book and other databases on campus have been used to generate the section of this report dedicated
to demographic data.
Ratings of application essays: As the MS Education program-specific applications are processed, the Program Director rates each essay using
the rubric presented in Appendix B. Essay responses are rated on a scale of 0 (Unsatisfactory) to 3 (Proficient/Distinguished) on seven
different factors: 1. Philosophy of Education; 2. Goals as an Educator; 3. Commitment to Learning; 4. Commitment to Excellence; 5. Respect
for Others; 6. Mechanics; and 7. Overall impression. Each of these areas is aligned with the National Board Standards.
Disposition ratings: Accrediting bodies in education (DPI and NCATE) are concerned with candidates’ knowledge, skills, and dispositions.
To address the disposition domain, data was collected from a key instructor and the students using a rating scale (see Appendix C). The
instructor of EDUC-729 (which is typically taken in the first semester of the program) filled out the rubric in Appendix C regarding the
performance of each student. Each student also rated themselves on the same rubric.
Portfolio: All students in the MSE program complete a required course in which they submit a Professional Portfolio. The portfolio is rated
by the instructor of EDUC-790. During the spring and fall semesters of 2010, the program director taught EDUC-790 and rated each
student’s e-portfolio according to the rubric presented in Appendix B. The five areas of assessment align with the National Board Teaching
Standards: 1. Commitment to learners and learning; 2. Content knowledge and content-specific pedagogy; 3. Managing and monitoring
student learning; 4. Systematic reflection and lifelong learning; and 5. Collaboration and participation in learning communities. The
portfolios were scored directly on the Chalk and Wire system on which they were created, allowing for efficient data collection and analysis.
4.
NumberofStudentsInvolved.



The enrollment data presented includes the 121 students enrolled Fall 2010.
A total of 49 applicant essays from Fall 2010 are included in the data presented.
Sixteen students were enrolled in EDUC-790 Professional Portfolio during the Spring semester of 2010 and sixteen students were enrolled in
the Fall semester of 2010. The data from all 32 portfolios will be presented.
EducationAIMReport2010
Page3
5.
LevelofStudents
Only graduate students were involved in the collection of data presented in this report.
6.
DataAnalysisTechniques
Information collected will be placed in tables. Where appropriate, descriptive statistics will be used to analyze the data.
7.
AnalysisofData
EnrollmentData
There were 121 students enrolled Fall 2010 in the MSE online program. About 51 were new students. The characteristics of the students are
summarized in the graphics below (N=121).
EducationAIMReport2010
Page4
ApplicationtoProgram:EssayRatings
As part of the application to program, all applicants write an essay in response to the following prompt: “Attach a statement of approximately 500
words (not to exceed 3 double spaced typewritten pages) which describes your philosophy of education and your goals as an educator. The statement should
clearly give examples of your commitment to learning, your commitment to excellence, and your respect for others. Your statement of purpose may also include
indications of your efforts to create a learning environment in which all can learn and reference other accomplishments that make you a particularly good
candidate for admission to this program. Remember, your personal statement is a very important part of your application. Proper use of grammar and correct
spelling are expected.”
Each essay is read and scored according to the rubric in Appendix B. The mean scores assigned by the program director for 2010 applicants are
summarized in the table and graph below:
Area
Average Score
Philosophy of Education
2.34
Goals as an Educator
2.43
Commitment to Learning
2.47
Commitment to Excellence
2.21
Respect for Others
2.32
Mechanics
2.66
Overall Impression
2.19
EducationAIMReport2010
Page5
The average scores for all areas were between the “Basic/Proficient” and “Proficient/Distinguished” catergories. Mechanics was slightly higher than
the other average scores. This may be because applicants used extra care proofreading the document knowing that accuracy was important. Some
applicants may have also had other individuals proofread their essay prior to submission.
DispositionRatings
During the first semester of the MS Education program most students take the course EDUC-729 Introduction to Educational Research. At the end
of this course, the instructor rated each student on the rubric found in Appendix C. Students also rated themselves using the same rubric. Scores
were assigned as follows:
1 = Unsatisfactory/Emerging
2 = Basic
3 = Proficient
4 = Distinguished
Not Observed (no rating assigned)
Instructor Ratings
Area
Average Score
Initiative, Preparedness, & Continuous Learning
2.83
Academic Excellence through Critical Reflective Thinking
2.94
Expression of Beliefs, Collaboration and Professional Communication
3.00
Adherence to Ethical and Legal Considerations
3.00
Respect for Diversity
3.33
EducationAIMReport2010
Page6
Student Self-Ratings
Area
Average Score
Initiative, Preparedness, & Continuous Learning
3.25
Academic Excellence through Critical Reflective Thinking
3.34
Expression of Beliefs, Collaboration and Professional Communication
3.48
Adherence to Ethical and Legal Considerations
3.48
Respect for Diversity
3.48
EducationAIMReport2010
Page7
On average, students rated themselves higher than their instructor did across all areas. The average student and instructor ratings were lowest in the
area of Initiative, Preparedness & Continuous Learning. Both instructors and students identified Respect for Diversity as an area of relative strength.
EducationAIMReport2010
Page8
MSE Portfolio Results
Students create an e-portfolio as part of the EDUC-790 Professional Portfolio. Starting Spring 2010, the program director taught the course and
evaluated each portfolio using the rubric included in Appendix B. Students were rated according to the following scale:
1 = Unsatisfactory/Emerging
The individual does not yet appear to have the knowledge or skills underlying the area. Fundamental knowledge or
skills are required. Performance is below expected level for a beginning teacher.
2 = Basic:
Beginning knowledge and skill development is evidenced, often due to limited education or experience in the area.
Some teaching and supervision are required.
3 = Proficient:
The individual exhibits knowledge, proficiency, and competence in most aspects of the area. The individual may
seek consultation, professional development, or assistance in some aspects of the area.
4 = Distinguished:
Advanced knowledge and skill development are evidenced in the area. Individuals with this level of skill and
knowledge can teach, mentor and/or supervise others in this domain area.
Mean scores for each of the five areas are presented in the table below:
EducationAIMReport2010
Page9
Areas (Adapted from National Board for Teaching Standards)
1. Commitment to Learners and Learning
a. Understanding of how students develop and learn, respect for differences and belief that all students
can learn is indicated by evidence of differentiation of instruction to meet the diverse needs of
learners.
b. Evidence of a thorough understanding of the school, family, and community contexts is indicated by
evidence of how contextual factors influence instruction and collaboration with the professional
community.
2. Content Knowledge and Content Specific Pedagogy
a. Content knowledge if indicated by evidence of mastery of the subject(s) taught.
b. Knowledge and application of theories related to pedagogy and learning is indicated by evidence of
the use of a broad range of instructional strategies and an explanation of the choices made in
practice.
c. Pedagogical expertise is indicated by evidence of the use of instructional and adaptive technologies
to improve student learning.
3. Managing and Monitoring Student Learning
a. Thorough understanding of the major concepts and theories related to assessing student learning
indicated by evidence of the use of a range of strategies to assess individual students and group.
b. Thorough understanding of student learning is evidenced by an analysis of student, classroom and
school performance at multiple points.
c. Managing and monitoring student learning is indicated by evidence of the use of assessment data to
make data-driven decisions.
4. Systematic Reflection and Lifelong Learning
a. Systematic reflection is evidenced by continuous examination of own practice.
b. Systematic reflection and learning is evidenced by development and monitoring of professional
development goals.
c. Lifelong learning is evidenced by the application of research, analysis of self performance assessment
data, and reflection to improve practice.
d. Awareness of, and the ability to critically analyze current educational research, issues and policies is
indicated by evidence of how research and policies have affected practice.
5. Collaboration and participation in learning communities
a. Professionalism is indicated by evidence of effective collaboration with families, colleagues, and
communities to improve student learning.
b. Evidence is provided of active membership in professional associations.
EducationAIMReport2010
Spring 2010
N=16
Fall 2010
N=16
3.12
2.63
2.81
2.63
2.94
2.69
2.94
2.94
2.94
2.88
Page10
Portfolio Ratings by Area in 2010
2010 MS Education Portfolio Ratings
4
3.5
3
2.5
2
1.5
1
0.5
0
Collaboration a
Participation in Le
Communitie
Systematic Reflec
Lifelong Learn
Managing and Mo
Student Learn
Content Knowled
Content Specific P
Commitment to L
and Learnin
Page11
EducationAIMReport2010
8.
HowResultsAreSharedwithKeyInstructors
This report, in its entirety, will be presented at program advisory committee meetings. Because the MS Education is an online program, many
key instructors are not available for on-campus meetings. For this reason, the report will also be shared electronically with key instructors
who will be given the opportunity to provide feedback.
9.
PlansforImprovementsBasedonResults
Data will be shared with key instructors and the program advisory committee. Future agenda items will include a discussion of the data and
what meaningful changes can be made to improve the program. Initial review of the data suggests the following items for improvement:
1. The economic situation both nationally and at the state level has significantly affected the market for advanced degrees in education.
To date, the MS Education has maintained steady enrollment despite the economic challenges that teachers and school districts are
facing. One of the primary goals of the program is to find ways to continue to meet the needs of educators who are seeking advanced
degrees. This goal will be accomplished by developing coursework that meets the changing needs of today’s educators. The program
director works with the advisory board and key faculty and staff to design, update, and implement instruction that will have a positive
impact on enrollment.
2. An additional goal is to finalize the implementation of the program revision, new coursework, and new assessment system. Once full
implementation has occurred, it will be important to monitor data for continuous improvement.
3. The MS Education program will participate in the Planning and Review Committee process in 2011 which will provide additional data
and insights into program improvement.
EducationAIMReport2010
Page12
AppendixA:MSEBenchmarks
BenchmarkI:AdmissiontoProgram





Philosophy of Education/Statement of Purpose Review (RUBRIC) Official Transcript GPA 2.75 from previous institution or transcript review Current Resume Recommender ratings (3) BenchmarkII:CompletionofAssessmentforLearning(approx.10credits)



Disposition self‐rating EDUC‐729 GPA 3.0 (transcript review) Disposition review (RUBRIC) BenchmarkIII:Graduation





GPA 3.0 (transcript review) Application for Degree Candidacy “B” or better in EDUC‐735 (old program) or EDUC‐780 Action Research (revised program) Disposition review by observer in the field ePortfolio review (RUBRIC) o Table of Contents organized by 5 Teacher Standards o Includes reflective statements for each area of the Portfolio Assessment Rubric o Includes at least one piece of evidence for each area of the Portfolio Assessment Rubric EducationAIMReport2010
PersonResponsible
MSE Program Director MSE Program Director MSE Program Director MSE Program Director MSE Program Director Instructor of Assessment for Learning and/or Intro to Research MSE Program Director MSE Program Director EDUC‐735 Instructor Identified by candidate EDUC‐790 Instructor or PD Page13
AppendixB:ScoringRubricforMSEProgramApplication
Unsatisfactory
0
No evidence
provided by the
candidate.
Emerging
1
Candidate provides some general
statements about education, but
does not articulate a philosophy
of education.
Basic/Proficient
2
Candidate clearly describes
his/her philosophy of education
but does not make effective use of
examples and/or literature to
support their views.
Goals as an Educator
(NBPTS 1, 4; Domain
4)
No evidence
provided by the
candidate.
Candidate provides a weak
statement of goals that are
marginally appropriate for the
program.
Candidate states goals that are
appropriate to their profession,
but the response lacks depth.
Stated goals are consistent with
the goals of the program.
Commitment to
Learning
(NBPTS 1,4,5;
Domain 4)
Commitment to
Excellence
(NBPTS 1,4,5;
Domain 4)
Respect for others
(NBPTS 1,4,5;
Domain 4)
No evidence
provided by the
candidate.
Candidate provides a weak
description of his/her
commitment to learning.
Candidate provides adequate
examples of his/her commitment
to learning.
No evidence
provided by the
candidate.
Candidate provides a weak
description of his/her
commitment to excellence.
Candidate provides adequate
examples of his/her commitment
to excellence.
No evidence
provided by the
candidate.
Candidate provides a weak
description of his/her respect for
others.
Candidate provides adequate
examples of his/her respect for
others.
Mechanics
Major errors in
spelling and
grammar
throughout the
essay.
Not a viable
candidate for the
MSE program at
UW-Stout
Errors in spelling and grammar
are a significant detriment to the
essay.
A few minor errors in spelling
and/or grammar.
Proficient/Distinguished
3
Candidate clearly and articulately
describes his/her philosophy of
education using powerful examples
and/or literature to support their views.
Response reflects systematic thinking
about their own practice.
Candidate clearly and articulately states
goals appropriate to their profession.
Response indicates insight that is
highly reflective and based on analysis
of their own practice. Stated goals are
consistent with the goals of the
program.
Candidate clearly and articulately
provides solid examples of his/her
belief in and commitment to all
students’ learning.
Candidate clearly and articulately
provides solid examples of his/her
belief in and commitment to
excellence.
Candidate clearly and articulately
provides solid examples of his/her
belief in and commitment to respect for
others.
Proper grammar and spelling evident
throughout the essay.
A weak candidate who may be
considered for admission to
program, dependent on other
factors.
An adequate candidate who
should be admitted to program
An impressive candidate who should
be admitted to program
Philosophy of
Education
(NBPTS 1, 4; Domain
4)
Overall Impression
EducationAIMReport2010
Page14
AppendixC:DispositionsRatingForm
Unsatisfactory/Emerging:
Rarely demonstrates the disposition. The area
needs to be improved before the next
disposition rating.
Dispositions
Unsatisfactory/
Emerging
(U)
Basic:
Occasionally
demonstrates the
disposition, but growth is
needed
Proficient:
Usually demonstrates the
disposition with no areas
of concern.
Basic
(B)
Distinguished:
Demonstrates model
dispositions; the individual
could teach, supervise, or
mentor other teachers in this
area.
Proficient
(P)
Not
Observed:Observation
of this disposition did
not take place, for this
student.
Distinguished
(D)
Rating
U
- Did not participate in class activities.
Initiative, Preparedness
& Continuous Learning
- Often missed assignment deadlines.
- Did not meet minimal expectations
in completing assignments.
- Usually participated in class and
discussion board activities.
- Missed some assignment deadlines.
- Met the basic expectations in completing
assignments.
- Demonstrated timely
- Consistently demonstrated timely
participation and consistent
participation and presence on the
presence on the class discussion class discussion boards.
board.
- Exceeded requirements and met
- Met requirements and met
assignment deadlines.
assignment deadlines.
- Exceeded expectations in assigned
- Met expectations in assigned
work and was consistently well
work, prepared for class as
prepared for class as evidenced by
evidenced by completed
completed readings and
readings and assignments.
assignments.
B
P
D
Not
Observed
- Frequently considered multiple
- Made some effort to consider
perspectives when making
multiple perspectives and
reasoned decisions based upon
determine how written discourse
supporting evidence from
affected others.
previous readings and course
experiences.
- Did not consider student
- Occasionally considered
outcomes, research, and
student outcomes, research, and
- Frequently considered student
data when developing
data when developing
outcomes, research and data
educational materials for use educational materials for use in
when developing educational
in classroom and clinical
classroom and clinical situations.
materials for use in classroom &
situations.
clinical situations.
- Read but didn’t analyze
- Ignored feedback
feedback.
- Analyzed feedback and made
- Written discourse reflected
little or no concern for
multiple perspectives or how
others may be affected.
Academic Excellence through
Critical Reflective Thinking
EducationAIMReport2010
- Consistently considered multiple
perspectives when making reasoned
decisions based upon supporting
evidence from previous readings
and course experiences.
- Consistently considered student
outcomes, research and data when
developing educational materials for
use in classroom & clinical
situations.
- Analyzed feedback and made
appropriate adjustments to enhance
U
B
P
D
Not
Page15
some appropriate adjustments to personal growth and learning.
enhance personal growth and
learning.
Observed
U
Expression of Beliefs,
Collaboration and Professional
Communication
- Demonstrated no evidence
of willingness or ability to
articulate ideas or beliefs in
written communication.
- Showed some evidence of the
ability to articulate ideas or
beliefs in written
communications.
- Written communications
were unprofessional and/or
inappropriate to the audience
and setting.
- Some written communications
were professional and
appropriate to the audience and
setting.
- Conveyed ideas or beliefs with
success in written
communications and
demonstrated sensitivity with
respect to language use.
- Consistently communicated ideas
or beliefs with success in written
communication and demonstrated
sensitivity with respect to language
use.
- Most communications were
written in a professional manner
and appropriate to the audience
and setting.
- All communications were
professionally delivered and
appropriate to the audience and
setting.
B
P
D
Not
Observed
U
- Plagiarized the work of
others.
Adherence to Ethical and
Legal Considerations
- Sometimes cited the works of
others.
- Cited the works of others.
- Always appropriately cited the
works of others.
- Frequently articulated thoughts
or demonstrated behaviors that
- Articulated some thoughts that
- Consistently demonstrated
- Breached established rules
considered the rights of course
considered the rights of course
behaviors that safeguarded the
for protecting the rights of
colleagues and/or PK-12 student
colleagues and/or PK-12 student
rights of course colleagues and/or
course colleagues and/or PKlearners.
learners.
PK-12 student learners).
12 student learners.
-Usually exhibited behaviors
- Exhibited some behaviors
- Modeled adherence to the
- Violated ethical/legal
demonstrating adherence to
demonstrating legal/ethical
ethical/legal standards of the
standards of the profession.
ethical/legal standards of the
standards of the profession.
profession.
profession.
B
P
D
Not
Observed
Respect for Diversity
- Showed little or no
acknowledgement of the
belief that all students can
learn during class
discussions.
- Showed some
acknowledgement and
responsiveness to the belief that
all students can learn during
class discussions.
- Showed little or no interest
in learning about educational
strategies to address the
needs of diverse learners.
- Demonstrated some interest in
learning about educational
strategies to address the needs
of diverse learners.
EducationAIMReport2010
- Frequently considered and
demonstrated the belief that all
students can learn in class
discussions or discussion board
communication.
- Sought out information on how
to address the needs of diverse
learners.
- Consistently considered and
demonstrated the belief that all
students can learn during class
discussions by providing solutions,
URL's for web resources or other
resources.
- Demonstrated ways to address the
needs of diverse learners in
educational settings.
U
B
P
Page16
D
Not
EducationAIMReport2010
Page17
AppendixD:MSERubricforPortfolio
Portfolio Assessment Rubric
Student Name: _______________________________________
Reviewer: ___________________________________
Unsatisfactory/Emerging:
Basic:
Proficient:
Distinguished:
Student ID #: ___________________ _____
Date: __________________
Reviewer Signature: ______________________________________________________
The individual does not yet appear to have the knowledge or skills underlying the area. Fundamental knowledge or skills are required. Performance is below expected level for a beginning teacher.
Beginning knowledge and skill development is evidenced, often due to limited education or experience in the area. Some teaching and supervision are required.
The individual exhibits knowledge, proficiency, and competence in most aspects of the area. The individual may seek consultation, professional development, or assistance in some aspects of the
area.
Advanced knowledge and skill development are evidenced in the area. Individuals with this level of skill and knowledge can teach, mentor and/or supervise others in this domain area.
Areas*
Unsatisfactory
1
Basic
2
Proficient
3
Distinguished
4
1. Commitment to Learners and Learning
c. Understanding of how students develop and learn, respect for differences
and belief that all students can learn is indicated by evidence of
differentiation of instruction to meet the diverse needs of learners.
d. Evidence of a thorough understanding of the school, family, and community
contexts is indicated by evidence of how contextual factors influence
instruction and collaboration with the professional community.
2. Content Knowledge and Content Specific Pedagogy
d. Content knowledge if indicated by evidence of mastery of the subject(s)
taught.
e. Knowledge and application of theories related to pedagogy and learning is
indicated by evidence of the use of a broad range of instructional strategies
and an explanation of the choices made in practice.
f. Pedagogical expertise is indicated by evidence of the use of instructional
and adaptive technologies to improve student learning.
3. Managing and Monitoring Student Learning
a. Thorough understanding of the major concepts and theories related to
assessing student learning indicated by evidence of the use of a range of
strategies to assess individual students and group.
b. Thorough understanding of student learning is evidenced by an analysis of
student, classroom and school performance at multiple points.
c. Managing and monitoring student learning is indicated by evidence of the
use of assessment data to make data-driven decisions.
4. Systematic Reflection and Lifelong Learning.
a. Systematic reflection is evidenced by continuous examination of own
EducationAIMReport2010
Page18
b.
c.
d.
practice.
Systematic reflection and learning is evidenced by development and
monitoring of professional development goals.
Lifelong learning is evidenced by the application of research , analysis of self
performance assessment data, and reflection to improve practice.
Awareness of, and the ability to critically analyze current educational
research, issues and policies is indicated by evidence of how research and
policies have affected practice.
5. Collaboration and participation in learning communities
a. Professionalism is indicated by evidence of effective collaboration with
families, colleagues, and communities to improve student learning.
b. Evidence is provided of active membership in professional associations.
*Note: Areas 1-5 from National Board for Teaching Standards
EducationAIMReport2010
Page19