Western Illinois University: Department of Special Education
Masters in Science in Education
Assessment Report: 2009-‐2010
Learning Outcomes
A.
Graduate Special Education candidates will demonstrate their content knowledge of special education.
B.
Graduate special education candidates will demonstrate their ability to effectively plan instruction.
C.
Graduate special education candidates will demonstrate their ability to successfully implement instruction.
D.
Graduate special education candidates will demonstrate their ability to effect student learning.
Direct Measures
Special Education 581 Language Assessment
Curriculum Planning Project from SPED 536, SPED 552, or SPED 553
Data Driven Instruction project from SPED 536, SPED 552, or SPED 553
Special Education 624 Action Research Project
Assessment Results
The Council for Exceptional Children (CEC), the professional association that governs accreditation for special education programs, and the Illinois State Board of Education (ISBE) have established a framework for all accredited special education programs to utilize as they collect, aggregate, and report data. Faculty continue to evaluate graduate student performance using direct measures of knowledge and skills related to learning outcomes identified for all graduate students. Assessment results contribute to a set of data used during the 2009-‐2010 academic year to evaluate the effectiveness of the special education program and the extent to which candidates meet state and national standards.
Outcome A: Graduate special education candidates will demonstrate their knowledge of
special education.
Assessment: Language Project (SPED 581)
The ability to use language and speech may impact individuals with disabilities across the life span. Because language based issues are so basic to appropriate instruction, the second content assessment in SPED 581, Special Education Law, is a project that requires students to, in collaborative groups, identify evidence based best practice across the five major areas of CEC
Standard 6 (language). Students research and then present recommendations regarding best practices in promoting language development generally as well as for students who are English
Language Learners, service delivery choices (push in /pull out), legal aspects of language services, best practices for English language learners (ELL), and in Assistive Technology (AT).
Students collaborate face to face and using WIKI board technology and present results in class.
7
Standard Component
6 Language
Semester
Language Project
Spring 2010
#
23
4 3
13 (56%) 10 (44%)
# of
Candidates
2
0
4 = proficient, 3 = proficient with assistance, 2 = novice, 1 = no skill
Data Analysis
One hundred percent of special education graduate candidates demonstrated the ability to review and synthesize evidence-‐based practice, collaborate, and evaluate best practices in language instruction used in a case study school. Over 50% of the candidates earned
“proficient” or “proficient with assistance” ratings for the Council for Exceptional Children (CEC)
Standard 6 (Language).
Outcome B: Graduate special education candidates will demonstrate their ability to effectively plan instruction.
Assessment: Curriculum Planning Project (SPED 536, SPED 552, SPED 553)
All special education graduate students complete a Curriculum Planning Project in methods
courses. All candidates are required to take methods classes but they are not required to take the same courses because their undergraduate backgrounds and certification goals differ. All methods courses require the Curriculum Planning Project, which is evaluated using a common rubric.
Curriculum Planning Project
Standard Component Semester Course 1 2 3
1
0
Planning Fall 2009
SPED 536
SPED 552
19
14
4
3 (16%) 16 (84%)
1(7%) 2(14%) 7(50%) 4(29%)
Spring 2010 SPED 553 9
1 = not proficient, 2 = novice, 3 = proficient with assistance, 4 = proficient
Data Analysis
During Fall 2009, 79% of the candidates were ranked as “proficient” or “proficient with assistance.” Three candidates did not meet expectations for their ability to plan effective instruction for students with special needs. During Spring 2010, all candidates were ranked at
“proficient” or “”proficient with assistance” levels for CEC Standard 7.
Outcome C: Graduate special education candidates will demonstrate their ability to successfully implement instruction.
Assessment: Data Driven Instruction Project in SPED 536, SPED 552, and SPED 553
Data Driven Instruction
Standard
3
Standard
4
Comp
Indiv Learn Dif
Instructional
Strategies
Semester
Fall
09
Spring 10
Fall
09
Spring 10
Course
SPED 536
SPED 552
SPED 553
SPED 536
SPED 552
SPED 553
#of candidates
19
14
9
19
14
9
1
1(5%)
2(14%)
1(5%)
2(14%)
2
3
4
18(95%)
10(72%) 2(14%)
2(22%) 7(79%)
2(11%) 16(84%)
10(72%) 2(14%)
2(22%) 2(22%) 5(56%)
Standard
5
Standard
7
Learning
Environment
Planning
Fall
09
Spring 10
Fall
09
Spring 10
SPED 536
SPED 552
SPED 553
SPED 536
SPED 552
SPED 553
19
14
9
19
14
9
2(11%)
2(14%)
1(5%)
2(14%)
17(89%)
10(72%) 2(14%)
1(11%) 3(33%) 5(56%)
2(11%)
16(84%)
10(72%) 2(14%)
Standard
8
Standard
9
Assess
Prof ethics
Fall
09
Spring 10
Fall
09
Spring 10
Fall
09
SPED 536
SPED 552
SPED 553
SPED 536
SPED 552
SPED 553
SPED 536
19
14
9
19
14
9
19
1(5%)
2(14%)
1(5%)
2(14%)
2(22%)
2(11%)
1(11%)
3(16%) 15(79%)
10(72%) 2(14%)
7(78%)
16(84%)
10(72%) 2(14%)
8(89%)
12(100%)
Standard
10
Collaboration
Spring 10
SPED 552
SPED 553
14
9
2(14%)
10(72%) 2(14%)
2(22%) 7(79%)
1 = not proficient, 2 = novice, 3 = proficient with assistance, 4 = proficient
Data Analysis
The Data Driven Instruction Project is completed in each methods course offered in the program. Both the assignment and rubric are the same across all courses. Seven of the CEC standards are assessed through this project. In Fall 2009, 78-‐95% of candidates in SPED 536 and 86% of candidates in SPED 552 were rated “proficient with assistance” or “proficient” across all standards. Seventy-‐eight to 100% of candidates in SPED 553 in Spring 2010 were successful at the “proficient” or “proficient with assistance” levels. Overall, the weakest performances as measured by the number of candidates who earned scores of “no skill” and/or
“novice” were evident in the performance of Standard 5 (Learning Environment) and Standard
9 (Professional Ethics). The fewest “no skill” or “novice” ratings were earned for Standard 10
(Collaboration). The greatest number of “proficient” ratings was earned for performances related to Standard 3 (Individual Learning Differences) and Standard 9 (Professional Ethics).
Outcome D: Graduate special education candidates will demonstrate their ability to affect student learning.
Assessment: Action Research Project (SPED 622, SPED 624)
The Action Research course sequence, Action Research in Education (SPED 622) and Action
Research Project (SPED 624), is designed to provide special education graduate candidates with the skills and knowledge necessary to become agents of change in their own classrooms.
Candidates learn the benefits and limitations of various research designs and to effectively use action research. To that end, each candidate must design and implement an action research project that will demonstrate his/her ability to impact the learning and/or behavior of his/her students. Candidates must complete the Institutional Review Board (IRB) training, obtain an
IRB clearance, and function within both institutional and professional ethical standards.
The Action Research Project (SPED 624) is taught only during the spring semester, following
Action Research in Education (SPED 622), which is scheduled each fall semester.
Comp
Standard 4
Strategies
Semester
Spring 2010
Action Research Project
4 3
10 (75%) 1 (17%)
2
1 (8%)
1
Standard 5
Learning
Environment
Standard 7
Instructional
Planning
Standard 8
Assessment
Spring 2010
Spring 2010
Spring 2010
10 (75%)
8 (67%)
11 (92%)
1 (17%)
4 (33%)
1 (8%)
1 (8%)
4 = proficient, 3 = proficient with assistance, 2 = novice, 1 = not proficient
Data Analysis
Four of the Council for Exceptional Children Standards are assessed through the Action
Research project, which is the capstone project for the program. Twelve (100%) candidates earned “proficient” or “proficient with assistance” ratings on competencies related to Standard
7 (Instructional Planning) and Standard 8 (Assessment). One candidate earned “novice” ratings for Standard 4 (Instructional Strategies) and Standard 5 (Learning Environment). The greatest number of “proficient” ratings was earned for performances related to Standard 8 (Assessment) and the fewest for Standard 7 (Instructional Planning).
Summary of Data
Overall, at least 78% of candidates were rated as “proficient” or “proficient with assistance” across eight CEC standards. The greatest number of “proficient” ratings was earned for
Standard 3 (Individual Learning Differences). The greatest number of “no skill” ratings was
earned for Standard 5 (Learning Environment).
Plan of Action
Assessments indicate that most candidates are successful in demonstrating the crucial activities of planning and implementing instruction, monitoring student progress, and affecting student learning. The effectiveness of the revisions of the capstone sequence, which were
implemented in 2009-‐2010, will continue to be monitored.
In order to meet unit requirements for NCATE, the program will re-‐examine the extent to which candidates’ content knowledge is assessed. Faculty will determine whether the assessment of
Standard 9 (Professional Ethics) adequately addresses the requirement for the assessment of candidates’ dispositions. NCATE requires assessment data that provides evidence of candidates’ impact on student learning. Faculty will identify the specific standards and competencies that collectively provide this evidence as measured through the capstone action research project in SPED 624. To provide specific evidence that can be used to more effectively monitor candidate performance and program effectiveness, rubrics will be revised to include
more precise descriptions of each competency and each level of performance.
In the “Continual Improvement” model required by NCATE, all programs in the teacher education unit must move towards meeting the criteria for “target” performance on at least
one standard. Faculty will examine curriculum, course requirements, assessments, and assessment data to insure that the special education graduate program is moving towards
“target” performance for NCATE Standard 1 (Knowledge, Skills, and Dispositions).
Western Illinois University: Department of Special Education
Masters in Science in Education
Assessment Report: 2008-‐2009
Learning Outcomes
A. Graduate Special Education candidates will demonstrate their content knowledge of
special education.
B. Graduate special education candidates will demonstrate their ability to effectively plan
instruction.
C. Graduate special education candidates will demonstrate their ability to successfully
implement instruction.
D. Graduate special education candidates will demonstrate their ability to effect student
learning.
E. Graduate special education candidates will demonstrate their ability to use assessment
while supervising paraprofessionals.
Direct Measures
Curriculum Planning Project from SPED 523, SPED 526, SPED 536, SPED 546, SPED 552, or
SPED 553
Data Driven Instruction project from SPED 523, SPED 526, SPED 536, SPED 546, SPED 552, or
SPED 553
Special Education 624 Action Research Project
Special Education 514 Para-‐educator Assessment
Special Education 581 Language Assessment
Results
Ongoing review of graduate assignments and policies in 2008-‐2009 led to a revision of the
Action Research protocol, which will assist students by more clearly delineating timelines for that capstone project. The information was shared with students through announcements in
courses and in the placement of the new policies on the SPED website.
Outcome A: Graduate special education candidates will demonstrate their content
knowledge of special education
Assessment: Language Project (SPED 581)
The ability to use language and speech may impact individuals with disabilities across the life span. Because language based issues are so basic to appropriate instruction, the second content assessment in SPED 581, Special Education Law, is a project that requires students to, in collaborative groups, identify evidence based best practice across the five major areas of CEC
Standard 6 (language). Students research and then present recommendations regarding best practices in language development, service delivery choices (push in /pull out), legal aspects of language services, best practices for English language learners (ELL), and in Assistive Technology
(AT). Students collaborate face to face and using WIKI board technology and present results in
class.
Standard Component Semester
6 Language
Language Project
Spring 2009
Definitions found in the appendix.
#
8
4
50 (4)
3
50 (4)
4 = proficient, 3 = proficient with assistance, 2 = novice, 1 = no skill
2
1
Analysis: Special education graduate candidates demonstrate the ability to review and synthesize evidence based practice, collaborate, and to evaluate best practice in language for a
fictional school district. No change is indicated.
Outcome B: Graduate special education candidates will demonstrate their ability to effectively plan instruction.
Assessment: Curriculum Planning Project (SPED 523, SPED 526, SPED 536, SPED 546, SPED 552,
SPED 553)
All special education graduate students complete a Curriculum Planning Project in methods classes. All students are required to take methods classes but all are not required to take the
same courses because their undergraduate backgrounds and certification goals differ. All methods courses require the Curriculum Planning Project, which is always evaluated using the
same rubric.
Curriculum Planning Project
Standard Component Semester # of
Candidates
1
7
Planning
Fall 2008
Spring 2009
21
23
1
2
0
3 4
28.5 (6) 71.5 (15)
0 5.2 (1) 47.8 (11) 47.8 (11)
1 = not proficient, 2 = novice, 3 = proficient with assistance, 4 = proficient
Definitions found in the appendix.
Analysis: During the fall, all students were ranked at proficient or proficient with assistance levels. During the spring, all but one student was ranked as proficient or proficient with assistance. It is important to remember that although all SPED graduate majors hold teaching certificates, all do not have teaching experience or experience teaching special populations. So, the assessment of the ability to plan is critical.
Action: None needed at this time.
Outcome C: Graduate special education candidates will demonstrate their ability to
successfully implement instruction.
Assessments: Data Driven Instruction Project in SPED 523, SPED 526, SPED 536, SPED 546, SPED
552, or SPED 553
Assessment: Candidate Ability to Implement Instruction
Data Driven Instruction Project
Fall 2008
Spring 2009
# of Candidates
21
23
1
5% (1)
0
2
0
0
3
9% (2) 86% (18)
39% (9)
1 = not proficient, 2 = novice, 3 = proficient with assistance, 4 = proficient
Data Driven Instruction
4
61% (14)
Comp Semester #1 4 3 2 1
Indiv Learn
Dif
Instructional
Strategies
Fall 10 80 (10) 10 (1) -‐ 10 (1)
Standard 3
Standard 4
Standard 5
Standard 7
Standard 8
Learning
Environment
Planning
Assess
Standard 9 Prof ethics
Standard 10 Collaboration
Spring
Fall
Spring
Fall
Spring
Fall
Spring
Fall
Spring
Fall
Spring
Fall
Spring
23
10
23
10
23
10
23
10
23
23
10
23
10
73.9 (17) 17.3 (4) 3.1 (1)
80 (10) 10 (1)
65.2 (15) 26 (6)
80 (10) 10 (1)
-‐
8 (2)
-‐
65 (15) 30 (7)
80 (10) 10 (1)
69.5 (16) 30.4 (7)
80 (10) 10 (1)
4 (1)
-‐
-‐
-‐
52.3(12) 30.4 (7) 17.3 (4)
80 (10) 10 (1) -‐
73.9 (17) 17 (4) 8.6 (2)
80 (10) 10 (1) -‐
78.2 (18) 13 (3) 8.6 (2)
-‐
10 (1)
-‐
10 (1)
10 (1)
-‐
10 (1)
-‐
10 (1)
-‐
10 (1)
4 = proficient, 3 = proficient with assistance, 2 = novice, 1 = not proficient
Definitions found in the appendix.
Analysis:
The Data Driven Instruction project that is required for each Special Education graduate student is one element in each methods course. Both the assignment and rubric are the same across
classes. During 2008-‐2009, only a single student failed to demonstrate the ability to successfully use data in instructional decision making. 95% of all special education candidates in graduate methods classes were successful at the proficient or proficient with assistance levels. No action is indicated.
Outcome D: Graduate special education candidates will demonstrate their ability to effect student learning.
Assessment: Action Research Project (SPED 622, SPED 624)
The Action Research course sequence, Action Research in Education (SPED 622) and Action
Research Project (SPED 624), is designed to provide special education graduate students with the skills and knowledge necessary to become agents of change in their own classrooms.
Students learn the benefits and limitations of various research designs and to effectively use action research. To that end, each student must design and implement an action research project that will demonstrate his/her ability to impact the learning and/or behavior of his/her students. Students must complete the Institutional Review Board (IRB) training, obtain an IRB
clearance, and function within both institutional and professional ethical standards.
The Action Research Project (SPED 624) is only taught each spring, following Action Research in
Education, which is scheduled each fall semester.
Comp
Standard 4
Strategies
Semester
Spring
Action Research Project
4 3
80 (4) 20 (1)
2
1
Standard 5
Learning
Environment
Standard 7
Instructional
Planning
Standard 8
Assessment
Spring
Spring
Spring
80 (4)
80 (4)
80 (4)
20 (1)
20 (1)
20 (1)
4 = proficient, 3 = proficient with assistance, 2 = novice, 1 = not proficient
Definitions found in the appendix.
Seventeen students were cleared for the Action Research Project in Spring 2009. Five of those capstone projects were completed. Ten additional projects are likely to be completed during the summer. Although the department would prefer for students to complete the Action
Research Project during our academic calendar, the requirements of the project and the
specific challenges in the classrooms in which our students teach frequently makes completion within our academic timelines difficult.
Action: None needed at this time. The change in 622/624 policy, which is posted on the SPED website, has been implemented for a single academic year. The effectiveness of that policy
may need to be reviewed in 2010-‐2011.
Outcome E: Graduate special education candidates will demonstrate their ability to use assessment while supervising paraeducators.
Assessment: SPED 514 Paraeducator Assessment
The Paraeducator Project is designed to teach and provide practice in determining both a clear and appropriate job description for paraeducators in a specific setting and to provide a fair and consistent method of evaluating paraprofessions in that setting.
Comp
Standard 8
Semester
Summer
Spring
#
11
12
Paraeducator Assessment
4
82 (9)
33 (4)
3
9 (1)
50 (6)
2
-‐
-‐
1
9 (1)
16 (2)
Standard 9 Summer
Spring
Standard 10 Summer
11
12
11
100 (11)
33 (4)
100 (11)
50 (6)
Spring 12 33 (4) 50 (6)
4 = proficient, 3 = proficient with assistance, 2 = novice, 1 = not proficient
-‐
-‐
-‐
-‐
16 (2)
16 (2)
Assessment Ratings: Definitions are in the Appendix.
Analysis: Overall, graduate special education students performed successfully on the
Paraprofessional Assessment. There is no clear indicator that action is needed.
Summary of Actions:
Assessments indicate that candidates are successful in demonstrating the crucial activities of
planning and implementing instruction, monitoring student progress, and effecting student learning. While assessments indicate no need for change, ongoing review of assessments and
graduate policies are in the best interest of the department and the candidates we serve.
Department of Special Education
2007-‐2008 Annual Assessment Report
Masters of Science in Education Degree
I. Learning Outcomes
A. Graduate special education candidates will demonstrate their ability to effectively plan instruction.
B. Graduate special education candidates will demonstrate their ability to successfully
Implement instruction.
C. Graduate special education candidates will demonstrate their ability to effect student
learning.
D. Graduate special education candidates will demonstrate their ability to use assessment
while supervising paraeducators.
Direct Measurement
Curriculum Planning Project from SPED 523, SPED 526, SPED 536, SPED 546, SPED 552, or SPED 553
Data Driven Instruction Project from SPED 523, SPED 526, SPED 536, SPED 546, SPED
552, or SPED 553
Special Education 624 Action Research Project
Special Education 514 Paraeducator Assessment
Outcome A: Graduate special education candidates will demonstrate their ability to effectively plan instruction.
Assessment: Curriculum Planning Project in SPED 523, SPED 526, SPED 536, SPED 546, SPED 552, or SPED 553
Assessment Ratings (complete rubric available in Appendix A):
1 – Not proficient
2 – Novice
3 – Proficient with assistance
4 – Proficient
Curriculum Planning Project
Fa 2006 Semester # of Candidates
15
1
0
2
7%
3
13%
4
80%
Sp 2007 Semester # of Candidates 1 2 3 4
SPED 536
9
Fa 2007 Semester # of Candidates
SPED 552 19
19
Sp 2008 Semester
SPED 533
SPED 553
# of Candidates
8
13*
0
1
0
0
1
0
0
* 3 students had an incomplete at the time of this report
0
2
2 (11%)
2 (11%)
2
0
3 (23%)
0
3
4 (21%)
6 (32%)
3
3 (37%)
3 (23%)
100%
4
13 (68%)
11 (58%)
4
5 (63%)
4 (23%)
Analysis of Data: Outcome A
During FY06, graduate special education faculty revised the rubric for the curriculum planning project to assess CEC/ISBE requirements for program review. The department began implementation of this new assessment during the Fall 2006. The collected data indicate that during FY08, almost all of the graduate special education candidates were able to plan instruction at the proficient or proficient with assistance levels. The exceptions were two students who performed at the “novice” level in SPED 552 and 536.
Outcome B: Graduate special education candidates will demonstrate their ability to
successfully implement instruction.
Assessments: Data Driven Instruction Project in SPED 523, SPED 526, SPED 536, SPED 546, SPED 552, or SPED 553
Assessment Ratings (complete rubric available in Appendix A):
1 – Not proficient
2 – Novice
3 – Proficient with assistance
4 – Proficient
Data Driven Instruction Project
Fa 2006 Semester # of Candidates
21
1
0
2
0
3
14%
4
86%
Fa 2007 Semester # of Candidates
SPED 552 19
SPED 536 19
1
0
0
Sp 2008 Semester # of Candidates
SPED 523 8
1
0
SPED 553 13* 0
3 students had an incomplete at the time of this report
2
0
0
2
0
3 (23%)
3
4 (21%)
6 (32%)
3
0
3 (23%)
4
15 (79%)
13 (68%)
4
8 (100%)
4 (31%)
Analysis of Data: Outcome B
During FY06, graduate special education faculty revised the rubric for the data driven instruction project to assess CEC/ISBE requirements for program review. The department began implementation of this new assessment in the Fall 2006 semester. The collected data indicate that during FY08, all of the graduate special education candidates were able to successfully implement instruction at the proficient or proficient with assistance levels
Outcome C: Graduate special education candidates will demonstrate their ability to effect
student learning.
Assessment : Action Research Project in SPED 624
Assessment Ratings (complete rubric available in Appendix A):
1 – Not proficient
2 – Novice
3 – Proficient with assistance
4 – Proficient
Action Research Project
Sp 2007 Semester # of Candidates
9
Sp 2008 Semester # of Candidates
SPED 624 6
1
0
1
0
2
0
2
0
3
0
3
1 (17%)
4
100%
4
5 (83%)
Analysis of Data: Outcome C
During FY06, graduate special education faculty revised the rubric for the action research project to assess CEC/ISBE requirements for program review. The department began implementation of this new assessment during the Fall 2006. The collected data indicate that during both the Spring 2007 and the Spring 2008 semester, 100% of the special education candidates were able to demonstrate their ability to effect student learning at the proficient or
proficient with assistance levels
Outcome D: Graduate special education candidates will demonstrate their ability to use assessment while supervising paraeducators.
Assessment : SPED 514 Paraeducator Assessment
Assessment Ratings (complete rubric available in Appendix A):
1 – Not proficient
2 – Novice
3 – Proficient with assistance
4 – Proficient
Paraeducator Assessment
Su2007 # of Candidates
23
Su2008 # of Candidates
1
0
1
2
4%
2
3
26%
3
4
70%
4
SPED 514 0
Note: SPED 514 is being taught during the June/July 2008 Summer Session.
Analysis of Data: Outcome D
During FY06, graduate special education faculty revised the rubric for the paraeducator assessment project to assess CEC/ISBE requirements for program review. The department began implementation of this new assessment during FY07. The collected data indicate that during the May Term, 2007, 96 percent of the graduate special education candidates were able to successfully develop a paraeducator assessment at the proficient or proficient with
assistance levels. Summer 2008 data will not be available until August 2008.
D. Feedback
1. Changes planned in reaction to 2007-‐2008 assessment data. a. The assessment system will be reviewed at the August 2008 faculty meeting to ensure that all data will continue to be collected and analyzed in a timely fashion. b. The assessment developed to reflect the CEC/ISBE standard related to language development will be reviewed.
2. Changes implemented from 2005-‐2006 assessment data.
a. Evaluation of all graduate candidate projects/assignments utilized newly revised rubrics aligned to CEC/ISBE standards.