School of Education
Unit Assessment Report
Advanced and Other Programs
October 1, 2007
Submitted By
Juli Hastings Taylor
Assessment Coordinator
Table of Contents
Introduction ......................................................................................................................................2
Graduate Programs:
MS Guidance and Counseling .................................................................. 3 and attached separately
EdS School Psychology .......................................................................... 10 and attached separately
MS & EdS in Career and Technical Education .................................................. attached separately
MS in Industrial/Technology Education ............................................................. attached separately
MS in Education ................................................................................................. attached separately
Other Programs:
BS in Career, Technical Education and Training ............................................... attached separately
1
School of Education Unit Assessment Report
Advanced and Other Programs
October 2007
Introduction
This report is a summary of the University of Wisconsin-Stout School of Education (SOE)
advanced program assessment data gathered from the fall semester 2003 through December
2006. In the School of Education, data is gathered from several sources to inform unit and
program decisions. Data from this report will be used to develop unit and program goals, inform
curriculum changes, and enhance course delivery in order to improve candidate learning. This
report contains data from a variety of sources.
Graduate programs within the School of Education include School Counseling, School
Psychology, Education, Industrial/Technical Education and Career and Technical Education.
Graduate students in Education and Industrial/Technical Education who want to gain an initial
teacher education license must meet all the requirements of PI34 including PRAXIS I: PreProfessional Skills Test and PRAXIS II: Content Test and student teach at the undergraduate
level. Thus their student teaching ratings would be included in the undergraduate student
teaching report as all student teaching final evaluations use the Danielson framework of
domains/components and the 10 Wisconsin Teacher Standards at UW-Stout.
Certification-only students who already have an initial teaching certification can add on
certification by meeting PI34 requirements, passing PRAXIS II: Content Test and student
teaching. For example, a number of teachers seek certification in Special Education as an add-on
certification to their initial teacher certification. Those students who student taught Fall 2004 or
3rd quarter were included in the student teaching report.
Program Specific Reports
Program specific reports attached to this summary provide data and narrative descriptions of
Graduate Follow-up Surveys and other sources which aid program directors in making program
decisions. The program specific reports also describe how this assessment data is used to
improve the program, program curriculum, and delivery of courses. In addition, program
directors identify and describe program goals for the upcoming year.
Assessment Data Uses
The unit and program assessment reports are shared with School of Education and individual
program advisory committees. Advisory committee members discuss trends and make
recommendations for improvement to program directors and the SOE Dean. The Dean and
Assessment Coordinator meet each semester with individual program directors to discuss
program data, yearly goals, and progress toward achieving short-term and long-term goals. SOE
unit and program goals are in alignment with University goals and priorities. This year, a
comprehensive analysis will also be conducted to ensure SOE unit and program goals align with
external standards developed by certification and accrediting agencies.
2
MS Guidance and Counseling
The M.S. Guidance and Counseling program at UW-Stout prepares graduate students for
assuming the important position of the school counselor. Because the program has been reviewed
and designated as fully approved by the Wisconsin Department of Public Instruction (most recent
review 11/2004), it provides extensive classroom-based and experiential instruction, along with
continuous assessment of student progress, in accordance with Wisconsin’s Educator Licensing
Statutes (PI 34).
Assessment of student progress occurs at various transition points. It should be noted that
currently the Praxis Content area exam is not yet in place for licensure of school counselors in
Wisconsin. State implementation of the Praxis exam is anticipated in fall 2008. Assessment
methods this year have included the following; detailed descriptions of each follow the list:
•
•
•
•
•
Performance-based assessment system: Benchmarks I-IV: Student progress is assessed at
four critical points in each student’s program. An array of measures, including transcript
review, dispositions ratings, portfolio review, and performance rubrics, is used in the
process.
o Faculty review/selection of program applicants
o Student dispositions review
o Student portfolio review
o Criterion-referenced performance rubric for experiential component of the
program
Additional faculty review of student performance
o Transcript review
o Biannual program faculty review
One and five-year follow-up study of graduates
Employers’ follow-up study
Qualitative Review of Program Performance Standards
Faculty Selection of Program Applicants: Benchmark I
Currently, through fall 2006, all student applicants to the program submit the Graduate College
application, transcripts, and three letters of recommendation. From the pool of initial applicants,
approximately 10-20 more candidates than the number of openings are invited to interview for a
position in the program. Average folio rating scores (a minimum cut off score is established)
determine which applicants are selected for an interview.
Applicants selected for the interview meet with a faculty team of 2-3 faculty and respond to a
standard set of questions in the following areas: summary of background and experiences;
experience working with children, youth and individuals of diverse (multicultural) background;
candidates understanding and view of the school counselors roles and duties; candidate’s
understanding of the impact of current issues on education and child well-being.
The review committee reviews and rates all applicants’ written applications and interview
responses in the following areas: ability to do graduate work (based on GPA and courses
completed); related undergraduate/graduate preparation; related experience with youth; content
and quality of responses to essay questions; knowledge of the role and function of school
3
counselors; and letters of recommendation. Committee members do not consult with each other
during this blind review process. Combining the folio ratings and the interview ratings yields a
ranking of all applicants. Final selection of admitted candidates is based on this ranked list.
The 2007 program revision includes an admissions policy change. A standardized measure of
professional-level skills in reading, writing, and math has been added to application requirements
beginning in spring 2008. Applicants may furnish scores from either the PPST (Teacher Praxis I
exam) or the Graduate Records Exam to fulfill this requirement.
Student Dispositions Review: Benchmarks II, III
As part of the Benchmark System for assessing student progress, student dispositions are
reviewed for each student after s/he has earned 24 credits and again during the semester prior to
practicum placement. The rubric for this review is attached in Appendix B. The eight identified
dispositions areas (attendance, preparedness, continuous learning, positive climate, reflectivity,
thoughtful/responsive listener, cooperativeness/collaboration, and respect) are consistent with
those required of all School of Education students in order to meet state licensing requirements
under PI 34.
Portfolio Assessment: Benchmarks II, IV
Students establish components of a portfolio in three School Counseling courses (Career
Development; Career Occupations and Transitions, Practicum). Students currently complete the
portfolio over the course of three+ semesters, including in the portfolio all 12 DPI established
Content Guidelines of competence (See Appendix C for portfolio review rubric.) The instructor
of the courses evaluates each of the portfolios. The university practicum supervisor evaluates the
completed portfolio near the end of practicum, immediately prior to graduation (Benchmark IV).
Results are shared in monthly program faculty meetings and disseminated to the Program
Advisory Committee each semester.
Benchmark IV: Criterion-referenced performance rubric for capstone experiential program
component
The 600-hour practicum follows core coursework and serves as the capstone experience for
graduate students. Practicum site supervisors are selected in accordance with DPI requirements
as outlined on page 19 of the Handbook of Standards, Procedures, and Policies for the Approval
of Professional Education Programs in the State of Wisconsin, under “Clinical Program
Requirements,” Practicum. Site supervisors providing supervision to UW-Stout Guidance and
Counseling students must meet the following criteria:
•
•
•
•
Hold a Wisconsin license in School Counseling
Have at least 3 years of clinical experience as a guidance counselor
Have at least one year of employment as a guidance counselor in their current
employment placement
Have completed education and training in clinical supervision of students and the
Wisconsin Standards for guidance counselors and pupil services personnel.
4
•
o Graduates of UW-Stout’s Guidance and Counseling program meet this
requirement via a required course entitled “Organization and Administration of
Guidance Programs” (SCOUN 765, 3 credits)
Site supervisors for UW-Stout practicum must sign a “verification form” that they have
met all of the above requirements
A list of verified site supervisors is kept in the School of Education under the care of the
Practicum Placement Coordinator.
The site supervisor-counselors who supervise our students during their capstone course complete
a criterion-referenced performance based rubric at the mid-point and completion of the
experience. This performance rubric changed significantly between spring 2006 and fall 2006, so
unit-level scores on the two are not combined. The supervising university faculty member also
completes the rubric in consultation with the cooperating counselor and other supervising faculty,
thus assigning a final grade. The student completes the rubric as a self-assessment and the faculty
member discusses any discrepancies with the student.
Additional Faculty Review of Student Performance
In addition to systematic review of student progress at the benchmark points, the program
director and program faculty monitor student learning through the following:
Transcript Review
Student transcripts are reviewed by the program director at the completion of each semester.
Students who fall below the minimum GPA of 3.0 in a program core course and/or fail any
course are informed of the deficiency and the necessity for meeting with the program director to
develop a remediation plan.
Two critical pre-practicum counseling skill development courses are required in the Guidance
and Counseling program. Students must successfully (3.0 GPA minimum) complete the
following courses before being admitted to practicum, as noted in transcript:
COUN 788 Counseling Process Laboratory. Guidance and Counseling students are
required to complete 25 video taped sessions with a diverse group of clients. Video taped
sessions are reviewed and written assessment provided to the student. Students must have
clients featuring the following demographic variables:
• At least one client age older than 50 years
• At least 3 clients aged 13 or under, and
• At least two clients belonging to an American minority group
SCOUN 705 Play Therapy. Guidance and Counseling students are required to meet with
2-3 children for a minimum of nine sessions
• Child clients are typically under the age of 12 years
• All sessions are videotaped. Video taped sessions are reviewed
with the instructor or graduate assistant and the student receives
written feedback on his/her performance.
5
In addition, a third class includes a service learning component. SCOUN 738, Guidance in the
Elementary Schools requires students to complete 15 service learning hours in an elementary or
middle school tutoring children.
Biannual Program Faculty Review
School Counseling faculty meet minimally twice-per-year to review student progress in the
major. Students may be reviewed more frequently if requested by any faculty member. Students
sign an acknowledgement of this “Student Review, Retention, and Dismissal” Policy upon
admission into the program. During this biannual review, benchmark assessment is conducted at
specific credit intervals assessing student dispositions and progress in the program. In addition,
students who are not at benchmark stages may be reviewed for academic or behavioral issues.
Students who are determined not to be making satisfactory progress due to academic,
dispositional, and/or conduct concerns are notified of any recommendations or decisions made by
the faculty and their due process rights.
One and Five-Year Follow-Up Studies of Graduates
The Office of Career Placement and Coop Services distributes one and five-year follow-up
surveys to graduates of the program. The program director has the opportunity to recommend
revisions to the program specific component based upon the finding from these surveys. These
surveys are conducted every other year. Data for 2006 were therefore not available.
Employers’ Surveys
The Office of Career Placement and Coop Services distributes surveys to employers of graduates
of the program on an every-other-year basis. The program director has the opportunity to
recommend revisions to the program specific component based upon the finding from these
surveys. This year, no data were available.
Qualitative Review of Program Performance Outcomes and Pupil Services Standards
All required courses were analyzed by program faculty during the 2006-2007 program revision
process to determine how existing courses matched with national accreditation standards. A
program revision matrix was developed. Some courses were dropped, others were revised, and a
new course was developed during the revision process. Overall credit requirements remained the
same for the revised program (50 credits). The program revision will be phased in during 20072008, with full implementation in place by fall 2008. See Appendix E for the program revision
matrix.
Practicum site supervisors evaluated student performance in each of the Pupil Services Standards
as part of their final evaluation.
6
Guidance and Counseling Practicum Student Evaluation
Guidance and Counseling graduate students all have a final practicum in the schools at the
elementary and secondary levels. Using the site supervisor ratings for Fall and Spring 2004/05
and 2005/06 the practicum student final evaluations were compiled and means were calculated
for each competency. Site supervisors used a 1-7 scale with 1= weak to 7=strong. In addition,
overall means were calculated for each category and each competency within the six categories.
Average Means and Rank Order of School Counseling Categories
Category
Personal Qualities
Developmental Guidance Skill Areas
Student Relations
Staff Relations
Parents and Community Relations
Use of Time
2004/05
Mean Rank
6.83
1
6.64
5
6.68
3.5
6.68
3.5
6.53
6
6.75
2
2005/06
Mean Rank
6.91
1
6.72
6
6.74
5
6.79
3
6.75
4
6.84
2
Based on the mean for each category in 2004/05 and 2005/06, the six categories were rank
ordered. An inspection of the table noted:
• Mean scores for each of the 6 categories increased from 2004/05 to 2005/06.
• Personal Qualities category had the highest mean in both 2004/05 and 2005/06.
• Use of Time category had the second highest mean in both 2004/05 and 2005/06.
• Staff Relations category was third in 2005/06 and tied for 3 & 4th position in 2004/05.
• The categories of Parents and Community Relations, Student Relations and
Developmental Guidance Skill Areas remained in the 4th, 5th and 6th spots even though
the rank order was slightly changed from 2004/05 to 2005/06.
School Guidance and Counseling Practicum Student Final Evaluation Fall 2004 and
Spring 2005 and 2005/2006 Academic Year
Category and Competency
PERSONAL QUALITIES
1. Dependable
2. Sincere
3. Flexible
4. Uses professional and ethical judgment
5. Communicates well with
a. Staff
b. Students
c. Supervisors
6. Understands and supports the goals
of the school
2004/05
N= 59
6.83
6.93
6.90
6.90
6.86
2005/06
N= 47
6.91
6.96
6.98
6.94
6.96
6.46
6.86
6.93
6.72
6.91
6.98
6.80
6.81
7
Category and Competency
DEVELOPMENTAL GUIDANCE
SKILL AREAS
1. Individual counseling
2. Small group facilitation
3. Classroom guidance activities
a. Lesson planning
b. Implementation
4. Assessment interpretation to parents, students
and colleagues
5. Plan & implement school or special group
programs
6. Contributes new & unique knowledge/ ideas to
existing program
STUDENT RELATIONS
1. Maintains confidentiality/understands explains
exceptions to clients
2. Gains a knowledge of client(s) using all
available resources
a. Records
b. Staff
c. Parents
d. Outside agencies
3. Pursues follow-up with
a. client(s)
b. staff
c. parents
d. agencies/others
STAFF RELATIONS
1. Initiates appropriate staff contacts
2. Shares information with staff
according to supervisors guidelines
3. Knowledge about institutional philosophy and
objectives
PARENTS AND COMMUNITY RELATIONS
1. Initiates contacts with parents
2. Conducts conferences with parents
3. Establishes parent rapport
4. Demonstrates knowledge of referral agencies
5. Initiates referral agency contacts
USE OF TIME
1. Plans time to meet student needs
2. Plans time to meet supervisor’s needs
3. Plans time to meet personal needs without
infringing on placement times
4. Plans on time to meet staff needs
5. Demonstrates creativity &
industriousness in utilizing unscheduled tim e
SUMMARY: Overall, how would you expect the
student to function as a beginning professional
2004/05
N= 59
2005/06
N= 47
6.64
6.74
6.75
6.72
6.82
6.78
6.64
6.68
6.45
6.71
6.80
6.67
6.65
6.70
6.58
6.58
6.68
6.74
6.88
6.91
6.73
6.72
6.53
6.46
6.79
6.72
6.67
6.67
6.77
6.77
6.64
6.62
6.68
6.64
6.91
6.81
6.71
6.46
6.79
6.75
6.73
6.68
6.85
6.78
6.53
6.45
6.50
6.62
6.54
6.55
6.75
6.76
6.79
6.76
6.75
6.66
6.89
6.74
6.75
6.73
6.84
6.90
6.91
6.89
6.65
6.86
6.81
6.68
6.79
6.83
8
The data presented in the above tables are from an old version of the Guidance & Counseling
practicum evaluation form which had a 7-point rating scale. The evaluation form has been
changed to a 6-point rating scale, and was first implemented in the fall of 2006. The old form
will no longer be used. In addition no other program data was entered into Datatel. See the
program director’s program specific report for data and recommendations.
9
EdS School Psychology
Graduate students in School Psychology are seeking licensure to be School Psychologists. They
must meet the 7 Wisconsin Pupil Services Standards. Practicum final evaluations and internship
final evaluations are included in this Assessment Report.
The National Association of School Psychologists (NASP) has developed stages of development
for graduate students in training. The School Psychology Development Performance Appraisal
involves assessing student performance on the Domains of School Psychology Training and
Practice. The four developmental stages include the following:
Stage 1: Observer
Stage I: This stage is characteristic of an observer. Minimal knowledge and skill
development are evidenced in the domain area, often due to lack of education,
training and experience. Training and supervision are required.
Stage 2: Novice
Stage 2: This stage is characteristic of a novice. Beginning knowledge and skill
development is evidenced in the domain area, often due to limited experience.
Some teaching and supervision are required.
Stage 3: Practitioner
This stage is characteristic of a practitioner. The individual exhibits knowledge,
proficiency, and competence in most aspects of the domain. The individual
may seek consultation for assistance in some aspects of the domain.
Stage 4: Mastery.
This stage is characteristic of an individual who has attained mastery.
The individual exhibits advanced knowledge in the domain. This stage is
characteristic of practitioners who are leaders and able to mentor, teacher, and/or
supervise novice school psychologists in the domain area.
As part of the practicum and internship final evaluation by the site supervisors, they are
asked to indicate the stage each graduate student is at on knowledge and performance for
each of the 11 NASP Domains which include:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Data-Based Decision-Making and Accountability,
Consultation and Collaboration—Effective Instruction and Development,
Cognitive and Academic Skills,
Socialization and Development of Life Skills,
Student Diversity in Development and Learning,
School and Systems Organization, Policy Development and Climate,
Prevention, Crisis Intervention, and Mental Health,
Home/School/Community Collaboration,
Research and Program Evaluation,
School Psychology Practice and Development, and
Information Technology
Each graduate student in School Psychology has three field experiences in school districts.
Practicum I is usually scheduled in fall semester and Practicum II is scheduled in spring
semester. The SPSY-792 internship is a year long experience.
10
During the SPSY-792 Internship experience, each School Psychology candidate has been rated
on their knowledge of and performance in the 11 NASP domains. Inspection of the average total
means for each of the domains reveals the means as calculated from use of the Developmental
Appraisal Form.
Developmental Appraisal Form
KNOWLEDGE
Data-Based Decision-Making & Accountability
Consultation & Collaboration
Effective Instruction & Development of Cognitive/
Academic Skills
Socialization & Development of Life Skills
Student Diversity in Development & Learning
School & Systems Organization, Policy Development,
& Climate
Prevention, Crisis, Intervention, & Mental Health
Home/School/Community Collaboration
Research & Program Evaluation
School Psychology Practice & Development
Information Technology
PERFORMANCE
Data-Based Decision-Making & Accountability
Consultation & Collaboration
Effective Instruction & Development of Cognitive/
Academic Skills
Socialization & Development of Life Skills
Student Diversity in Development & Learning
School & Systems Organization, Policy Development,
& Climate
Prevention, Crisis, Intervention, & Mental Health
Home/School/Community Collaboration
Research & Program Evaluation
School Psychology Practice & Development
Information Technology
Calendar Year
2004
2005
2006
N = 26
N = 40
N = 31
3.00
2.98
2.94
2.88
2.93
2.87
2.52
2.68
2.50
2.68
2.96
2.74
2.83
2.71
2.81
2.56
2.68
2.58
2.76
2.64
2.65
2.79
3.04
2.69
2.80
2.67
2.77
2.71
2.68
2.95
2.90
3.18
3.23
Calendar Year
2004
2005
2006
N = 26
N = 40
N = 31
2.88
2.80
2.81
2.65
2.78
2.68
2.46
2.40
2.40
2.42
2.81
2.46
2.58
2.52
2.52
2.38
2.43
2.48
2.50
2.58
2.50
2.68
2.96
2.31
2.44
2.38
2.82
3.20
2.50
2.48
2.40
2.71
3.16
Rating scale = Stage 1: Observer; Stage 2: Novice; Stage 3: Practitioner; Stage 4: Mastery
11
Based upon the domain scores of knowledge and performance of graduate students, mean
domain scores gradually increased from 2003/04 to 2005/06 as the student gained additional
experience in the schools from Practicum I to Practicum II to Internship. This analysis was not
conducted in calendar years 2004, 2005, 2006 but will be in next year’s report.
•
Practicum I
o Highest means—Knowledge of
ƒ Information Technology = 3.03
ƒ Data-Based Decision-Making and Accountability = 2.90
o Lowest means---Knowledge of
ƒ Cognitive /Academic Skills =2.4
ƒ Home/School/Community Collaboration = 2.47
o Highest means---Performance
ƒ Information Technology = 2.97
ƒ Data-Based Decision-Making and Accountability = 2.74
o Lowest mean—Performance
ƒ Prevention, Crisis Intervention and Mental Health = 2.29
ƒ Socialization and Development of Life Skills = 2.32
•
Practicum II
o Highest means---Knowledge of
ƒ Information Technology = 3.19
ƒ Data-Based Decision-Making and Accountability = 3.03
o Lowest means---Knowledge of
ƒ Prevention, Crisis Intervention and Mental Health = 2.37
ƒ Cognitive/Academic Skills = 2.42
o Highest means---Performance
ƒ Information Technology = 3.13
ƒ Data-Based Decision-Making and Accountability = 2.88
o Lowest means---Performance
ƒ Prevention, Crisis Intervention and Mental Health = 2.37
ƒ Cognitive/Academic Skills = 2.42
Internship
o Highest means---Knowledge of
ƒ Information Technology = 3.33
ƒ Data-Based Decision-Making and Accountability = 3.17
o Lowest means---Knowledge of
ƒ School & System Organization, Policy Development and Climate = 2.87
ƒ Research and Program Evaluation = 2.87
o Highest means---Performance
ƒ Information Technology = 3.37
ƒ Data-Based Decision-Making and Accountability = 3.00
ƒ Consultation & Collaboration, Effective Instruction & Development =
3.00
o Lowest means---Performance
ƒ Research and Program Evaluation = 2.67
ƒ School & System Organization, Policy Development & Climate = 2.71
ƒ
Socialization and Development of Life Skills = 2.72.
•
12
School Psychology Student Practicum and Internship Finals
Graduate student in the School Psychology program have supervised field experiences in the
schools both as practicum students and culminating in a graduate internship. The final evaluation
is completed by the field supervisor for both the practicum and the internship. The final
evaluation for the practicum and the internship consists of seven categories with the Wisconsin
Pupil Services Standards forming the eighth category. Each competency within each category is
rated. A scale of 1-5 is used with 1= needs improvement, 3=satisfactory and 5= highly
satisfactory.
The final evaluations from the field supervisors were compiled for the calendar years of 2004
and 2005 for each competency in the final evaluation instrument. Mean averages for each
category were computed and are shown.
Using the mean average for each category for 2004, 2005, and 2006 combined, practicum
students and the internships students can be compared in the table on the following page. The
order of the category amount the seven categories is based upon the category overall mean
average.
The table on the following page reveals quite a consistent pattern in practicum and internship
performance ratings by site supervisors. The following patterns were noted:
•
•
•
•
•
In almost all cases, the means for the internship categories were lower than those
of Practicum I and Practicum II.
Personal Characteristics and Supervisory Relationships mean rank orders were 1
and 2 in both practicum experiences and the internship.
Interpersonal Relationships, Communications & Consultation Skills were
consistently in 3rd or 4th ranks over the three year period.
General category was consistently in 5th or 6th place among the rank order
positions. This category deals with the role and functions of the school
psychologist, the place within the school and understanding district policies, etc.
Direct Service category typically had the lowest means in all three experiences
except in the 2006 internship category.
13
Overall Category Means and Rank Orders for School Psychology Practicums/Internships
Practicum I
(Semester I
completed
by Site
Supervisor)
(A) General
(B) Personal Characteristics
(C) Interpersonal Relationships,
Communications & Consultation Skills
(D) Supervisory Relationships
(E) Assessment Skills
(F) Professional Growth
(G) Direct Service
Practicum II
(Semester
II)
(A) General
(B) Personal Characteristics
(C) Interpersonal Relationships,
Communications & Consultation Skills
(D) Supervisory Relationships
(E) Assessment Skills
(F) Professional Growth
(G) Direct Service
Internship
(Full Year)
(A) General
(B) Personal Characteristics
(C) Interpersonal Relationships,
Communications & Consultation Skills
(D) Supervisory Relationships
(E) Assessment Skills
(F) Professional Growth
(G) Direct Service
Mean (rank)
2004
N = 16
4.39 (6)
4.80 (1)
4.60 (3)
Mean (rank)
2005
N = 11
4.48 (4)
4.82 (1.5)
4.62 (3)
Mean (rank)
2006
N=8
4.16 (6)
4.66 (2)
4.36 (3)
4.72 (2)
4.46 (4.5)
4.46 (4.5)
4.33 (7)
N=1
4.86 (*)
5.00 (*)
5.00 (*)
4.82 (1.5)
4.29 (6)
4.43 (5)
4.04 (7)
N = 14
4.49 (6)
4.83 (2)
4.67 (3.5)
4.78 (1)
4.31 (4)
4.18 (5)
4.14 (7)
N = 11
4.73 (4)
4.91 (1)
4.77 (3)
5.00 (*)
5.00 (*)
5.00 (*)
5.00 (*)
N = 11
4.01 (7)
4.58 (2)
4.31 (4)
4.93 (1)
4.50 (5)
4.67 (3.5)
4.26 (7)
N = 26
4.25 (6)
4.59 (2)
4.42 (4)
4.84 (2)
4.68 (5)
4.61 (6)
4.53 (7)
N = 13
4.65 (5)
4.74 (2)
4.71 (3)
4.61 (1)
4.17 (6)
4.32 (3)
4.27 (5)
4.61 (1)
4.34 (5)
4.48 (3)
4.15 (7)
4.75 (1)
4.58 (7)
4.66 (4)
4.60 (6)
Means calculated on a 5-point scale where 1=needs improvement, 3=satisfactory and 5=highly satisfactory
* differences in means are not statistically significant
14
Wisconsin Pupil Service Standards
In 2005 both School Counseling and School Psychology included the Wisconsin Pupil Services
Standards as part of the final evaluations for their candidates. Site supervisors rated each
candidate on these standards. This is the first time that a unit evaluation can be done on the basis
of meeting the Wisconsin Pupil Standards as part of the final candidate evaluation.
Inspection of the unit category means shown in the table reveals the following:
• Standard 4: Has knowledge of and skill in professional ethics and ethical behavior had
the highest unit mean of 4.87.
• Standard 3: Has knowledge and skill in research was second highest with a mean of
4.72.
• The lowest two means were for Standard 1: Understands the state teacher standards
(4.36) and Standard 2: Has knowledge of and skill in learning and instructional
strategies (4.53).
SOE Pupil Services Standards 2005 (School Counseling &
School Psychology)
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
Counsel. Pract.
S Psych Intern
SOE Unit
Has
Has
Understands
Has
the State knowledge of knowledge of knowledge of
& Skill in
& Skill in
Teacher
& Skill in
research
professional
Standards learning and
Ethics &
instructional
Ethical
technologies
behavior
Has
knowledge of
& Skill in
organization
& content of
effective pupil
services
Programs
15
Has
Has
knowledge of knowledge of
& Skill in
& Skill in a
wide array of consultation
&
intervention
strategies collaboration
SOE Pupil Services Standards 2005 (School Counseling and School Psychology)
Pupil Services Standards
Practicum I
1. Understands the State Teacher
Standards
2. Has knowledge of & skill in learning
and instructional strategies
3. Has knowledge of & skill in research
4. Has knowledge of & skill in
professional Ethics & ethical behavior
5. Has knowledge of & skill in
organization& content of effective
pupil services programs
6. Has knowledge of & skill in a wide
array of intervention strategies
7. Has knowledge of & skill in
consultation & collaboration
Pupil Services Standards Overall Mean
Pupil Services Standards
Practicum II
1. Understands the State Teacher
Standards
2. Has knowledge of & skill in learning
and instructional strategies
3. Has knowledge of & skill in research
4. Has knowledge of & skill in
professional Ethics & ethical behavior
5. Has knowledge of & skill in
organization& content of effective
pupil services programs
6. Has knowledge of & skill in a wide
array of intervention strategies
7. Has knowledge of & skill in
consultation & collaboration
Pupil Services Standards Overall Mean
16
Mean
2004
N = 16
Mean
2005
N = 11
Mean
2006
N=8
4.00
4.10
3.38
3.00
4.00
3.75
4.00
4.27
4.00
4.00
4.73
4.25
4.00
4.40
3.75
4.00
4.40
3.63
4.00
4.73
4.13
3.86
4.38
3.84
Mean
2004
N=1
Mean
2005
N = 14
Mean
2006
N = 11
-----
4.17
4.00
-----
4.11
4.11
-----
4.44
4.33
-----
4.60
4.60
-----
4.13
4.56
-----
4.40
4.30
-----
4.30
4.70
-----
4.32
4.39
Pupil Services Standards
Internship
1. Understands the State Teacher
Standards
2. Has knowledge of & skill in learning
and instructional strategies
3. Has knowledge of & skill in research
4. Has knowledge of & skill in
professional Ethics & ethical behavior
5. Has knowledge of & skill in
organization& content of effective
pupil services programs
6. Has knowledge of & skill in a wide
array of intervention strategies
7. Has knowledge of & skill in
consultation & collaboration
Pupil Services Standards Overall Mean
Mean
2004
N = 11
Mean
2005
N = 26
Mean
2006
N = 13
3.33
4.11
4.42
3.80
3.94
4.33
3.80
4.27
4.42
4.13
4.50
4.67
3.57
4.25
4.25
3.50
4.06
4.25
3.86
4.22
4.50
3.74
4.20
4.41
Means calculated on a 5-point scale where 1=needs improvement, 3=satisfactory, and 5=highly satisfactory.
17
PRAXIS II: Content Test
Benchmark II: Admission to Student Teaching requires candidates to pass PRAXIS II the
content test for their specific teacher certification. As of 8/31/2004, all Wisconsin teacher
education students must pass the content test to be eligible to student teach. Students who took
the content test during 2003-04 were “grandfathered in” as this was a no-fault year in Wisconsin.
School Psychology examines take the content test to meet NASP accreditation requirements, but
there was no cut score for 2004-05 in Wisconsin. The number of examinees taking a designated
content test varies by content test area. ETS did not include results of tests with fewer than 10
individuals in 2004/05 and or 5 individuals for 2005/06. Therefore, some content areas may not
be included in the tables for those years. However, these content areas are included in the data
reported by the UW-Stout Datatel System.
The UW-Stout Datatel System / Data Warehouse provides information on the UW-Stout
candidates who have taken PRAXIS II content test appropriate for their designated teaching
certification. Data from that system is reported on a calendar year basis.
Note: Comparing the ETS content test data with the Datatel content test data reveals a number of
discrepancies. This is due to the way individual information (i.e. first name, middle name or initial, and
last name) is entered into ETS by candidates at the time of testing. The inputted format must match
Datatel information exactly in order for the two systems to match for reporting and comparison purposes.
Thus, some content areas may depict a pass rate of less than 100% in ETS which is not accurate.
Likewise, the number of tests may not correctly match the Datatel system. The School of Education is
working with students, the state, and ETS to remedy this issue.
18
School Psychologist
Praxis Test Code - 10400
The Wisconsin School Psychologist exam cut score for passing was set in 2005/06. School
Psychologist data from the ETS report and Datatel is as follows:
School Psychologist – from Datatel
Number of Examinees:
Highest Observed Score:
Lowest Observed Score
WI Score Need to Pass:
Number with WI Passing Score
Percent with WI Passing Score
2004*
10
730
640
6/10
60%
2004
6
810
640
5/6
83%
2005
7
780
590
660
4/7
57%
2006
11
800
640
660
10/11
91%
* - scores from no-fault year – “grandfathered in” as passing
Content Test from ETS
Number of Examinees:
Highest Observed Score:
Lowest Observed Score:
Median:
Average Performance Range:
WI Score Needed to Pass:
Number with WI Passing
Score:
Percent with WI Passing
Score:
03/04
10
810
640
720
680-730
660
8/10
04/05
10
740
590
660
630-700
660
5/10
05/06
9
800
640
740
690-760
660
8/9
80%
50%
89%
Comparing the category scores for UW-Stout with Wisconsin and national percent correct shows
that School Psychology had one category where Stout results were same as or higher than state
average percent correct and all five categories were the same as or higher than the national
average percent correct in 2005/06.
19
Average Percent Correct
School
Psych Test
Category
Diag &
Fact-Finding
Prevent &
Intervention
Ap Psych
Found
Ap Ed
Found
Ethical &
Legal
Stout
Points
03/04 04/05 05/06 03/04
Available
%
%
%
%
State
04/05
%
National
05/06 03/04 04/05 05/06
%
%
%
%
29-30
84
76
81
85
80
82
79
77
77
29-30
76
72
77
78
79
78
75
74
75
23-24
73
68
77
75
75
76
72
71
74
12-14
71
57
70
72
71
72
69
67
67
19-22
79
74
81
81
78
79
77
76
76
The following table displays the number and percent of Stout Examinees Scoring in Each
Quartile in Each Category. Quartiles are calculated using all examinees taking this test during the
testing period. The number of examinee records used to compute the quartiles was 2,076 in
2003/04, 2,536 in 2004/05 and 2,677 in 2005/06.
School
Psych Test
Category
Diag & FactFinding
Prevent &
Intervention
Ap Psych
Found
Ap Ed Found
Ethical &
Legal
1st Q
Lowest
03/04
04/05
1
10%
2
20%
1
3
30%
2
20%
3
10%
1
10%
1
10%
30%
4
40%
3
30%
2nd Q
4th Q
Highest
3rd Q
05/06
1
11%
03/04
2
20%
04/05
2
20%
05/06
1
11%
03/04
5
50%
04/05
2
20%
05/06
4
44%
03/04
2
20%
04/05
3
30%
05/06
3
33%
1
5
5
5
1
3
1
2
0
2
11%
1
50%
5
50%
4
56%
4
10%
3
30%
2
11%
1
20%
1
0%
1
22%
3
11%
50%
40%
44%
30%
20%
11%
10%
10%
33%
1
11%
1
11%
3
3
5
3
0
4
40%
6
60%
3
30%
3
33%
2
22%
1
2
20%
2
22%
1
11%
1
10%
0
0%
3
33%
5
56%
20
Disposition Ratings
The School of Education has developed a system to assess candidate dispositions from the
beginning of the program through program completion. Dispositions of Teaching ratings are
completed for candidates in the graduate pupil services programs along with the portfolio review
process. The two graduate programs use the rating scale definitions of: 1=Unsatisfactory: Rarely
demonstrates disposition; 2=Minimal: Occasionally demonstrates disposition; 3=Satisfactory:
Usually demonstrates disposition; 4=Proficient: Consistently demonstrates disposition. In the
table below, Benchmark IV = MS Guidance & Counseling dispositions at two benchmark levels
and Benchmark V = School Psychology dispositions at two benchmark levels.
Attendance
Mean (N)
2004
3.77 (13)
3.63 (38)
Mean (N)
2005
3.82 (28)
3.67 (46)
Mean (N)
2006
3.78 (18)
3.34 (67)
3.62 (13)
3.61 (38)
3.71 (28)
3.70 (46)
3.95 (19)
3.72 (67)
Continuous Learning
BM IV
3.62 (13)
BM V
3.71 (38)
3.89 (28)
3.70 (46)
3.95 (19)
3.55 (67)
Positive Climate
BM IV
BM V
3.62 (13)
3.66 (38)
3.89 (28)
3.83 (46)
4.00 (19)
3.82 (67)
Reflective
BM IV
BM V
3.31 (13)
3.63 (38)
3.86 (28)
3.72 (46)
3.89 (19)
3.70 (67)
Thoughtful & Responsive Listener
BM IV
3.85 (13)
BM V
3.76 (38)
3.89 (28)
3.85 (46)
4.00 (19)
3.67 (67)
Cooperative / Collaborative
BM IV
3.77 (13)
BM V
3.84 (38)
3.96 (28)
3.89 (46)
3.84 (19)
3.91 (67)
Respectful
BM IV
BM V
3.93 (28)
3.93 (46)
3.89 (19)
3.94 (67)
BM IV
BM V
Preparedness
BM IV
BM V
3.92 (13)
3.61 (38)
21
1
Assessment in the Major: M.S. K-12 Guidance and Counseling
2006 Calendar Year
Submitted by: Dr. Barbara L. Flom, Program Director
This report presents findings of assessment of student learning and progress for
the M.S. Guidance & Counseling Program. It includes the following:
•
•
•
•
Goals and objectives of the program
Description of assessment methods
Assessment outcomes, calendar 2006
Plan for dissemination and application of assessment findings
Goals of the M. S. Guidance and Counseling Program
The M.S. in Guidance and Counseling initiated a program revision in 2006-2007
and completed the program revision approval process in summer 2007. The revised
program has been renamed M.S. in School Counseling. The revised program mission
statement follows:
School counseling is founded on the assumption of worth, dignity, and the
developmental potential of all individuals. School counselors work with children,
families, teachers, administrators, and other service providers in a team process to
promote student learning. School counselors assist youth in discovering, appreciating,
and actualizing their unique personalities and strengths. School counselors support the
academic, personal/social, and career development of students in PK-12 educational
settings.
UW-Stout’s School Counseling program prepares its students with a strong base
of foundational knowledge in counseling theory and practice, anchored to state and
national standards. In addition, the program develops the professional attitudes, skills,
and dispositions required for service in PK-12 school settings. UW-Stout’s School
Counseling Program helps graduate students develop these skills through an
experientially-oriented curriculum that includes clinical and field-based learning
activities. A primary goal of the program is to train future school counselors to work
with, support, and advocate for America’s diverse groups and individuals in educational
contexts, respecting and affirming differences in race, sex, age, religion, ability, and
cultural heritage.
Description of Assessment Methods
The M.S. Guidance and Counseling program at U.W.-Stout prepares graduate
students for assuming the important position of the school counselor. Because the
program has been reviewed and designated as fully approved by the Wisconsin
Department of Public Instruction (most recent review 11/2004), it provides extensive
2
classroom-based and experiential instruction, along with continuous assessment of
student progress, in accordance with Wisconsin’s Educator Licensing Statutes (PI 34).
Assessment of student progress occurs through a number of means. It should be
noted that currently the Praxis Content area exam is not yet in place for licensure of
school counselors in Wisconsin. State implementation of the Praxis exam is anticipated
in fall 2008. Assessment methods this year have included the following; detailed
descriptions of each follow the list:
•
•
•
•
•
Performance-based assessment system: Benchmarks I-IV
o Faculty review/selection of program applicants
o Student dispositions review
o Student portfolio review
o Criterion-referenced performance rubric for experiential component of the
program
Additional faculty review of student performance
o Transcript review
o Biannual program faculty review
One and five-year follow-up study of graduates
Employers’ follow-up study
Qualitative Review of Program Performance Standards
Performance Based Assessment System: Benchmarks I-IV
Student progress is assessed at four critical points in each student’s program. An
array of measures, including transcript review, dispositions ratings, portfolio review, and
performance rubrics, is used in the process. Appendix A lists, by benchmark, the
measures used at each.
Faculty Selection of Program Applicants: Benchmark I. Currently, through fall
2006, all student applicants to the program submit the Graduate College application,
transcripts, and three letters of recommendation. From the initial applicants
approximately 10-20 more candidates than the number of openings are invited to
interview for a position in the program. Average folio rating scores (a minimum cut off
score is established) determine which applicants are selected for an interview.
Applicants selected for the interview meet with a faculty team of 2-3 faculty and
respond to a standard set of questions in the following areas: summary of background and
experiences; experience working with children, youth and individuals of diverse
(multicultural) background; candidates understanding and view of the school counselors
roles and duties; candidate’s understanding of the impact of current issues on education
and child well-being.
The review committee reviews and rates all applicants’ written applications and
interview responses in the following areas: ability to do graduate work (based on GPA
and courses completed); related undergraduate/graduate preparation; related experience
with youth; content and quality of responses to essay questions; knowledge of the role
and function of school counselors; and letters of recommendation. Committee members
3
do not consult with each other during this blind review process. Combining the folio
ratings and the interview ratings yields a ranking of all applicants. Final selection of
admitted candidates is based on this ranked list.
The 2007 program revision includes an admissions policy change. A standardized
measure of professional-level skills in reading, writing, and math has been added to
application requirements beginning in spring 2008. Applicants may furnish scores from
either the PPST (Teacher Praxis I exam) or the Graduate Records Exam to fulfill this
requirement.
Student Dispositions Review: Benchmarks II, III. As part of the Benchmark
System for assessing student progress, student dispositions are reviewed for each student
after s/he has earned 24 credits and again during the semester prior to practicum
placement. The rubric for this review is attached in Appendix B. The eight identified
dispositions areas (attendance, preparedness, continuous learning, positive climate,
reflectivity, thoughtful/responsive listener, cooperativeness/collaboration, and respect) are
consistent with those required of all School of Education students in order to meet state
licensing requirements under PI 34.
Portfolio Assessment: Benchmarks II, IV. Students establish components of a
portfolio in three School Counseling courses (Career Development; Career Occupations
and Transitions, Practicum). Students currently complete the portfolio over the course of
three+ semesters, including in the portfolio all 12 DPI established Content Guidelines of
competence (See Appendix C for portfolio review rubric.) The instructor of the courses
evaluates each of the portfolios. The university practicum supervisor evaluates the
completed portfolio near the end of practicum, immediately prior to graduation
(Benchmark IV). Results are shared in monthly program faculty meetings and
disseminated to the Program Advisory Committee each semester.
Criterion-referenced performance rubric for capstone experiential program
component: Benchmark IV. The 600-hour practicum follows core coursework and
serves as the capstone experience for graduate students. Practicum site supervisors are
selected in accordance with DPI requirements as outlined on page 19 of the Handbook of
Standards, Procedures, and Policies for the Approval of Professional Education Programs
in the State of Wisconsin, under “Clinical Program Requirements,” Practicum. Site
supervisors providing supervision to UW-Stout Guidance and Counseling students must
meet the following criteria:
•
•
•
•
Hold a Wisconsin license in School Counseling
Have at least 3 years of clinical experience as a guidance counselor
Have at least one year of employment as a guidance counselor in their current
employment placement
Have completed education and training in clinical supervision of students and the
Wisconsin Standards for guidance counselors and pupil services personnel.
4
•
o Graduates of UW-Stout’s Guidance and Counseling program meet this
requirement via a required course entitled “Organization and
Administration of Guidance Programs” (SCOUN 765, 3 credits)
Site supervisors for UW-Stout practicum must sign a “verification form” that they
have met all of the above requirements
A list of verified site supervisors is kept in the Office of Teacher Education under the care
of the Practicum Placement Coordinator.
The site supervisor-counselors who supervise our students during their capstone
course complete a criterion-referenced performance based rubric (see Appendix D) at the
mid-point and completion of the experience. This performance rubric changed
significantly between spring 2006 and fall 2006, so unit-level scores on the two are not
combined. The supervising university faculty member also completes the rubric in
consultation with the cooperating counselor and other supervising faculty, thus assigning
a final grade. The student completes the rubric as a self-assessment and the faculty
member discusses any discrepancies with the student.
Additional Faculty Review of Student Performance:
In addition to systematic review of student progress at the benchmark points, the
program director and program faculty monitor student learning through the following:
Transcript Review. Student transcripts are reviewed by the program director at the
completion of each semester. Students who fall below the minimum GPA of 3.0 in a
program core course and/or fail any course are informed of the deficiency and the
necessity for meeting with the program director to develop a remediation plan.
Two critical pre-practicum counseling skill development courses are required in
the Guidance and Counseling program. Students must successfully (3.0 GPA minimum)
complete the following courses before being admitted to practicum, as noted in transcript:
COUN 788 Counseling Process Laboratory. Guidance and Counseling students
are required to complete 25 video taped sessions with a diverse group of clients.
Video taped sessions are reviewed and written assessment provided to the student.
Students must have clients featuring the following demographic variables:
• At least one client age older than 50 years
• At least 3 clients aged 13 or under, and
• At least two clients belonging to an American minority
group
SCOUN 705 Play Therapy. Guidance and Counseling students are required to
meet with 2-3 children for a minimum of nine sessions
• Child clients are typically under the age of 12 years
• All sessions are videotaped. Video taped sessions are
reviewed with the instructor or graduate assistant and the
student receives written feedback on his/her performance.
5
In addition, a third class includes a service learning component. SCOUN 738, Guidance
in the Elementary Schools requires students to complete 15 service learning hours in an
elementary or middle school tutoring children.
Biannual Program Faculty Review. School Counseling faculty meet minimally
twice-per-year to review student progress in the major. Students may be reviewed more
frequently if requested by any faculty member. Students sign an acknowledgement of this
“Student Review, Retention, and Dismissal” Policy upon admission into the program.
During this biannual review, benchmark assessment is conducted at specific credit
intervals assessing student dispositions and progress in the program. In addition, students
who are not at benchmark stages may be reviewed for academic or behavioral issues.
Students who are determined not to be making satisfactory progress due to academic,
dispositional, and/or conduct concerns are notified of any recommendations or decisions
made by the faculty and their due process rights.
One and Five-Year Follow-Up Studies of Graduates
The Office of Career Placement and Coop Services distributes one and five-year
follow-up surveys to graduates of the program. The program director has the opportunity
to recommend revisions to the program specific component based upon the finding from
these surveys. These surveys are conducted every other year. Data for 2006 were
therefore not available.
Employers’ Surveys
The Office of Career Placement and Coop Services also distributes surveys to
employers of graduates of the program on an every-other-year basis. The program
director has the opportunity to recommend revisions to the program specific component
based upon the finding from these surveys. This year, no data were available.
Qualitative Review of Program Performance Outcomes and Pupil Services Standards
All required courses were analyzed by program faculty during the 2006-2007
program revision process to determine how existing courses matched with national
accreditation standards. A program revision matrix was developed. Some courses were
dropped, others were revised, and a new course was developed during the revision
process. Overall credit requirements remained the same for the revised program (50
credits). The program revision will be phased in during 2007-2008, with full
implementation in place by fall 2008. See Appendix E for the program revision matrix.
Practicum site supervisors evaluated student performance in each of the Pupil
Services Standards as part of their final evaluation.
6
Assessment Outcomes 2006
Benchmark I: Admissions and Enrollment Outcomes
Application, Review, Interview and Acceptance Process
Benchmark I Results for 2006:
•
•
•
Application, Review, Interview and Acceptance Process are described on page
1 of this document.
Graduate school data indicate 64 applications were received in spring for
summer/fall 2006 enrollment, and 26 applications were received in fall for
spring 2007 enrollment in the M. S. in Guidance and Counseling program.
These numbers reflect the typical pattern of approximately twice as many
applications for summer/fall enrollment.
o In spring 2006, 37 applicants were recommended to be interviewed. In
fall 2006, 15 students were recommended to be interviewed. As a
result of the folio review and interviews, 31 candidates were accepted
for summer/fall 2006 and 13 were accepted for spring 2007. Thus, 44
of 52 interviewed and 90 total applicants were accepted for enrollment
into the program. Of these, 39 enrolled in the program. Our overall
enrollment target for 2006 was 35-40 enrollees.
Students who were not recommended for admission were denied due to some
combination of low grade point average (below 2.75); lack of experience with
youth; low rating on the interview responses; and/or program capacity having
been reached.
Benchmark II: Candidate Review of Progress at 24 Credits
Dispositions Ratings, Portfolio Review for individual students
Benchmark II Results for 2006:
•
•
•
Dispositions rating process and portfolio review process are described on
pages 2-3 of this document.
Thirteen students received Benchmark II Dispositions ratings in Spring 2006,
and 20 students received Benchmark II Dispositions ratings in Fall 2006. In
spring, two students received unsatisfactory dispositions ratings. One student
received an unsatisfactory rating in the fall. Program faculty met with all three
students individually and developed a remediation plan for each to allow for
progress in the program. One of these students was subsequently found to
have committed serious academic misconduct during fall semester and was
dismissed from the program.
Two faculty independently reviewed a total of 33 portfolios in progress for the
2006 calendar year. No portfolio reviews were unsatisfactory based on student
position in the program (e.g., early versus end of program). Portfolio ratings
ranged from 1-2 (observer), 3-4 (novice) and 5-6 (practitioner). Given the
7
experiential foundation of the program, the majority of students had artifact
ratings at the “novice” stage at the Benchmark II assessment stage.
Benchmark III: Pre-Practicum Review
Dispositions Ratings for individual students; Completion of Course Requirements
with “B” grade or Better
Benchmark III Results for 2006:
•
•
•
A total of 20 students received Benchmark III Reviews in Spring 2006, and 18
students were reviewed for this benchmark in Fall 2006, for a total of 38
Benchmark III reviews in 2006. It should be noted that, for a number of
students in the program, the 24-credit review occurs during the same semester
as the pre-practicum review. In 2006, 24 of 38 students had completed 24
credits during the same semester when they were preparing for practicum, so
the Benchmark II-III dispositions reviews coincided.
In spring 2006, 18 of 20 students received satisfactory or proficient ratings at
Benchmark III. The remaining two students, whose Benchmark III ratings
coincided with their Benchmark II ratings (thus, these are the same two
students identified above under Benchmark II), met with program faculty to
develop remediation plans in spring.
In fall 2006, 17 of 18 reviewed students received satisfactory or proficient
ratings at Benchmark III. The remaining student met with faculty and
continued a remediation plan which followed her into the practicum setting
during spring 2007.
Benchmark IV: Final Review
Portfolio Review, Practicum Final Evaluation, Completion of Degree
Requirements for individual students.
Benchmark IV Results for 2005-2006:
•
•
In 2006, 20 portfolios received Benchmark IV reviews independently by two
faculty members. Twelve portfolios were reviewed in the spring, and 8 were
reviewed in fall. All 20 portfolios received satisfactory end-of program
ratings. The majority of artifact ratings were in the Initial Practitioner (5/6)
range.
In 2006, criterion-referenced performance rubrics, or practicum student
evaluations, were collected from site supervisors of 20 students, 12 in spring
and 8 in fall. Practicum students had anywhere from 1-3 site supervisors
depending on the size of their K-12 district placements. All scores were in the
satisfactory to outstanding range. School of Education data analysis showed
item means for 2005-2006 all fell above 6 on a 7-point scale. A new form was
adopted in Fall 2006, and item means have not been derived for this
instrument.
8
•
For spring, summer, and fall 2006, 21 students successfully completed all
degree and certification requirements. These 21 students were awarded the
M.S. degree. All are eligible for licensure in Wisconsin as school counselors.
Results of Faculty Review of Student Performance:
Transcript Review, Biannual Program Faculty Review:
•
•
•
The end-of-semester transcript review revealed that one student fell below the
minimum GPA standards in one class in the 2006 calendar year. Also, an
additional student issue was addressed successfully related to academic and
nonacademic performance in the program:
o A student satisfactorily repeated a course (earning an A) in which she had
previously received an F.
All program students enrolled in preclinical courses Counseling Process Lab and
Play Therapy, as well as capstone courses Elementary Counseling Practicum and
Secondary Counseling Practicum, completed the experiential courses with a grade
of 3.0 or better.
Student performance was reviewed by program faculty as a group in fall 2005 and
spring 2006. With the exceptions of the two students noted previously under
disposition and transcript reviews, all students were performing satisfactorily in
the program.
One-Year and Five-Year Follow-up Studies
•
One-year and five-year follow studies were not conducted in 2006.
Employer Surveys
•
Employer surveys were not conducted in 2006.
Program Performance Outcomes:
Qualitative review of program curriculum against state and national standards;
Analysis of Pupil Services Standards ratings.
•
•
Content guidelines and pupil services standards have been presented in new
student orientation and practicum orientation sessions. They have also been
incorporated into the newly-published student program handbook.
During the program revision process, program performance outcomes were
compared with outcomes required by the national accrediting body for school
counseling programs, the Council for Accreditation of Counseling-Related
Programs (CACREP). A program revision was in progress during 2006. (It has
been approved as of summer 2007 for implementation in summer 2008.) Notably,
9
the revision will entail additional pre-clinical experience hours as well as infusion
of diversity-related content into all coursework.
Dissemination and Use of Assessment Results for Program Improvement
Results of the assessment information will be shared with program faculty and
staff, and adjunct faculty at the first program meeting in the fall. In addition, the Program
Advisory Committee will receive the report and have an opportunity for response at its
meeting in fall 2007. The report will be available for student review as well.
Performance-Based Benchmark Student Assessment System:
•
•
Program instructors are pleased with the revised criterion-referenced practicum
final evaluation instrument, which was piloted in fall 2006. The revised
instrument will be utilized also in the pre-clinical courses. This change will better
align with NCATE and CACREP requirements.
We added a dispositions review by the practicum site supervisor to our practicum
assessment process in fall 2006. Based on feedback from supervisors, the review
is fruitful and will be continued.
Faculty Review/ Selection of Program Applicants:
•
•
Feedback from applicants and faculty indicate that a common orientation has been
established and maintained. The only anticipated change in applications
procedures is the inclusion of a GRE or PPST score for fall 2008 admissions.
The Program Director typically meets with or telephone conferences with 2-3
prospective applicants per week for advisement purposes. And additional 3-4
potential applicants are advised via email per week.
Transcript Review:
•
The transcript review process will remain in place with no changes anticipated
before our program revision. The Program Director typically meets with or
telephone conferences with 5-6 School Counseling students per week. Information
gleaned from transcript review, e.g., appropriate workload, is constantly used in
advisement.
Faculty Review of Student Performance:
•
The results of faculty review have resulted in changes to individual student’s
program plans, remediation plans, and successful student outcomes. The faculty
10
will continue to meet regularly to review student performance via dispositions as
well as mastery of curriculum material (portfolio assessment).
Qualitative Review of Performance Standards:
•
SOE had identified a program revision as a priority for 2006-2007. This revision
has been ushered through the curriculum process. Curricular changes were based
upon faculty review of one and three year follow up studies, student portfolio
reviews, CACREP/NCATE requirements, and the changing needs of guidance
counseling in the 21st century.
Respectfully Submitted,
Barbara Flom, Ph.D., L.P.C.
Program Director, K-12 Guidance and Counseling
403 McCalmont Hall
(715)232-1343
flomb@uwstout.edu
11
Appendix A
M.S. Guidance & Counseling Program
Benchmark Assessment System
Performance Based Assessment System
School Counseling
The School of Education has adopted a Pupil Services Performance-Based
Assessment System to ensure that all pupil service candidates possess the
knowledge, skills, and dispositions required for successful careers in education.
This assessment system establishes benchmarks at which all candidates for
pupil services certification will be assessed. In addition, all pupil services
candidates will be required to maintain a portfolio with evidence of meeting the
Wisconsin Department of Public Instruction School Counseling Content
Guidelines.
Benchmark I: Acceptance into the Guidance and Counseling Program
▫ Earned bachelor’s degree
▫ Undergraduate GPA of 2.75 or greater
▫ Satisfactory scores on folio review rating scale
▫ Satisfactory scores on interview rating scale
▫ Passed educator background check
Benchmark II: Candidate review of progress at 24 credits
▫ Satisfactory scores on portfolio review rating scale
▫ Satisfactory scores on dispositions’ review rating scale (>24 total score or
satisfactory or above ratings in each area)
Benchmark III: Pre-practicum review
▫ Satisfactory scores on dispositions’ review rating scale (>24 total score of
satisfactory or above ratings in each area)
▫ “B” (3.0) or better in all courses
▫ Completion of all course requirements
Benchmark IV: Final review
▫ Satisfactory scores on dispositions’ review for all probationary students
▫ Satisfactory scores on portfolio review rating scale
▫ Satisfactory completion of all degree/certification requirements
▫ When available, completion of Wisconsin’s content exam
12
Appendix B: Disposition Review
School Guidance and Counseling
UW-Stout School of Education
Raters: _____________________Date:_________________
Student: _________________________________________
Review:
Benchmark 2: 24 Credits
Benchmark 3: Semester Prior to Practicum
Benchmark 4: During Practicum (if total score <24 previously or
any one rating less than 3 previously)
Dispositions
Unsatisfactory
1
Minimal
2
1 = Unsatisfactory:
2 = Minimal:
3 = Satisfactory:
4 = Proficient:
Rarely demonstrates disposition
Occasionally demonstrates disposition
Usually demonstrates disposition
Consistently demonstrates disposition
Satisfactory
3
Proficient
4
Numerous tardies/early
Departures OR 4+ absences
Several tardies/early
departures OR 3 absences
Few tardies/early
departures OR 2 absences
Timely and consistent
presence in class/school
Preparedness
Rarely, if ever, well prepared for
class/practicum; e.g.,
evidence of completed
reading/assignments
and engagement of reading
materials - written notes,
questions, other responsibilities
Occasionally well prepared
for class/practicum;
e.g., evidence of completed
reading/assignments and
engagement of reading
materials - written notes,
questions, other
responsibilities
Usually well prepared for
class/practicum; e.g.,
evidence of completed
reading/assignments and
engagement of reading
materials - written notes,
questions, other
responsibilities
Consistently well prepared for
class/practicum; e.g.,
evidence of completed
reading/assignments
and engagement of reading
materials - written notes,
questions, other
responsibilities
Continuous
Learning
Rarely, if ever, demonstrates
curiosity, creativity and flexibility
regarding course content,
processes and tasks; sets
high expectations for self
Occasionally demonstrates
curiosity, creativity and
flexibility regarding course
content, processes and
tasks; sets high expectations
for self
Usually demonstrates
curiosity, creativity and
flexibility regarding course
content, processes and
tasks; sets high expectations
for self
Consistently demonstrates
curiosity, creativity and
flexibility regarding course
content, processes and tasks; ,
sets high expectations for self
Positive
Climate
Rarely, if ever, views feedback
and situations maturely;
analyzes feedback and makes
appropriate adjustments to
enhance personal growth &
learning; analyzes comments &
interactions to make
appropriate adjustments that
promote a positive learning
environment
Occasionally views
feedback and situations
maturely; analyzes
feedback and makes
appropriate adjustments to
enhance personal growth &
learning; analyzes
comments & interactions to
make appropriate
adjustments that promote a
positive learning
environment
Usually views feedback and
situations maturely;
analyzes feedback and
makes appropriate
adjustments to enhance
personal growth & learning;
analyzes comments &
interactions to make
appropriate adjustments
that promote a positive
learning environment
Consistently views feedback
and situations maturely;
analyzes feedback and makes
appropriate adjustments to
enhance personal growth &
learning; analyzes comments
& interactions to make
appropriate adjustments that
promote a positive learning
environment
Attendance
Points
13
Dispositions
Unsatisfactory
1
Reflective
Rarely, if ever, willing to suspend
initial judgments, receptive of a
critical examination of multiple
perspectives, generate
effective/productive options,
make reasoned decisions with
supporting evidence, makes
connections to previous
reading/courses/experience, etc.
Minimal
2
Satisfactory
3
Occasionally willing to
Usually willing to suspend
suspend initial judgments,
initial judgments, receptive of
receptive of a critical
a critical examination of
examination of multiple
multiple perspectives,
perspectives, generate
generate effective/productive
effective/productive options, options, make reasoned
make reasoned decisions with decisions with supporting
supporting evidence, makes evidence, makes connections
connections to previous
to previous
reading/courses/experience, reading/courses/experience,
etc.
etc.
Proficient
4
Consistently willing to suspend
initial judgments, receptive of a
critical examination of multiple
perspectives, generate
effective/productive options, make
reasoned decisions with
supporting evidence, makes
connections to previous
reading/courses/experience, etc.
Thoughtful &
Responsive
Listener
Occasionally demonstrates
Usually demonstrates the
Clearly demonstrates the ability to
Rarely, if ever, demonstrates the
the ability to thoughtfully listen ability to thoughtfully listen & thoughtfully listen & respond to
ability to listen to people's
& respond to people's
respond to people's insights, people's insights, needs, &
insights, needs, & concerns;
insights, needs, & concerns, needs, & concerns, e.g. asks concerns, e.g. asks questions,
appears unable to respond either
e.g. asks questions,
questions, summarizes points, summarizes points, etc.
positively or thoughtfully
summarizes points, etc.
etc.
Cooperative &
Collaborative
Rarely, if ever works to keep
group on task, maximize
individual talents, evenly
distribute responsibility, etc.
Occasionally works to keep
group on task, maximize
individual talents, evenly
distribute responsibility, etc.
Usually works to keep group
on task, maximize individual
talents, evenly distribute
responsibility, etc.
Consistently works well with
others, e.g., keeps group on task;
maximizes individuals' talents;
evenly distributes responsibility,
etc.
Rarely, if ever, shows due
courtesy & consideration for
people & ideas; demonstrates
sensitivity with respect to
language use
Occasionally shows due
courtesy & consideration for
people & ideas; demonstrates
sensitivity with respect to
language use
Usually shows due courtesy &
consideration for people
&ideas; demonstrates
sensitivity with respect to
language use
Consistently shows due courtesy
& consideration for people &
ideas; demonstrates sensitivity
with respect to language use
Respectful
Unsatisfactory Level:
Minimal Level:
Satisfactory Level:
Proficient Level:
8-15 points (or a rating of 1 or 2 in any category)
16-23 points
24-31 points
32 points
Points
TOTAL SCORE = _________
14
Appendix C: Portfolio Review Rubric
School Guidance and Counseling
University of Wisconsin—Stout
Student Name: ___________________________
Review:
Reviewer Signature: ______________________
Observer:
Novice:
Practitioner:
Date: ___________________
Minimal knowledge and skill development are evidenced in the domain area, often due to a lack of education,
training and experience. Training and supervision are required.
Beginning knowledge and skill development are evidenced in the domain area, often due to limited experience.
Some teaching and supervision are required.
The individual exhibits knowledge, proficiency, and competence in most aspects of the domain. The individual may
seek consultation for assistance in some aspects of the domain.
Observer
1
1. Psychological &
Sociological Foundations
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
2. Program Organization &
Administration
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
3. Prevention, Intervention,
& School Consultation
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
4. Diversity & Inclusion
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
5. Individual & Group
Counseling
Reflection:
Artifact:
Artifact:
Artifact:
Benchmark 2: 24 Credits
Benchmark 4: During Practicum
Yes
No
2
Novice
3
Practitioner
4
5
6
Comments
15
Observer
1
2
6. Career Development
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
7. PK-12 Developmental
Guidance Transitions
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
8. School Law & Ethics
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
9. Assessment & Research
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
10. Technology
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
11. Professional
Development
Reflection:
Yes
No
Artifact:
Artifact:
Artifact:
12. Practicum
Reflection:
Yes
No
Artifact: University Supervisor Evaluations
Artifact: Site Supervisor Final Evaluation
Artifact:
Novice
3
4
Practitioner
5
6
Comments
16
Appendix D: Internship Evaluation Rubric: Midterm Assessment
School Guidance and Counseling
University of Wisconsin—Stout
Student Name: _____________________________________________
Student ID: __________________
School Site: _________________________________________________
Review Date: ________________
Please rate the student named above on the following items of professional knowledge and skill. Please
use the scale described below, remembering that students are not expected to reach practitioner level
until the end of their internship experience.
Observer:
Minimal knowledge and skill development are evidenced in the domain area, often due
to a lack of training and experience. Training and supervision are required.
Novice:
Beginning knowledge and skill development are evidenced in the domain area, often
due to limited experience. Some teaching and supervision are required.
Practitioner: The individual exhibits knowledge, proficiency, and competence in most aspects of the
domain. The individual may seek consultation for assistance in some aspects of the
domain.
Demonstrates Professional
School Counseling Program
Delivery Skills
Individual
Counseling/Planning
Demonstrates facilitative
interpersonal skills
Interprets assessment
data
Monitors student progress
Group
Counseling
Demonstrates facilitative
interpersonal skills
Monitors relationships
between group members
Tracks group progress and
group functioning
Developmental Guidance
Curriculum
Plans lessons with
appropriate content and
materials
Uses effective instructional
strategies
Connects curriculum to
state/national student
standards
Observer
1
2
Novice
3
Initial Practitioner
4
5
6
17
Demonstrates Professional
School Counseling Program
Delivery Skills, continued
Observer
1
2
Novice
3
Initial Practitioner
4
5
6
Transition
Services
Assists students/parents at
educational transitions
(home to school,
elementary to high school,
secondary to postsecondary)
Consultation &
Collaboration
Communicates effectively
with staff
Communicates effectively
with families
Refers to
school/community
resources
System
Support
Implements school-wide
prevention programs
Participates in other
school activities
Demonstrates Knowledge of
School Counseling
Foundations
Applies knowledge of child
development, psychology, and
counseling theory in daily
practice
Understands and supports
institution’s mission
Applies knowledge of the impact
of diversity on student academic,
personal/social, and career
development
Advocates for individual students
and for systemic change
Applies knowledge of state and
federal laws in daily practice
Acts in accordance with
professional ethical standards of
the American School Counselor
Association
Observer
1
2
Novice
3
Initial Practitioner
4
5
6
18
Demonstrates Skill in Program
Management &
Accountability
Observer
1
Novice
2
3
Initial Practitioner
4
5
6
Connects school counseling
program to national standards,
state standards, and needs of
school
Evaluates impact of school
counseling program components
and interventions
Uses research, evaluation, and
student assessment data for
program planning
Uses technology for program
implementation
Plans and uses time effectively to
accomplish program goals
Please rate the student on his/her knowledge of and skill in the Pupil Service Standards.
Pupil Service Standard
1
Student understands the state
teacher standards.
Student has knowledge of and
skill in learning and instructional
strategies.
Student had knowledge of and
skill in research.
Student has knowledge of and
skill in professional ethics and
ethical behavior.
Student has knowledge of and
skills in the organization and
content of effective pupil service
programs.
Student has knowledge of and
skill in a wide array of
intervention strategies.
Student has knowledge of and
skill in consultation and
collaboration.
2
3
4
5
19
Appendix E: Program Revision
Title:______M.S. School Counseling__________________________________________________________
Proposed Implementation/Catalog Year____Spring 2008 ______________
EXISTING PROGRAM
PROPOSED ACTION
REVISED PROGRAM
COUN 750 (3 cr.): Counseling Theory
Maintain course
COUN 750 (3 cr.): Counseling Theory
COUN 788 (3 cr.): Counseling Process Lab
Maintain course
COUN 788 (3 cr.): Counseling Process Lab
COUN 752 (3 cr.): Group Dynamics
Maintain course
COUN 752 (3 cr.): Group Dynamics
Add course
HDFS 742 (3 cr.): Lifespan Family & Human
Development
SPSY 753 (2 cr.): Psychometric Theory &
Application
Maintain course
SPSY 753 (2 cr.): Psychometric Theory &
Application
SPSY 775 (2 cr.): School Consultation
Revise course
*SPSY 775 (2 cr.): School Consultation
EDUC 729 (1 cr.): Intro to Ed Research
Change to minimum competency
EDUC 740 (4 cr.): Research Foundations
Maintain course
EDUC 740 (4 cr.): Research Foundations
SCOUN 735 (2 cr.): Problems in
Counseling/Psychological Services OR
Maintain course
SCOUN 735 (2 cr.): Problems in
Counseling/Psych Services, OR
SCOUN 770 (6 cr.): Thesis
Maintain course
SCOUN 770 (6 cr.): Thesis
SCOUN 782 (3 cr.): PK-12 Developmental
Guidance Prereq: COUN 675
Revise course
*SCOUN 782 (3 cr.): PK-12 Developmental
Guidance Curriculum. .Prereq: COUN 750,
SCOUN 765
SPSY 753A (1 cr.): Guidance Lab. Prereq/coreq: SPSY 753
Revise course
*SPSY 753A (1 cr.): School Counseling Lab
Prereq/coreq: SPSY 753
SCOUN 733 (3 cr): Career Development
Theories, Issues, Information. Prereq:
COUN 675
Revise course
*SCOUN 733 (3 cr): Career Development
Theories, Issues, Information. Prerequ:
Coun 750
SCOUN 765 (3 cr.): Organization &
Administration of Guidance. Prerequ: COUN
675
Revise course
*SCOUN 765 (3 cr.): Professional
Orientation: School Counseling
20
Add course
*SCOUN 788 (3 cr.): School Counseling
Practicum. Prereq: COUN 750, COUN 788
SCOUN 647 (3 cr.): Behavior Problems of
Children
Revise course
*SCOUN 647 (3 cr.): Emotional/Behavioral
Problems of Children
SCOUN 690 (2 cr.): Behavioral Interventions
in the Schools
Revise course; Move to Selective
SPSY 777 (2 cr.): Legal/Ethical Issues
Revise course
SCOUN 738 (2 cr.): Guidance in the
Elementary School
Delete course
SCOUN 705 (2 cr.): Play Therapy
Revise course; Move to Selective
SCOUN 787 (2 cr.): Career/Occupational
Placement and Transitions
Delete course
SCOUN 789 (3 cr.): Supervised Elementary
Counseling Practicum. Prereq: COUN 675,
COUN 788, COUN 752, SCOUN 733, SCOUN
647, SCOUN 760, SCOUN 738, SCOUN 705,
SPSY 753, SPSY 753A, SCOUN 765, SPSY
775, SPSY 777, SCOUN 787
Revise course
*SCOUN 789 (3 cr.): Elementary School
Counseling Internship. Prereq: COUN 750,
COUN 788, COUN 752, SCOUN 733, SCOUN
647, SPSY 753, SPSY 753A,
SCOUN 765, SPSY 775, SPSY 777,SCOUN
782, SCOUN 788, HDFS 742*
SCOUN 790 (3 cr.): Supervised Secondary
School Counseling Practicum
Revise course
*SCOUN 790 (3 cr.): Secondary School
Counseling Internship. Prereq: Prereq:
COUN 750, COUN 788, COUN 752, SCOUN
733, SCOUN 647, SPSY 753, SPSY 753A,
SCOUN 765, SPSY 775, SPSY 777,SCOUN
782, SCOUN 788, HDFS 742
Add course
*SCOUN 793 (2 cr.): School Counseling
Internship Seminar
Add one credit to requirement; Additional courses
Selective Options (2 cr. required; self-designed
concentrations encouraged)
Selective Options (1 cr. required)
PSYC 551 (3 cr.): Child Social Reasoning
EDUC 600 (2 cr.): Self-Esteem
SPED 500 (3 cr.): Cognitive Disabilities
*SPSY 777 (2 cr.): Legal/Ethical Issues*
21
SPSY 768 (2 cr.): Diag. & Rem. Of Learning
Disabilities
PSYC 666 (2 cr.): Alcohol & Family Systems
PSYC 666 (2 cr.): Alcohol & Family Systems
EDUC 600 (1 cr.): Domestic Violence &
Classroom
EDUC 600 (1 cr.): Domestic Violence &
Classroom
EDUC 600 (1 cr.): Children from Chaos
Revise Course
Revise Course
EDUC 600 (1 cr.): Children from Chaos
*SCOUN 690 (2 cr.): Behavioral
Interventions in the Schools
*SCOUN 705 (2 cr.): Play Therapy
CTE 502 (2 cr.): Principles of Career and Tech
Ed
SPED 524 (3 cr.): Curriculum & Instruction:
Career and Transition Education
EDUC 615 (3 cr.): Classroom Management
EDUC 750 (3 cr.): Curriculum Theory &
Practice
FCSE 708 (2 cr.): Curriculum Studies
SCOUN 792 (2 cr.): Postsecondary Practicum
ANTH 620 (3 cr.) Anthropological Study of
Family Systems
Minimum Competency Courses for Certification
Minimum Competency Courses for Certification
Psychology of Learning Requirement (2 cr.):
EDUC 303, PSY 330/530, PSY 730
Maintain requirement
Curriculum & Instruction Requirement (2 cr.):
FCSE 708, EDUC 312, EDUC 750
Replace with revised required course (SCOUN 782)
Inclusion Requirement (3 cr.)
SPED 430/630
Multiculturalism Requirement (3 cr.)
Maintain requirement
Maintain requirement
Psychology of Learning Requirement (2 cr.):
EDUC 303/503, PSY 330/530, PSY 730
Inclusion Requirement (3 cr.)
SPED 430/630
Multiculturalism Requirement (3 cr.)
22
EDUC 336/536 + EDUC 376/576
Developmental Psychology Requirement (3 cr.):
PSY 251+ PSY 352/552; HDFL 124+HDFL
325; PSY 850
Organization, Administration, Operation of Schools
Requirement (2 cr.):
EDUC 326, EDUC 726
EDUC 336/536 + EDUC 376/576
Move to program requirement; Psy 750 or HDFS
742
Maintain Requirement
Organization, Administration, Operation of Schools
Requirement (2 cr.):
EDUC 326, EDUC 726
Add Minimum Competency
Minimum Competency: Preparation for Research (1
cr.)
EDUC 729
Policy Change: Admissions Requirements
Graduate School Application, G.P.A. 2.75, three
letters of recommendation, personal goal
statement, personal interview.
Add exam requirement
Total Credits
Total Credits:
50 Credits
Graduate School Application, G.P.A. 2.75, three
letters of recommendation, personal goal
statement, personal interview, submission of
either PPST or GRE results.
No Change
50 Credits
Assessment in the Major: School Psychology (M.S.Ed. & Ed.S.)
2006
Submitted by: Dr. Jacalyn Weissenburger, Program Director
Description of Methods
The following assessment tools and strategies were utilized to determine student
performance, progress, and levels of satisfaction in the major:
•
•
•
•
•
•
•
•
•
•
•
2004-2005 Assessment in the Major report outcomes
Faculty review/selection of program applicants
Transcript reviews
Criterion-referenced (The Evaluation Guide for School Psychology Practicum and
Internship) and developmental appraisal (School Psychology Developmental
Performance Appraisal) rubric scores for each experiential component of the
program
Student portfolio reviews
Faculty reviews of student performance, including academic progress,
experiential evaluations, and professional conduct
Student disposition reviews
Review of placement results
Student scores on the National Certification in School Psychology Exam (NCSP)
or Praxis II
Survey of interns regarding their perceptions of program competencies and
coursework
2006 NASP Review Report
1. PRAXIS II: Content Test Summary or National Association of School Psychologists
Exam Results
It is a program requirement to take the National Certification in School Psychology
(NCSP) exam (Praxis II National School Psychology Exam) used by the National
Association of School Psychologists as a requirement for national certification. The
National Association of School Psychologist’s passing score is 660. Further, Wisconsin’s
Department of Public Instruction has adopted a cut-off score of 660 or above for program
completers after August 31, 2007.
In addition to the total score, the NCSP or Praxis II exam also reports area scores
addressing Diagnosis and Fact-finding, Prevention and Intervention, Applied
Psychological Foundations, Applied Educational Foundations, and Legal/Ethical
Considerations. Students and interns can take the national certification exam at any point
during their graduate program.
1
Outcomes of the NCSP Results
In 2006, 91% of the students (10 out of 11 students) obtained scores of 660 or better. As
such, a majority met the passing criteria of NASP and Wisconsin’s Department of Public
Instruction. Further, all met or exceeded the criterion score (i.e., 620) set by Wisconsin’s
Board of Psychology.
Area scores indicate 77% obtained scores in the upper two quartiles in Diagnosis & Factfinding, 33% obtained scores in the upper two quartiles in Prevention & Intervention,
44% obtained scores in the upper two quartiles in Applied Psychological Foundations,
66% obtained scores in the upper two quartiles in Applied Educational Foundations, and
77% obtained scores in the upper two quartiles in Ethical & Legal.
Conclusions:
Overall results indicate UW-Stout’s School Psychology program is preparing students to
achieve the knowledge necessary to pass the NCSP test. In the past, NCSP mean area
scores and trend scores suggested a possible program need for additional emphasis on
Applied Educational Foundations. However, current area scores suggest recent
programmatic changes have led to increased knowledge in that area. Current results
suggest more emphasis may need to be given to preparing students in the areas of
prevention, intervention, and applied psychological foundations. However, because of the
limited sample size, NCSP data from future cohorts is needed before definitive
conclusions can be made regarding the need for other programmatic changes.
2. Program Disposition Review
Disposition reviews were conducted for 31 school psychology graduate students
(BENCHMARK II and BENCHMARK) in 2004 and 2005. Area scores range from 1
(unsatisfactory) to 2 (minimal) to 3(satisfactory) to 4 (proficient). Total scores range from
≤ 15 (unsatisfactory) to 32 (proficient).
Outcomes of the Disposition Review Results
As a group, the 2006 school psychology students displayed satisfactory to proficient
attitudes and behaviors within the program. No student earned composite scores
indicating minimal performance in 2006.
Conclusions:
Benchmark I and Benchmark II disposition review results did not indicate a need to
address any deficient disposition ratings in 2006.
2
3. Practicum and Internship Student Evaluations:
Graduate students in the school psychology program complete two supervised field
practica and one year-long internship. They are evaluated through the use of two
assessment tools: 1) The Evaluation Guide for School Psychology, and 2) The School
Psychology Developmental Appraisal. The Evaluation Guide uses a Likert-style format
ranging from 1 (needs improvement) to 5 (highly satisfactory). The Developmental
Appraisal evaluates students on a developmental continuum ranging from Stage 1
(observer) to Stage 4 (mastery).
Outcomes of the Practicum and Internship Student Evaluations
Practicum Evaluations
Results from the 2006 evaluations indicate practicum area scores (means) ranged from
4.14 (Practicum I: Direct Service) to 4.91 (Practicum II: Personal Characteristics) on the
5-point scale Evaluation Guide. Results indicate that practicum students earned above
satisfactory scores in each area (General; Personal Characteristics; Interpersonal
Relations; Communication & Consultation Skills; Supervisory Relationships; Assessment
Skills, Professional Growth, and Direct Service).
Mean scores on the Developmental Appraisal form indicated that although most
practicum students were between the Novice Stage and the Practitioner Stage, some
students were at the Practitioner Stage in the Knowledge and Performance competency
areas. Particular strengths were noted in the Data-Based Decision-Making and
Accountability domain (x = 2.74 to 3.03 or Practitioner) and the Information Technology
domain (x = 2.97 to 3.19 or Practitioner). Lower means in both the Knowledge and
Performance areas were indicated in the Cognitive/Academic domain (x = 2.4 to 2.42 or
Novice); Prevention, Crisis Intervention, & Mental Health domain (x = 2.29 to 2.37 or
Novice); Home/School/Community Collaboration domain (x = 2.47 or Novice); and
Socialization and Development of Life Skills domain (x = 2.32 or Novice).
Internship Evaluations
Results indicate internship area scores (means) ranged from 4.58 (Assessment Skills) to
4.75 (Supervisory Relationships) on the 5-point Evaluation Guide. Results indicate that
interns earned above satisfactory to highly satisfactory scores in each assessment area.
Mean scores on the Developmental Appraisal form indicated that most interns were near
or at the Practitioner Stage in the eleven Knowledge and Performance competency areas
at the conclusion of their internship year. Particular strengths were indicated in DataBased Decision-Making and Accountability (x = 3.00 to 3.17 or Practitioner),
Consultation & Collaboration (x = 3.00 or Practitioner), and Information Technology (x
= 3.37 or Practitioner) domains. Lower means occurred in Research and Program
Evaluation (x = 2.67 to 2.87 or near Practitioner), School & Systems Organization, Policy
3
Development, & Climate (x = 2.71 to 2.87 or near Practitioner), and Socialization and
Development of Life Skills (x = 2.72 or near Practitioner).
Conclusions:
Reviews of the Evaluation Guide and the Developmental Appraisal forms indicate
students met or exceeded the performance and developmental standards relative to their
practica and internship. No changes or program improvements are suggested by the
practica or internship evaluations.
4. Other Information
a. Portfolio Assessment (BENCHMARK TWO and THREE):
Students initiate and complete components of a portfolio over the course of the program.
The completed portfolio includes evidences or artifacts demonstrating professional
competence in all 11 DPI content guidelines (the same as NASP’s eleven domains of
school psychology training and practice). A partially completed portfolio demonstrating
competency in 7 out of the 11 domains is submitted to the program faculty for review
after completion of the M.S.Ed. and prior to admittance to the Ed.S. in School
Psychology Program. At BENCHMARK THREE, students resubmit their portfolio to
demonstrate competency in all 11 domains. Faculty members rate the portfolio materials.
Based on a review of the portfolio ratings, academic performance, and professional
conduct, a student is either recommended for, or denied admittance to, the Ed.S. in
School Psychology Program (for BENCHMARK II) or internship (BENCHMARK III).
Outcomes of the Portfolio Assessment
A review of the portfolio ratings in 2006 revealed all students produced satisfactory
portfolios at BENCHMARK TWO and THREE.
Conclusions:
Portfolio Assessment results do not point to a need for curricular or programmatic
changes.
b. Survey of Intern Results
As in previous years, school psychology interns were surveyed in 2006 to determine their
perceptions regarding the program’s coursework. Each intern is asked to respond to
questions on a 5-point Likert-style survey (1 = No Knowledge or Skill Gained to 5 = A
Lot of Knowledge or Excellent Skills Gained) to assess how much knowledge or skill the
students gained from each course. Further, interns are asked to identify the five most
important courses in terms of how helpful they were in preparing them for their
internship year. They also are asked identify which courses were not helpful or had
overlapping content.
4
Outcomes of the Intern Survey
Results from the spring of 2006 survey indicate students believed that they gained above
average to a lot of knowledge (x = 4.54) and skill (x = 4.5) in most program courses.
However, students indicated that they did not gain above average knowledge (x = 2.81)
or skill (x = 2.27) in the Advanced Psychology of Learning (PSYC-730) class. In the
narrative section, few students mentioned that they would have appreciated getting more
information on behavioral interventions, autism, and academic interventions. Several
interns reported that that the program was excellent, and most commented they believed
they were well prepared for the field of school psychology despite the recent faculty
turnover within the program.
Conclusions:
Intern Survey responses suggest a need for further consideration of the PSYC-730
requirement. Further, responses suggest a need for more curricular emphasis on
behavioral interventions, autism, and academic interventions.
c. Program Advisory Committee Results
The program faculty members meet with the School Psychology Program’s Advisory
Committee two times a year. Internal and external colleagues meet to discuss updates in
the program and solicit information from the committee about program needs and future
goals. In 2006, two meetings were held.
Outcomes of the Program Advisory Committee Results
Comments from the most current meeting indicated support of the current school
psychology program and its recent changes.
Conclusions:
No new goals or program changes were indicated based on the last Program Advisory
Committee meeting.
d. 2006 NASP Review Results
The program was reviewed by the National Association of School Psychologists in the
fall of 2006. As a result of the review, the program received “Conditional Approval.” The
report commended the school psychology program for its substantial changes to the
faculty workload, its effective advocacy for additional faculty, and the reduction in the
number of students admitted into the program. Further, the report indicated it was evident
that the program uses data in a systematic fashion to improve the quality of the program.
The report also pointed to a need for:
5
1. A more systematic approach to preparation in the areas of organization and
operation of schools and school systems.
2. A more systematic approach to prevention and crisis intervention at the systems
level.
3. Adding required, specific activities related to the development and
implementation of academic interventions for all students.
4. Adding a requirement that all students engage in systematic consultation with
teachers.
5. Expanding the rubrics for practicum/internship evaluation tools to differentiate
between the depth and breadth of knowledge and skills expected for practicum versus
interning students.
6. Collecting data across all candidates related to the assessment of measurable
positive outcomes for clients.
Outcomes of the NASP 2006
The report indicated a need for revisions to course, practicum, and internship
requirements. The review report also suggested a need for the additional content in the
areas of systems, crisis intervention, and prevention.
Conclusions
In response to the NASP report, several curricular modifications have been made or are
currently in process. In addition, the program met with a consultant (i.e., Andrea Canter)
in the summer of 2007 to discuss how best to implement changes in the program to meet
NASP standards. Prior to the visit by the consultant, the program revised SPSY-775
School Consultation in the spring of 2007 to meet current standards in the field. In
response to the NASP report and the recommendations of the consultant, SPSY 786
Diagnosis and Remediation is now under revision to reflect current requirements for
students to implement academic interventions and collect data demonstrating their effect
on Pre-K student learning. In addition, program faculty members are in process of
developing a course that would cover information on systems consultation, prevention,
and interventions.
6. Plan for Communicating Assessment Results
The school psychology program faculty meet on a monthly basis to discuss program
issues and plan for the future. This Assessment in the Major report will be disseminated
to all members of the program faculty, and the results will be discussed by the program
faculty in the program’s October meeting. Further, this report will be disseminated to the
Program Advisory Committee in the fall of the 2007-2008 academic year.
7. & 8. Conclusions and Action Plan to Improve Teaching and/or Advisement in the
School Psychology Program
6
A. Previous results indicated students were in need of more information about academic
interventions and educational foundations. As a result, increasing emphasis on academic
interventions and educational foundations has been included in the Diagnosis and
Remediation of Learning Disabilities (SPSY-768) class since the spring of 2004.
Although current PRAXIS II scores suggest these changes have increased student
knowledge in this area, an ongoing review of the need for information on academic
interventions and the educational foundations of the profession will occur during the
2007-2008 academic year. Further, a revision of SPSY-768 to reflect current practice and
needs in the areas of learning disabilities, academic assessments, and academic
interventions is underway, with anticipated implementation in the spring of 2008.
B. Results from the 2006 NASP review indicated the need for program changes to
increase student knowledge related to systems, crisis intervention, and prevention. The
report also reported the need for students to become competent in evaluating the
effectiveness of academic interventions. Although the 2006 PRAXIS II (NCSP) exam
results suggest students are now more knowledgeable in the areas of Applied Educational
Foundations, there appears to be a need to more thoroughly address the areas of
prevention, intervention, and psychological foundations. The faculty will collaborate
during the 2007-2008 academic year to develop a course and/or make course revisions.
C. Over several years, surveys of interns and alumni have consistently indicated students
believe two courses (PSYC-730 and PSYC-850) have lacked rigor and have not provided
substantive information about child/adolescent development and learning. The program
director met with the Chair of the Department of Psychology to address this concern in
the fall of 2005. After meeting with the Department Chair of the Department of
Psychology and after a subsequent meeting in the spring of 2007 with the course
instructors, a plan was developed to address this issue. Results from future surveys will
be considered to determine whether a program revision is needed.
D. Past NCSP (Praxis II) results indicated students were more likely to obtain lower than
average scores on the Applied Educational Foundations area of the Praxis II. Current
results indicate 2006 test takers were more likely to obtain lower scores in the areas of
Psychological Foundations, Prevention & Intervention. During the next year, the Program
Director and the faculty will consider taking the Praxis II to get a clearer picture of its
content. It is hoped that this assessment experience will inform the faculty about the
nature of the items designed to assess student knowledge in all five areas. After taking the
test, the Program Director and the faculty will convene to discuss potential course or
programmatic revisions.
Respectfully submitted,
Jacalyn W. Weissenburger, Ph.D.
Program Director, School Psychology
7
8
MS and EdS in Career and Technical Education
2007 Assessment Report
Prepared by
Howard D. Lee
Program Director
MS and EdS in Career and Technical Education
School of Education
225A Applied Arts
University of Wisconsin-Stout
August, 2007
Table of Contents
Page
1.
Outcomes of Previous Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.
Questions to be Answered from This Year’s Assessment . . . . . . . . . . . . . . . . . . 3
3.
Methods Used to Gather Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
4.
Number of Students Involved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
5.
Level of Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
6.
Data Analysis Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
7.
Analysis of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
8.
How Results Were Shared with Key Instructors . . . . . . . . . . . . . . . . . . . . . . . . . 7
9.
Plans for Improvements based on Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
List of Tables
Table 1: Survey Return Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Table 2: Survey Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Appendices
Appendix A: Minutes From Fall 2006 meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
Appendix B: Web Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Appendix C: Mailed Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2
1.
Outcomes of Previous Assessment
Results for the 2006 MS and EdS-Career and Technical Education Survey shows that
graduates are acquiring the professional skills they need to perform well in the educational
arena. This is especially true for using and applying research methods.
However, data of 2006 MS graduates showed weakness in leadership competencies.
They included the student’s need to:
1.
identify leadership skills
2.
explain leadership philosophy
3.
discuss leadership responsibility
4.
demonstrate the application to Total Quality Management Principles
Follow-up of EdS graduates also showed some weakness in leadership competencies
relating to career and career information. Information on these leadership competencies were
shared with content area staff, the advisory committee, and with key instructors. Students
were advised to enroll in EDUC-726 Administration (see Appendix A for meeting minutes).
In addition, methods and content approaches, and e-portfolio strategies in EDUC-726
Administration were explored.
2.
Questions to be Answered from This Year’s Assessment
Based on the assessment feed back recommendation, an impromptu committee was
formed to discuss leadership competencies on August 29, 2006, and met for a second
meeting on September 12, 2006. The Career and Technical Education program director,
requested substantive curricular input from Hector Cruz and Carolyn Barnhart, who were
teaching the Fall 2006, EDUC 726 Administration course. Both instructors felt it was proper
and appropriate to integrate leadership resources and assignments into course curriculum in
3
effort to improve student’s leadership skills and knowledge to meet the program
competencies. Appropriate references and textbooks were shared at these meetings, and an
actuation plan was determined.
At the conclusion of the Fall 2006, semester, the Career & Technical Education
graduate assistant met with both Dr. Cruz and Dr. Barnhart to identify the characteristics of
leadership taught in the two sessions. A survey was developed and reviewed by the program
director, and the course instructors, which employed a Likert psychometric scale to gauge
student responsiveness to the 25 characteristics contained within it (see Appendix B). The
questions were thus based on content delivered by the instructors that addressed the
leadership competencies for the MS and EdS in CTE program.
3.
Methods Used to Gather Data
A web survey was developed and used to gather data. It was felt that this would be
the best method to gather the data since students were across the stat, has options of pull
down menus, and can be used with animation and other interactive elements. A cover letter
was written and assurance of confidentiality was shared. A copy of the final survey used can
be found in Appendix C. Because of a low return rate a follow-up survey consisting of a
mailed cover letter and instrument was also mailed out. This produced better results.
4.
Number of Students Involved
The target audience of the survey was 24 or the 25 students enrolled in both sections
of EDUC-726 Administration course during the fall semester. One student living overseas
was excluded from the study because of inadequate mailing information.
4
5.
Level of Students
Because this is graduate course, all the students enrolled were graduate students. One
student was enrolled in the EdS in CTE program and the rest were enrolled in the MS
program.
6.
Data Analysis Techniques
The survey was mailed on April 10, 2007, and no returns were received any later than
April 30, 2007. Of the 24 students in Administration 726, Sections 001 & 900C, the
response rate was 33%, or 8 respondents. The surveys were anonymous but two follow-up
notices were sent to each student. Survey return rates are indicated in the following table:
Table 1: Survey Return Rates
Program
Population
Surveys returned
Return %
Actual % returned
MS & EdS-CTE
24
8
33%
33%
7.
Analysis of Data
The following pages shows the pattern of response. Mean score has been plotted with
standard deviation noted. Results indicate that graduates seem to have a good grasp of the
leadership content. Only two items were below 4.0; Coaching and the TQM Principle
Characteristics of “High quality is cheaper then low quality”. Participants also rated two
items very high: Item #4. “I explained my leadership Philosophy in the course” and Item #5
“There was dialogue on leadership responsibility in the course”.
5
Table 2: Survey Results
6
In summary it seems that data from the survey indicate that students enrolled in EDUC-726
Administration are learning the content on leadership and exceed the program competencies
in that area.
8.
How Results Were Shared with Key Instructors:
At the Spring program committee meeting, results of the survey were shared. Key
instructors, Carolyn Barnhart and Dr. Hector Cruz will be provided with this report.
9.
Plans for Improvements Based on Results:
Analysis results indicate that students enrolled in EDUC-726 Administration are
acquiring the knowledge they need to meet the EdS and MS leadership competencies. No
additional follow-up will be made with the leadership competencies.
7
Appendix A
Minutes From Fall 2006 Meetings
Special Meeting Minutes
Impromptu Committee to discuss Leadership Competencies
August 29th, 2006, 11 a.m. at Huff’s Room in the SMC
Howard Lee Presiding
Present:
Dr. Howard Lee
Dr. Bob Hendricks
Mr. Mark Fenton
Ms. Juliann Taylor
Ms. Jodi Olmsted
Ms. Amelia Fox
Dr. Hector Cruz
Meeting Purpose: Ideas to improve leadership skills by graduate students..
Howard reviewed the assessment in the major report and the four issues for improvement
according to the 2005 MS/EdS survey.
1)
2)
3)
4)
Identify Leadership Skills
Explain Leadership Philosophy
Discuss Leadership Responsibility
Demonstrate Application to TQM Principles
Administration 726 is ideally suited to present such curriculum but Howard requested input
from all instructors present regarding their ability to integrate other coursework and
assignments towards the effort.
The subjects of leadership style surveys, philosophy, reflection, and roles in leadership
should be emphasized.
Hector and Jodi suggested an additional 1-credit course called Intro to Administration aimed
at undergrads and some graduate students. Howard suggested that instructors integrate
leadership theory and concepts into current coursework.
Bob and Mark will recommend that students consider application of concepts learned to
leadership skills in their classes. Juliann will emphasize leadership more in assignments for
CTE-502.
Jodi suggested two curricular objectives that will assist instructors present in increasing
leadership awareness in current administration course curriculum. A meet with Howard,
8
Jodi, Hector, and Carolyn to discuss possible infusion of these concepts identified by Jodi
will be arranged.
Special Leadership Competency Meeting Minutes
Impromptu Administration/Leadership Committee called by H. Lee
September 12th, 2006, 11 a.m. at the Badger Room in the SMC
Present:
Howard Lee
Jodi Olmsted
Hector Cruz
Mark Fenton
Carolyn Barnhart
Juli Taylor
Amelia Fox
Meeting Purpose: Review administration/leadership content for CTE graduate students.
1.
The meeting started by reviewing the minutes of the CTE Assessment in the Major
meeting of August 29 2006.
2.
Jodi began by presenting textbooks in leadership used in her “Leadership for
Excellence” class. For leadership style assessment, she proffered Finding Your
Leadership Style by Jeffery Glanz (2002). Readers can work through exercises in
effort to determine their personal style and gain an understanding about it. For
administration guidelines, Jodi offered School Leadership: Handbook for Excellence
by Smith & Deal. She also suggested Teacher Leadership That Strengthens
Professional Practice by Charlotte Danielson.
3.
Hector referred to the Bowman & Deal book, Framing Organizations, and the four
lenses by which we view leadership. This refers to the Structural, Human Resources,
Political, and Symbolic (metaphysical) lenses.
4.
Carolyn and Jodi discussed the topic of face-to-face coursework versus on-line. Both
found satisfaction with on-line coursework but Jodi preferred direct interaction with
students, particularly for class debriefing. The issue of community development
between learners is a key to the success of leadership training.
5.
Jodi furthered the discussion by referring to a book titled Laws of Leadership by John
Maxwells, which contains 21 laws of effective leadership and worked effectively
when dissected into 3 parts taught in units of 7 laws each over 3 months time.
6.
Concerns over how to customize issues to individual student’s disciplines, how to
articulate them, and how to cover TQM was brought up by Hector. He also stressed
9
the need to teach the servant leadership model and to help identify the forces that
inhibit teacher leadership.
7.
Howard expressing a desire to have college faculty identify to the students what
leadership skills consist of, help them assess their own style, and further the idea that
everyone can be a leader in some capacity as an educator. A desirable outcome of
reinforcing leadership skills in the classroom would be for future teachers to develop
and understand of the topic, increase their participation as leaders, and recognize its
force in the workplace.
8.
A reference list of the text and reference books used in the Administration course is
noted below:
Bolman and Deal. (2003) Reframing Organizations.
San Francisco: Jossey-Bass.Lambert, L. (2004) Leadership capacity for lasting school
improvement.
Alexandria, VA: Association for Supervision and Curriculum Development.
9.
A reference list of the text and reference books used in the Leadership for Excellence
course is noted below:
Jeffrey Glanz- Finding your leadership style
John Maxwell-21 Laws of Leadership WORKBOOK
Robert Greenleaf-The Servant Leader
Smith & Piele-School Leadership
10
Appendix B
Web Survey
11
Appendix C
Mailed Survey
Circle your answer to the statements below.
Scale:
SD = Strongly Disagree; D = Disagree; U = Undecided; A = Agree; SA = Strongly Agree
Characteristics of Outstanding Leadership
1.)
Kouzes & Posner identified some characteristics of outstanding leadership. They include the following:
A. Sets clear standards…….…..…………………………
SD
D
U
A
SA
B. Expect the best…………..….………………………….
SD
D
U
A
SA
C. Pay attention……………….….………………………..
SD
D
U
A
SA
D. Tell the story……………….……………………………
SD
D
U
A
SA
E. Personalize recognition…….………………………….
SD
D
U
A
SA
F. Celebrate together………….…………………………..
SD
D
U
A
SA
G. Set the example…………….………………………….
SD
D
U
A
SA
A. Listening…………………………………………………
SD
D
U
A
SA
B. Visionary leadership……………………………………
SD
D
U
A
SA
C. Decision making………………………………………..
SD
D
U
A
SA
D. Coaching………………………………………………...
SD
D
U
A
SA
E. Being creative…………………………………………...
SD
D
U
A
SA
F. Problem solving…………………………………………
SD
D
U
A
SA
G. Presentation information………………………………
SD
D
U
A
SA
H. Empowering others…………………………………….
SD
D
U
A
SA
I. Positive reinforcement…………………………………..
SD
D
U
A
SA
J. Managing conflict………………………………………..
SD
D
U
A
SA
K. Facilitating meetings……………………………………
SD
D
U
A
SA
The course helped me identify leadership skills that are required of my
as a professional.
SD
D
U
A
SA
4.)
I explained my leadership philosophy in the course.
SD
D
U
A
SA
5.)
There was dialogue on leadership responsibility in the course.
SD
D
U
A
SA
A. High quality is cheaper than low quality……………...
SD
D
U
A
SA
B. People want to do good work………………………….
SD
D
U
A
SA
C. Quality problems are cross-functional………………..
SD
D
U
A
SA
2.)
3.)
6.)
Brainstorm Dynamics can be characterized by:
TQM Principles have the following characteristics:
12
D. Top management is ultimately responsible for
quality………………………………………………………...
SD
When completed, please return this survey in the self-addressed, stamped envelope.
13
D
U
A
SA
Assessment in the Major
M.S. Industrial/Technology Education
University of Wisconsin-Stout
Prepared by:
Dr. Brian K. McAlister, Program Director
2006-07
Methods used to Assess Learning Outcomes
The program assessment measure used was based on the MSITE program
competencies. The Budget, Planning, and Analysis office at the University of WisconsinStout conducts a one- and five-year follow-up study of university graduates (prior to this
year they surveyed one and three year graduates). The MSITE program exercised its
option to have a program specific survey included as part of this study. A survey
instrument was developed to determine MSITE graduate’s perceptions as to what level
their MS degree prepared them with the skills identified in the program competencies
(see appendix A). According to the approved competencies, a master educator graduating
from the MS in Industrial Technology Education program should be technologically
literate, pedagogically skilled, and able to initiate continued and reflective professional
self-improvement. A similar survey was used in 2002 but was limited to program
objectives related to pedagogical skills. A new revised instrument incorporated items
that reflected all of the program competencies was implemented in 2004 and has been
used in subsequent years that the follow-up studies have been done.
Number of Students Involved
Program Objective Survey
The number of participants surveyed each year was dependent on the number of
graduates for each respective year. Four out of the 19 students one-year graduates
surveyed (21.1%) returned completed instruments. Two out of 10 five-year grads
returned surveys while 2 out of 9 one-year alumni of the program responded. The one
and five year alumni survey data was compared to data collected two and four years ago.
Level of Students
The MSITE program survey was administered to students one and five years after
they have graduated from the program. It has now been administered three times. Data
will become even more meaningful as longitudinal data is collected over time.
Data Analysis Techniques
Program Survey Data Analysis
The MSITE program survey instrument was developed to determine MSITE
graduate’s perceptions as to what level their MS degree prepared them with the skills
identified in the program competencies (see appendix A). Alumni were asked to indicate
the level that they believed their master’s degree prepared them with the skills by circling
the number that corresponded with the following Likert-type scale: 1 = strongly disagree,
2=disagree, 3=agree, or 4=strongly agree. This format was chosen to force students to
make a choice and not to register an apathetic response. The most recent survey was
conducted during spring 2006 and was mailed to graduates of 2004 and 2000.
Descriptive statistics were used to report the findings.
2
Results
Program Objective Survey Results
Four out of 19 program specific surveys mailed to MSITE 2000 and 2004
graduates by the Budget, Planning and Analysis Office were returned for a 21% response
rate. Of those responding, two indicated that they were certified in a discipline other
than technology education prior to entering the program. One indicated that they held
technology education teacher’s license and one indicated that they held a baccalaureate
degree in a non-teaching technical area prior to entering the program. None of the
respondents entered the program with a BS in a non-teaching and non-technical area.
Graduates were asked to indicate the level to which they agree that their Masters
degree prepared them in each respective program objective area. Eight (38.1%) of the
items on the survey instrument had a mean rating of 3.5 or above where 3 = agree and 4 =
strongly agree. This is an increase over the five items (23.8%) that received this level of
rating in 2004. All eight of the items were ranked higher than they were two and four
years ago when a version of the survey instrument was last administered. The following
represent those program objectives that received a mean rating of 3.5 or above indicating
strong agreement that their MS degree prepared them with the respective skills.
MSITE Program Objective
12. Select materials and hardware that will actively
engage students in the learning process.
1. Translate technological and societal developments,
trends, issues, problems, opportunities and impacts into
meaningful learning experiences for students.
4. Articulate an in depth knowledge of technology
education, its history, philosophy, purpose, current status
and future.
10. Develop programs, courses and units of instruction.
11. Design teaching strategies and student learning
activities.
15. Articulate a conceptual understanding of technology
8. Advance the discipline by providing vision, leadership,
and professionalism.
17. Define, plan, model, test, propose and report findings
resulting from technical research and experimentation
* indicates that the item was not part of the survey during
the respective year that it was administered.
Year
2002
Mean
3.00
Year
2004
Mean
3.50
Year
2006
3.25
3.00
3.75
3.13
3.30
3.75
3.25
3.13
3.70
3.50
3.75
3.75
*
2.88
3.50
3.40
3.75
3.50
*
3.30
3.75
4.00
Students that graduated from the MSITE program feel very confident about their
preparation to develop lessons and activities and their abilities to select resources to
facilitate those lessons. They also feel as though they were given a very good
foundational knowledge of the subject matter. The area that has shown the greatest
improvement involves graduate’s competencies related to developing activities that stress
technological developments and their impacts on society.
3
Ten (47.6%) of the items on the survey instrument had a mean rating between 3.0
and 3.5 where 3 = agree and 4 = strongly agree. The following table includes those
program objectives that received a mean rating between 3.0 and 3.5 indicating agreement
that their MS degree prepared them with the respective skills. Six of the ten items reflect
an increase in their rating over the past.
MSITE Program Objective
2. Apply research findings and tried best practices in
learning situations with students.
16. Identify, gather, organize and synthesize technology
based information
20. Address the problems, issues, and trends that are
shaping the future of technology and schooling
21. Submit papers, develop rationales and create conceptual
frameworks as evidence of understanding history,
philosophy, purpose and structure of knowledge.
5. Continue to systematically acquire new knowledge
through action research, the study of promising practices,
serious reflection and constant evaluation.
6. Make useful contributions to the field through innovation
and the dissemination of promising practices.
7. Assist with the development of other professionals by
serving as a mentor to individuals and the profession.
9. Conduct program evaluations and compile reports,
complete with recommendations.
13. Supervise student teachers.
19. Conduct applied research for the improvement of
learning
* indicates that the item was not part of the survey during
the respective year that it was administered.
Year
2002
Mean
2.88
Year
2004
Mean
2.70
Year
2006
Mean
3.25
*
3.3
3.25
*
3.10
3.25
*
3.30
3.25
2.75
3.50
3.00
2.29
2.80
3.00
2.43
2.90
3.00
3.00
3.30
3.00
2.17
*
2.70
2.90
3.00
3.0
When you combine this data with the previous “strongly agree” category, it
indicates that 18 out of the 21 program goals were ranked above the “agree” range.
Students, overall, felt that the program prepared them to conduct research, apply findings
to their teaching and disseminate promising practices.
Three of the 21 items (14.3%) received a mean rating between 2.25 and 3.0 where
2 = disagree and 3 = agree. All three items were ranked lower than they were on
previous surveys. It is important to note that while some of these items received rankings
that fell toward the agree side of the scale (2.5-2.75), the fact that the average rank was
not higher indicate that there was some dissention among the graduates. These goals
deserve further attention.
It is interesting that most students indicate that they were prepared to apply
findings to inform their practice and to make useful contributions to the field through
innovation and the dissemination of promising practices, yet there is an indication that
4
some of them do not feel prepared to attend professional conferences, present papers, and
write articles for professional newsletters and journals. This seems like a contradiction.
MSITE Program Objective
14. Attend professional conferences, present papers, and
write articles for professional newsletters and journals.
3. Assess program effectiveness, design optimal programs
and implement contemporary program designs.
18. Study the structure of knowledge
* indicates that the item was not part of the survey during
the respective year that it was administered.
Year
2002
Mean
2.86
Year
2004
Mean
3.10
Year
2006
Mean
2.75
3.00
3.00
2.5
*
2.89
2.25
Only one of the 21 objectives received a mean ranking lower than 2.5. This
means that only one of the objectives had a mean ranking that would indicate a skew
toward disagree since 2.5 represents the midpoint between 2 = disagree and 3 = agree.
More graduates agreed they were prepared across all of the objectives than disagreed. It
should also be noted that six items were ranked below the midpoint last time this
instrument was administered to alumni. The overall trend indicates that ratings are going
up.
Dissemination of findings
Technology Education faculty meet weekly during the academic school year. This
report will be shared with them during our regularly scheduled meetings. Key findings
will inform a program of work for the next academic year.
This report will also be shared with SOE administration and the MSITE program
advisory committee. Feedback will be sought during upcoming meetings.
5
Plans for Improvement Based on results
Based on the findings of the one and five-year program specific follow-up survey
the MSITE program should to continue to:
• stress and reinforce the identified program strengths in preparing teachers
to develop programs, courses, and units of instruction that include
valuable learning experience and to select the resources essential to
actively engage students in them.
• provide opportunities for students to articulate understanding of
technology and to translate technological and societal developments,
trends, issues, problems, opportunities and impacts into meaningful
learning experiences.
• facilitate opportunities for students to advance the profession by
providing vision, leadership, and professionalism.
Based on the findings of the one and three-year program specific follow-up
survey the MSITE program should consider:
• integrating content knowledge into courses related to cognition and metacognition to improve students understanding of the structure of
knowledge.
• communicating these findings with the instructors of respective courses so
that they can make adjustments to their curriculum where needed. This
will help guide instructors to make changes to course content and
activities that need to be developed or enhanced. This will also insure that
instructors don’t make unnecessary changes to course content and
activities that are apparently working well.
Based on a more personal connoissurial evaluation, the MSITE program should
consider:
• program revisions that focus on moving courses on-line so that it can be
made available to a larger number of students across the state, country and
the world.
• aligning the program with state and national professional development
standards
While not all of the students in the MSITE program are pursuing teaching
certification, it does provide a valuable opportunity to collect data on the performance of
students. And while student teaching should not provide the only data source for the
MSITE program, it is too valuable of a source of data to ignore. It provides valuable
insights as to how students are progressing through the program. It should be noted that
some of the students are in the MSITE program specifically to qualify for a teaching
license and have no interest in completing the MS degree. But most see it as a great
opportunity to change careers and complete an advanced degree in the process.
6
Masters of Science in Industrial/Technology Education Program Survey
DIRECTIONS: Complete the following survey based on your experiences as a student in the Masters
of Science in Industrial/Technology Education Program at UW-Stout.
Which of the following best describes your background prior to entering the program:
(1)
(2)
(3)
(4)
Certified Technology Education Teacher.
Certified Teacher in another discipline.
Baccalaureate degree in a non-teaching technical area.
Baccalaureate degree in a non-teaching and non-technical area.
Please circle the number that corresponds to the level that you agree your master's degree education
prepared you with the following skills.
1 = Strongly Disagree
2 = Disagree
3 = Agree
4 = Strongly Agree
My educational experience at UW-Stout enabled me to…
1.
Translate technological and societal developments, trends, issues,
problems, opportunities and impacts into meaningful learning
experiences for students.
1
2
3
4
2.
Apply research findings and tried best practices in learning situations
with students.
1
2
3
4
3.
Assess program effectiveness, design optimal programs and implement
contemporary program designs.
1
2
3
4
4.
Articulate an in depth knowledge of technology education, its history,
philosophy, purpose, current status and future.
1
2
3
4
5.
Continue to systematically acquire new knowledge through action
research, the study of promising practices, serious reflection and
constant evaluation.
1
2
3
4
6.
Make useful contributions to the field through innovation and the
dissemination of promising practices.
1
2
3
4
7.
Assist with the development of other professionals by serving as a
mentor to individuals and the profession.
1
2
3
4
8.
Advance the discipline by providing vision, leadership, and
professionalism.
1
2
3
4
7
9.
Conduct program evaluations and compile reports, complete with
recommendations.
1
2
3
4
10. Develop programs, courses and units of instruction.
1
2
3
4
11. Design teaching strategies and student learning activities.
1
2
3
4
12. Select materials and hardware that will actively engage students in the
learning process.
1
2
3
4
13. Supervise student teachers.
1
2
3
4
14. Attend professional conferences, present papers, and write
articles for professional newsletters and journals.
1
2
3
4
15. Articulate a conceptual understanding of technology.
1
2
3
4
16. Identify, gather, organize, and synthesize technology based information.
1
2
3
4
17. Define, plan, model, test, propose and report findings resulting from
technical research and experimentation.
1
2
3
4
18. Study the structure of knowledge.
1
2
3
4
19. Conduct applied research for the improvement of learning.
1
2
3
4
20. Address the problems, issues, and trends that are shaping the future
of technology and schooling.
1
2
3
4
21. Submit papers, develop rationales and create conceptual frameworks as
1
2
evidence of understanding history, philosophy, purpose and structure of knowledge.
3
4
8
MSITE Program Objective
1. Translate technological and societal developments,
trends, issues, problems, opportunities and impacts into
meaningful learning experiences for students.
2. Apply research findings and tried best practices in
learning situations with students.
3. Assess program effectiveness, design optimal programs
and implement contemporary program designs.
4. Articulate an in depth knowledge of technology
education, its history, philosophy, purpose, current status
and future.
5. Continue to systematically acquire new knowledge
through action research, the study of promising practices,
serious reflection and constant evaluation.
6. Make useful contributions to the field through
innovation and the dissemination of promising practices.
7. Assist with the development of other professionals by
serving as a mentor to individuals and the profession.
8. Advance the discipline by providing vision, leadership,
and professionalism.
9. Conduct program evaluations and compile reports,
complete with recommendations (3.00)
10. Develop programs, courses and units of instruction.
11. Design teaching strategies and student learning
activities.
12. Select materials and hardware that will actively
engage students in the learning process.
13. Supervise student teachers.
14. Attend professional conferences, present papers, and
write articles for professional newsletters and journals.
15. Articulate a conceptual understanding of technology
16. Identify, gather, organize and synthesize technology
based information
17. Define, plan, model, test, propose and report findings
resulting from technical research and experimentation
18. Study the structure of knowledge
19. Conduct applied research for the improvement of
learning
20. Address the problems, issues, and trends that are
shaping the future of technology and schooling
21. Submit papers, develop rationales and create
conceptual frameworks as evidence of understanding
history, philosophy, purpose and structure of knowledge.
Year
2002
Mean
3.25
Year
2004
Mean
3.00
Year
2006
Mean
3.75
2.88
2.70
3.25
3.00
3.00
2.5
3.13
3.30
3.75
2.75
3.5
3.0
2.29
2.8
3.0
2.43
2.9
3.0
2.88
3.4
3.5
3.00
3.30
3.0
3.25
3.13
3.70
3.50
3.75
3.75
3.00
3.50
4.0
2.17
2.86
2.70
3.10
3.0
2.75
*
*
3.5
3.3
3.75
3.25
*
3.3
3.5
*
*
2.89
2.90
3.25
3.0
*
3.10
3.25
*
3.30
3.25
9
10
Assessment in the Major ~ Master’s in Education ~ 2006-2007
Dr. Amy L. Gillett, Program Director
Introduction
There are two programs that are offered under the umbrella of our Master’s in Education (MS in
Ed.) program: online and on campus. Both program options are 30 credits minimum.
The online program allows students to strengthen their professional skills and graduate with a
master’s in education with a professional development focus. The online master’s in education is
offered to students in a cohort format. Each student is expected to follow the cohort to
completion, which is a two-year rotation, typically culminating with their Plan B thesis. There
are currently 110 students at various stages within the cohort online program.
The other program, our on campus program, has two options. The first option is the professional
development focus, which is the same offered to the online students, though the on campus
students have the option of taking the coursework here on campus or through the online format.
We currently have 41 in the professional development on campus program. The second option
for the on campus students is to either receive additional teacher certification(s) or to receive an
initial teacher license in the following programs: Early Childhood Education, Family and
Consumer Sciences Education, Science Education, Special Education (CD or Cross-Categorical).
Those who already have a teaching license, depending on their initial license, can enroll in the
Reading Education and Early Childhood Special Education licensure programs. Each licensure
or certification is additional credits beyond the required master’s in education 30 credit
minimum, though every attempt is made to provide a streamlined program, yet meet the
Wisconsin Department of Instruction requirements.
Current Enrollment
Online Cohorts ~ 110 in Professional Development
On Campus ~ 61
20 in certification programs
3 in Early Childhood Education
1 in Family and Consumer Sciences Education
10 in Special Education
1 in CD
9 in Cross-Categorical
6 in Early Childhood Special Education
41 in Professional Development
Inquiries to the program
Most of the inquiries to the program come via email. For instance, there were over 300 emails
from prospective students (some of them were second and third replies) between January and
November of this year (2007). In addition, there have been some letters of request for
information on our programs, most of which have interest in either the certification options or the
online Professional Development program. There have been numerous phone conversations with
prospective students, as well as face-to-face meetings, along with transcript analyses. Each
student has unique questions and situations; there is no “one size fits all” for the MS in
Education program. Recruitment for the online program has gone well and we are well over our
projected targets. The on campus program is currently below target, but tends to pick up each
spring according to the patterns reported by previous program directors. Because of the lower
enrollments for the on campus program options, we have had to cancel course(s) in fall and this
coming spring (i.e. EDUC 729 Introduction to Educational Research). Part of the impact on this
particular class has been that School Counseling went through a revision of their program and
they no longer require this course prior to enrollment in EDUC 740 Research Foundations. Since
nearly two-thirds of the previous enrollment for EDUC 729 was in School Counseling, it is
understandable that the course did not have adequate enrollment.
Assessment Plan and Evaluation
Students have to pass three benchmarks in order to graduate from the program. The first
benchmark is the entrance criteria to be admitted to the program. Initial application is sent to the
Graduate School where the staff reviews the application for its completeness. Once the
application is complete, it is sent to the program director. Grade point average, transcript
analysis, letters of support (2 minimum), and specific education application are reviewed. The
program director determines whether the applicant has met the minimum requirements for the
program. For instance, if a student is applying for the master’s in education for the online
program and has indicated that he/she wishes to receive an initial teaching license, the student is
contacted to receive additional information and informed that initial licensure is only available
on campus, not in the online format. For the master’s in education, a minimum undergraduate
GPA is 2.75. If the applicant has below a 2.75 GPA, but has graduate coursework indicating
successful completion, that student could be admitted on probation. The student’s transcript(s)
are reviewed again to ensure adequate background preparation for the degree. Only one student
has not met all of the criteria for admittance since Dr. Gillett has become program director,
though several students were admitted provisionally due to language deficiencies.
Benchmark I
Earned
bachelor’s
degree
Undergraduate
GPA of 2.75
or higher
Satisfactory
letters of
support (2)
Description of
educational
goals
Transcript
review
On Campus ~ certification
program
On Campus ~
Professional Development
Online ~ Professional
Development
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Evaluated by Program
Director
Benchmark II
Professional
By instructor of
goals reviewed EDUC 782 Instructional
Analysis
Completion of Required for all
PRAXIS II
certification programs ~
cut scores set by WI DPI
Transcript
Updated each year by PD
review and
GPA
Satisfactory
Required for all
interview
certification programs ~
faculty in cert. area
Satisfactory
Required for all
portfolio
certification programs ~
review
faculty in cert. area
Benchmark III
Satisfactory
Not required for
portfolio
Professional
review
Development
Exit
Required for all
interview
certification programs ~
faculty in cert. area
Transcript
Evaluated by Program
review and
Director
cleared for
graduation
By instructor of
EDUC 782
Instructional Analysis
Not required for
Professional
Development
Updated each year by
PD
By instructor of
EDUC 782
Instructional Analysis
Not required for
Professional
Development
Updated each year by
PD
Not required for
Professional
Development
By instructor of EDUC
790 Professional
Portfolio
Not required for
Professional
Development
By instructor of EDUC
790 Professional
Portfolio
Evaluated by Program
Director
Evaluated by Program
Director
Not required for
Professional
Development
Evaluated by Program
Director
Not required for
Professional
Development
Evaluated by Program
Director
The second benchmark for the online cohorts is an instructor review of their professional goals
for the Professional Development focus in the Instructional Analysis course (EDUC 782), The
coursework is designed to have students develop their professional objectives for their selective
coursework in the master’s program. They would then file a master’s program plan with the
program director after successfully completing this course. There has been inconsistency with
the course instructors, which has resulted in incomplete data to determine which students have
actually completed this second benchmark for the on campus and online cohort students in the
Professional Development option.
For the on campus certification students, for the second benchmark they are required to complete
the PRAXIS II exam and complete an interview and portfolio review with faculty before they are
admitted into the final stage of their program, which is the capstone course, their student
teaching experience (i.e. SPED 481). Because of the nature of the company administering the
PRAXIS II, scores are provided by undergraduate major and there is no feasible way to factor
out those who are working toward their degrees at the master’s level at this point in time. Those
results have been reported by the various undergraduate program directors in their assessment
reports (i.e. Kari Merritt for ECE).
The third benchmark for the online Professional Development cohort students is a formal
transcript review, portfolio evaluation, and meeting the exit criteria for the program. The
transcript evaluation has taken place for all students who have applied for graduation. Several
students have been missing coursework or have not completed the research requirement (i.e.
EDUC 735 or EDUC 770), and have been notified they are ineligible to graduate until those
requirements have been met. The portfolio evaluation has not been taking place, but efforts are
now underway to clear up the inconsistencies and making sure all students are aware of this
requirement.
The third benchmark for the on campus students who are doing Professional Development is the
same as for the online cohort students. For the certification students (n = 20), they are required to
do an exit interview and portfolio evaluation prior to being cleared for graduation. All of those
students who have applied for graduation have been cleared through this process.
Because of the inconsistencies with instructors of the core master’s in education courses and
inconsistencies and switches in program directors (5 turnovers in the past 6 years), there are
meetings scheduled for January to review the entire assessment process and program to
streamline the process, make sure that inconsistencies are dealt with, and that more accurate
assessment data can be reported in the future. Without appropriate data, it is difficult to write an
assessment report. That falls squarely on my shoulders and I take responsibility for not having
the data to report. Measures are being taken to address this matter. If these issues can not be
satisfactorily addressed, I will resign as program director to someone more capable and return to
my teaching duties.
Program Assessment Report and Plan
for the
Bachelor of Science Degree Program
with a Major in
Career, Technical Education and Training
Juli Taylor
Program Director
School of Education
University of Wisconsin-Stout
Menomonie, Wisconsin
October 2007
For Year 2006 Calendar Year
1
Overview
The purpose of the Assessment in the Major report is to provide a framework for assessment in
the Bachelor of Science in Career and Technical Education and Training (CTET) program, to
explain how assessment is accomplished, and to identify the outcomes of program assessment
efforts.
The most recent one- and five-year follow-up study data relating to general education program
objectives was included in last year’s assessment report. This information does not provide
valuable data relating to the core/required program courses since most CTET program students
complete the majority of their general education coursework at other institutions and transfer it in
to UW-Stout and the program. The Bachelor of Science Degree with a major in Career,
Technical Education and Training (CTET) has three program outcomes. An alignment of
program outcomes with program coursework and Wisconsin Technical College System (WTCS)
teacher education requirements for certification was updated and approved 3/9/04. Therefore, a
program specific follow-up survey was developed based on core competencies of the program.
The academic year 2005-2006 was the first year for the implementation of the program specific
follow-up study. No graduate follow-up study data was received in 2006.
In 2005/2006, the BS in CTET program advisory committee was re-established. The current
membership of the committee includes:
Jason Bergeron
Brenda Thalacker
Andrea Schullo
Sandy White
Linda Young
Urs Haltinner
Jodi Olmsted
Wendy Dittmann
Mark Fenton
David Johnson
Brian Bergquist
Kim Zagorksi
Student
Professional in the field (CVTC)
Professional in the field (WITC)
Continuing Education/Outreach Services
Admissions
SOE – Marketing and Business
SOE – CTE
CTEM - Management
CTEM – Business
CTEM – Op, Const, & Mgmt
CHD – Hospitality & Tourism
CAS – Social Science
2
Program Assessment Plan
In an effort to improve the quality of the Bachelor of Science Degree program with a major in
Career, Technical Education and Training, a two-year assessment plan is being proposed. Items
to assess have been developed by reviewing the University, program, and WTCS competencies,
by reviewing the one and three year follow-up studies, meetings with the advisory committee
and a review of past program assessment efforts. While the items identified may not address
every facet of the program, items being proposed should address large impact items that have a
long-term effect to guarantee and monitor quality. Priorities may change each year as new
situations arise.
2006 Calendar Year
Assessment Area: Program Competencies
Strategy to Address the Concern: The program competencies and alignment matrix will be
reviewed and validated by the program advisory committee. If the program advisory committee
recommends that the competencies be updated or curricular changes be made, a task analysis
will be undertaken with past program graduates and supervisors of the graduates.
Results: The program advisory committee approved the program competencies matrix and did
not recommend a task analysis be conducted at this time. Competencies will be reviewed again
in 2008.
Assessment Area: Student Dispositions
Strategy to Address the Concern: As other programs in the School of Education already assess
teacher candidate student dispositions, so must the CTET program. The SOE student disposition
form was piloted in spring 2007. At its spring 2007 meeting, the CTET Advisory Committee
endorsed its use. Beginning fall 2007, the disposition form may be used in any of the seven
Wisconsin Technical College System certification courses. Students are informed of this
program change on the program web site and in individual course syllabi. Disposition forms will
be submitted to the School of Education beginning fall 2007.
Assessment Area: Online Cohort Student Learning
Data on student learning in the new online cohort group will need to be gathered as the online
cohort begins in summer 2006.
Strategy to Address the Concern: Critical Incident Questionnaires will be designed to be used as
formative evaluations throughout each course. A summative course evaluation to be used at the
completion of each online course will also be created. Formative and summative data from the
new online cohort group will be reported with the 2006/07 assessment report.
3
Results:
The individual course evaluations are still being combined in aggregate form for analysis.
Completion of online cohort courses is now tracked at the individual student level. In addition, a
study of CTET online learner engagement was conducted in spring 2007. Results are as follows:
Twenty seven survey responses were received. Respondent participation by length of cohort
participation was:
Respondents by Length of Participation in Cohort
1 Semester
2 Semesters
(began spring 2007)
(began fall 2006)
Number Percent Number Percent
Respondents
2
7
6
22
(n=27)
3 Semesters
(began summer 2006)
Number
Percent
19
70
Online Cohort Respondents by Gender
Gender Number Percent
Male
12
44
Female 15
56
When asked to respond to, “My instructors influence my level of engagement in online
learning in the following ways…” This was followed by seven statements which participants
were asked to respond to on a four-point scale, with 1 being “strongly disagree”, 2 “disagree”, 3
“agree” and 4 “strongly agree”.
The four statements receiving the highest level of agreement (agree, strongly agree) were:
• They have high expectations of me (100%)
• They want me to succeed (100%)
• They challenge me to think in depth about topics (100%)
• I am more willing to participate in class when I feel my instructor respects me (100%).
The two statements receiving the lowest level of agreement were:
• They can change how I feel about a subject (89%) and
• They view me as a colleague (76%).
Another question asked participants to complete the statement, “When I’m engaged in online
learning, these things are happening in the online environment”. This was followed by 13
scaled statements.
The two statements receiving the highest level of agreement (agree, strongly agree) were:
• There is an open environment in the online classroom so every voice can be heard (96%)
• I can apply what I’m learning (92%)
The two statements receiving the lowest level of agreement were:
• I have a say in how the class is structured (20%)
• There’s “hands-on” learning (67%)
4
Cohort participants were asked, “I influence my own level of engagement in online learning
by doing the following”. This was followed by 8 scaled statements.
The statements receiving the highest level of agreement (agree, strongly agree) were:
• I am accountable for my learning (100%)
• I try and stay focused even when I don’t like the class (92%)
• I ask questions so I understand the material (89%)
• I take ownership for what happens in online group work (88%)
The statement receiving the lowest level of agreement was:
• I help others so I learn more (74%)
Overall, based on responses from participants in this study, it appears that what is meaningful to
online CTET students is that instructors have high expectations of them, that instructors want
them to succeed, that they are challenged to think in depth about topics, and that instructors
respect them. Respondents indicated that instructors influence learner engagement in online
environments through active participation throughout the course, effective and timely feedback,
contributing to and expanding thought processes through feedback on assignments and
discussion postings, and by making themselves available to learners.
Students reported that they are more engaged in online learning when there is an open online
environment where every voice can be heard and when they can apply what they are learning.
They are also more engaged online when they have access to course content through a variety
electronic formats and when they interact and communicate with other learners in the course.
Students indicated they influence their own level of engagement online by staying focused,
asking questions, taking ownership in group work, and being accountable for their own learning.
They are engaged online when they stay organized and on task, log on to the course frequently to
keep up with readings and discussions, and read and conduct additional research to supplement
and reinforce the online course content. Moreover, they apply what they are learning to other
classes and to their work and life as well as discuss course content and new learning with other
learners, colleagues, and other individuals not participating in the online class.
It seems apparent that the level of engagement in online learning is deeply rooted in the
“instructor to student” and “student to student” relationships that develop online in addition to
the quality and timeliness of communication and feedback that is nurtured by those relationships.
Assessment Area: On-Campus and Weekend Cohort Student Learning
The CTE-408 Student Teaching performance evaluation form continues to be used by on-site
mentors and University supervisors. This performance evaluation is used during the capstone
course in the program. Data from this capstone experience provides insights about student
learning in core/professional program courses and application of knowledge and skills to the
teaching environment.
5
Strategy to Address the Concern: The CTET Program Advisory Committee reviewed and
approved the student teaching performance evaluation form on April 27, 2006. Results from fall
2005 (pilot), spring 2006, and fall 2006 student teachers are as follows:
Student teachers showed improved performance in all components of planning and preparation
from fall 2005 and spring 2006 to fall 2006.
Domain I: Planning and Preparation Elements
1. Demonstrating knowledge of
content/pedagogy
2. Demonstrating knowledge of students
3. Selecting instructional goals
4. Demonstrating knowledge of resources
5. Designing coherent instruction
6. Assessing student learning
Fall 05
Spring 06
Fall 06
3.48
3.57
3.52
3.45
3.5
3.59
3.32
3.26
3.44
3.49
3.19
3.29
3.78
3.58
3.70
3.69
3.64
3.66
Domain I: Planning and Preparation Elements
3.8
3.7
3.6
3.5
3.4
Fall 05
Spring 06
Mean Scores 3.3
Fall 06
3.2
3.1
3
2.9
2.8
1. Demonstrating
knowledge of
content/pedagogy
2. Demonstrating
knowledge of
students
3. Selecting
instructional goals
4. Demonstrating
knowledge of
resources
5. Designing
6. Assessing student
coherent instruction
learning
Student teachers showed improved performance in all components of the classroom environment
from spring 2006 to fall 2006.
Domain II: The Classroom Environment
1. Creating an environment of respect and
rapport
2. Establishing a culture for learning
3. Managing classroom procedures
4. Managing student behavior
5. Organizing physical space
Fall 05
Spring 06
Fall 06
3.63
3.54
3.48
3.46
3.63
3.26
3.41
3.13
3.28
3.62
3.82
3.73
3.40
3.52
3.67
6
Domain II: The Classroom Environment
4.00
3.50
3.00
2.50
Mean Scores
2.00
Fall 05
Spring 06
Fall 06
1.50
1.00
0.50
0.00
2. Establishing a
1. Creating an
culture for learning
environment of
respect and rapport
3. Managing
classroom
procedures
4. Managing
student behavior
5. Organizing
physical space
Student teachers showed improved performance in all components of instruction from fall 2005
and spring 2006 to fall 2006.
Domain III: Instruction
1. Communicating clearly and accurately
2. Using questioning and discussion
techniques
3. Engaging students in learning
4. Providing feedback to students
5. Demonstrating flexibility and
responsiveness
Fall 05
3.38
Spring 06
3.47
Fall 06
3.61
3.38
3.35
3.56
3.09
3.22
3.40
3.63
3.69
3.77
3.62
3.22
3.77
Domain III: Instruction
4
3.5
3
2.5
Mean Scores
Fall 05
2
Spring 06
1.5
Fall 06
1
0.5
0
1. Communicating
2. Using questioning 3. Engaging students 4. Providing feedback
in learning
to students
clearly and accurately
and discussion
techniques
5. Demonstrating
flexibility and
responsiveness
7
Student teachers showed improved performance in all components of professional
responsibilities from fall 2005 and spring 2006 to fall 2006.
Domain IV: Professional Responsibilities
1. Reflecting on teaching
2. Maintaining accurate records
3. Contributing to instruction
4. Growing and developing professionally
Fall 05
3.48
3.60
3.71
3.74
Spring 06
3.29
3.29
3.56
3.48
Fall 06
3.74
3.73
3.82
3.74
Domain IV: Professional Responsibilities
3.90
3.80
3.70
3.60
Fall 05
3.50
Spring 06
Mean Scores
3.40
Fall 06
3.30
3.20
3.10
3.00
1. Reflecting on teaching
2. Maintaining accurate
records
3. Contributing to instruction
4. Growing and developing
professionally
GOAL FOR 2007: CTET student teaching evaluation forms will be entered into Datatel in
the School of Education for unit and program comparative purposes.