SOA - IT - IT Division
Student Outcomes Assessment Plan
Instructional Technology Program
Division of Instructional Technology
University of Northern Iowa
MA in Instructional Technology
I. Divisional Philosophy of Outcomes Assessment
The Instructional Technology Division is a member of the University of Northern Iowa’s
College of Education, Curriculum and Instruction Department. The division offers two
Masters of Arts degrees, Instructional Technology and Performance and Training
Technology.
Our Instructional Technology masters program is unique in that the focus is on the design
and implementation of technology within a K-12 setting. This environment includes both
students and staff development. As a part of our curriculum, students engage in designing
and developing products for use by their audiences as well as understanding the nature of
their prospective audiences and what role learning outcomes play in any setting. The
focus is on the application of technology tools to the learning process in the classroom or
in staff development.
The division has aligned its programs to the Association for Educational Communication
and Technology’s Educational Communication and Information Technology (ECIT)
standards (available at http://www.aect.org). Division faculty uses each standard’s
indicators as objectives for all courses, which can be found on the standardized course
syllabi. This approach not only demonstrates the deep alignment with national standards
but also presents students with a common language and a cohesive approach to the goals
and objectives of their coursework. Faculty members are able to gather data based upon
standards attainment due to their use of rubrics for assessing their students.
II.
Outcomes and Competencies
Iowa Board of Regents requires UNI departments and divisions to complete Student
Outcomes Assessments. The Department of Curriculum and Instruction determined that a
plan for all the masters programs would be similar. The Curriculum and Instruction
Outcomes Assessment Plan assesses the common elements of these programs. The major
areas that are assessed are:
 Leadership – Our graduates have the knowledge base and skills to assume
leadership roles in their respective professional career areas.
 Scholarship – Our graduates possess the skills to investigate and evaluate current
and proposed professional practices.
 Understanding of Theory and Research – Our graduates have an understanding of
the theory and research base underlying current professional practices.
 Application – Our graduates apply theory and research evidence to the solution of
practical problems.
Page 1 of 10
SOA - IT - IT Division
A review of the Division's mission statement reveals that it is based upon the Department
of Curriculum and Instruction’s Outcomes Assessment Plan in all four areas:
Student Outcomes Assessment
Components
Instructional Technology Mission Statement
Leadership
To prepare professionals who demonstrate
capable performance and insightful
leadership,
Scholarship
and to provide advanced study and professional growth
opportunities for practicing professionals.
Understanding Theory and
Research
Application
Based upon research and emerging
technologies,
the Division of Educational Technology is committed to:
•Advancing the study of effective learning techniques
•Designing instruction that integrates appropriate
application of technology •Advancing message
development
knowledge and skills
•Advocating effective integration of
technology to support learning
The alignment of the Instructional Technology Standards with the Curriculum and
Instruction Outcomes Assessment Plan components is demonstrated in the following
table:
Components
Leadership
Scholarship
Understanding Theory and
Research
Application
Standard Indicators
4.0.1, 4.1.1, 4.2.1, 4.3.1, 4.4.1, 5.4.1
1.1.1.a, 1.1.2.b, 1.1.4.b, 1.1.5.b, 1.2.a, 1.4.b, 2.0.2 (all
Advanced ECIT standards)
1.1.b, 1.1.c, 1.1.1.a, 1.1.1.b, 1.1.1.c, 1.1.1.d, 1.3.c, 1.3.d,
1.4.a, 1.4.b, 1.4.c, 2.0.1, 2.0.3, 2.0.4, 2.0.5, 3.1.1, 3.2.1,
3.3.3, 3.4.5
1.1.a, 1.1.2.a, 1.1.2.b, 1.1.3.a, 1.1.3.b, 1.1.4.a, 1.1.5.a,
1.1.5.b, 1.1.5.c, 1.2.a, 1.2.b, 1.2.c, 1.3.a, 1.3.b, 2.0.2, 2.0.6,
2.0.7, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.1, 2.2.2, 2.2.3, 2.2.4,
2.3.1, 2.3.2, 2.3.3, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.4.5, 2.4.6,
2.4.7, 3.1.2, 3.3.1, 3.3.2, 3.4.1, 3.4.2, 3.4.3, 3.4.4, 5.1.1,
5.2.1, 5.3.1
The division has aligned its programs to the following Association for Educational
Communication and Technology’s Educational Communication and Information
Technology (ECIT) standards and associated indicators. Division faculty use the
standard’s indicators as objectives for all courses, which can be found on the standardized
course syllabi. The indicators define what the student will be able to perform at the
completion of the program.
Page 2 of 10
SOA - IT - IT Division
Design:
1.1 Instructional Systems Design
1.1.a Utilize and implement design principles which specify optimal conditions
for learning.
1.1.b Identify a variety of instructional systems design models and apply at least
one model.
1.1.c Identify learning theories from which each model is derived and the
consequent implications.
1.1.1 Analyzing
1.1.1.a Write appropriate objectives for specific content within their area(s) of
preparation.
1.1.1.b Analyze instructional tasks and content.
1.1.1.c Categorize objectives within their area(s) of preparation (e.g., cognitive,
affective, psychomotor).
1.1.1.d Compare and contrast curriculum objectives for their area(s) of
preparation with federal, state, and/or professional content standards.
1.1.2 Designing
1.1.2.a Create a plan for a sub topic of their content area(s) (e.g., a thematic unit,
a text chapter, an interdisciplinary unit) to demonstrate application of the
principles of macro-level design.
1.1.2.b Create instructional plans (micro-level design) throughout their program
preparation and field experience(s).
1.1.3 Developing
1.1.3.a Produce instructional materials relevant to their content area(s) which
require the use of multiple media (e.g., computers, video, projection).
1.1.3.b Demonstrate personal skill development with at least one: computer
authoring application, video tool, or electronic communication application (not
telephone).
1.1.4 Implementing
1.1.4.a Use instructional plans and materials which they have produced in
contextualized instructional settings (e.g., practica, field experiences, training).
1.1.5 Evaluating
1.1.5.a Utilize both traditional and alternative assessment measures to determine
the adequacy of learning and instruction.
1.1.5.b Demonstrate the use of formative and summative evaluation within
practice and contextualized field experiences.
1.1.5.c Demonstrate congruency among goals/objectives, instructional strategies,
and assessment measures.
1.2 Message Design
1.2.a Apply known principles of attention, perception, and retention to the
selection of media for macro- and micro-level design of instruction.
1.2.b Apply known principles of attention, perception, and retention to the
development of instructional messages specific to the learning task.
1.2.c Understand, recognize and apply basic principles of message design in the
development of a variety of communications with their learners.
1.3 Instructional Strategies
1.3.a Select instructional strategies appropriate for a variety of learner
characteristics and learning situations.
1.3.b Identify at least one instructional model and demonstrate appropriate
contextualized application within practice and field experiences.
1.3.c Analyze their selection of instructional strategies and/or models as
influenced by the learning situation, nature of the specific content, and type of
learner objective.
1.3.d Select motivational strategies appropriate for the target learners, task, and
learning situation.
Page 3 of 10
SOA - IT - IT Division
1.4 Learner Characteristics
1.4.a Identify a broad range of observed and hypothetical learner characteristics
for their particular area(s) of preparation.
1.4.b Describe and/or document specific learner characteristics which influence
the selection of instructional strategies.
1.4.c Describe and/or document specific learner characteristics which influence
the implementation of instructional strategies.
2.0.1 Select appropriate media to produce effective learning environments using
technology resources
2.0.2 Select appropriate technological tools based on research and evaluation for
developing effective instructional and professional products.
2.0.3 Use appropriate analog and digital productivity tools to develop
instructional and professional products.
2.0.4 Apply instructional design principles to select appropriate technological
tools for the development of instructional and professional products.
2.0.5 Apply appropriate learning and psychological theories to the selection of
appropriate technological tools and to the development of instructional and
professional products.
2.0.6 Develop and implement evaluation strategies and techniques for assessing
effectiveness of instructional and professional products.
2.0.7 Implement evaluation methods and techniques for revising and updating
instructional and professional products.
2.1 Print Technologies
2.1.1 – Develop instructional and professional products using a variety of
technological tools to produce text for communicating information.
2.1.2 – Produce print communications (e.g., flyers, posters, brochures,
newsletters) combining words and images/graphics using desktop publishing
software.
2.1.3 - Use presentation application software to produce presentations and
supplementary materials for instructional and professional products.
2.1.4 - Produce instructional and professional products using various aspects of
integrated application programs.
2.2 Audiovisual Technologies
2.2.1 Apply principles of visual and media literacy for the development and
production of instructional and professional materials and products.
2.2.2 Apply development techniques such as storyboarding and or scriptwriting
to plan for the development audio/video technologies.
2.2.3 Use appropriate analog and digital video equipment (e.g., camcorders,
video editing) to prepare effective instructional and professional products.
2.2.4 Use appropriate analog and digital audio equipment to produce instructional
and professional products.
2.2.5 Use a variety of projection devices with appropriate technology tools to
facilitate presentations and instruction.
2.2.6 Use a variety of projection systems to facilitate presentations and
instruction.
2.3 Computer-Based Technologies
2.3.1 Synthesize content information in order to design and produce audio/video
instructional materials which use computer-based technologies.
2.3.2 Use theory and research to design, produce, and use digital information
with computer-based technologies.
2.3.3 Apply principles of learning theories to produce computer-based
instructional activities/environments.
2.3.4 Use imaging devices (e.g., digital cameras, video cameras, scanners) to
produce computer-based instructional materials.
Page 4 of 10
SOA - IT - IT Division
Utilization:
Management:
Evaluation:
2.4 Integrated Technologies
2.4.1 Use hypermedia authoring tools to create effective hypermedia instructional
materials or projects.
2.4.2 Develop and prepare instructional materials and products for various
distance education delivery technologies.
2.4.3 Combine electronic and non-electronic media to produce instructional
materials, presentations, and products.
2.4.4 Use telecommunications tools such as electronic mail and browsing tools
for the World Wide Web to develop instructional and professional products.
2.4.5 Develop effective Web pages with appropriate links using various
technological tools (e.g.,print technologies, imaging technologies, and video).
2.4.6 Use writeable CD-ROMs to record productions using various technological
tools.
2.4.7 Use appropriate software for capturing Web pages, audio wave files, and
video files for developing off-line presentations.
3.1 Media Utilization
3.1.1 Identify key factors in selecting and using technologies appropriate for
learning situations specified in the instructional design process.
3.1.2 Use ECIT resources in a variety of learning contexts.
3.2 Diffusion of Innovations
3.2.1 Identify strategies for the diffusion and adoption of innovations in learning
communities.
3.3 Implementation and Institutionalization
3.3.1 Use appropriate instructional materials and strategies in various learning
contexts.
3.3.2 Identify and apply techniques for integrating ECIT innovations in various
learning contexts.
3.3.3 – Identify strategies to maintain use after initial adoption.
3.4 Policies and Regulations
3.4.1 Identify standards for instructional technology utilization and application.
3.4.2 – Identify and apply policies which incorporate professional ethics within
practice.
3.4.3 Identify and apply copyright and fair use guidelines within practice.
3.4.4 – Identify and implement effective policies related to the utilization,
application, and integration of instructional technologies.
3.4.5 – Identify policies and regulations which apply to the utilization,
application, and integration of distance delivery technologies.
4.0.1 Demonstrate leadership attributes with individuals and groups (e.g.
interpersonal skills, group dynamics, team building).
4.1.1 Develop and apply project management techniques in various learning and
training contexts.
4.2.1 Develop and apply resource management techniques in various learning
and training contexts.
4.3.1 Develop and apply delivery system management techniques in various
learning and training contexts.
4.4.1 Develop and apply information management techniques in various learning
and training contexts.
5.1.1 - Identify and apply problem analysis skills in appropriate educational
communications and instructional technology contexts.
5.2.1 - Develop and apply criterion-referenced measures in a variety of ECIT
contexts.
5.3.1 - Develop and apply formative and summative evaluation strategies in a
variety of ECIT contexts.
5.4.1 - Develop a long-term strategic plan related to any of the domains or sub
domains.
Page 5 of 10
SOA - IT - IT Division
III.
Frequency of Assessments
Student assessment occurs continuously beginning with the students’ application for
admission to the program and ending up to two years beyond graduation. Planned
assessment activities are used in 1) pre-admission, 2) completion of each course, 3)
completion of the comprehensive exam/electronic portfolio, 4) completion of the research
requirement, and 5) completion of the post-graduation survey.
IV.
Methods of Assessment
Outcomes Assessment
The effectiveness of the program courses in addressing the ECIT standards and indicators
is evaluated using a representative project/assignment in each course. These
projects/assignments are selected as representative of a student’s success in mastering the
ECIT outcomes for the course. Rubrics are developed to reflect this alignment. These
rubrics are used to assess each project/assignment. The scored rubrics are reviewed and
the students’ level of attainment of each of the indicators is evaluated. These results are
combined into a program matrix that portrays the success of the program as a whole in
teaching the ECIT standards.
Authentic Experiences
Students are involved in authentic projects and practica as part of their studies, both as
specific courses and within courses. The specific courses dedicated to providing students
with authentic experiences outside of the typical classroom include 240:245 Applied
Instruction Design and 240:297 Practicum. Courses that include authentic experiences by
designing and developing products for actual clients include 240:139g Media Planning
and Production; 240:150g Instructional Television Production; 240:205 Instructional
Computing Design; 240:235 Managing Educational Technology Programs; and 240:237
Coordinating Technology in an Educational Setting.
Summary data from the students, from their practice/project coordinators, and from data
collected related to the results of the experience are all used to evaluate the success of the
learning experience. The nature of the practicum experience involves teaching or
designing instruction for UNI entities including the Center for Education Technology and
the Physical Plant Services. Until spring of 2005, practica were evaluated using the
Curriculum and Instruction Practicum Evaluation checklist. In spring 2005, the
Instructional Technology Practicum Rubric was implemented.
Page 6 of 10
SOA - IT - IT Division
Comprehensive Examinations
Until recently, every student was required to pass a comprehensive examination before
graduation. In the past five years, all but one student passed the written exams. On a
number of occasions, however, replies for one or more questions on the comprehensive
exam required the student to be retested. This retesting took the form of rewriting the test
question or having the student sit for an oral review of the question. The professor who
wrote the question was responsible for proctoring the written or oral exam.
After adopting the ECIT standards, the faculty determined the need to re-assess
comprehensive exam questions to ensure their alignment with the standards. A written
comprehensive examination policy was defined with the creation of the Written
Comprehensive Examination Reader Outline. This outline provided the two faculty
members who read the student responses with a guide to assessing the quality and
accuracy of the response. The guidelines were as follows:
First Reader: The first reader will read specific questions for content knowledge and
continuity. Areas to be addressed include:
 Fully responds to the intent of the question.
 Incorporates knowledgeable individuals within the response. (i.e.,
citations/references of authorities in the field).
 Includes theory and research to support the response.
 Demonstrates alignment with course goals.
 Uses correct spelling and grammar
Second Reader: The second reader is the individual's advisor. This reader will receive the
first readers’ comments and review the total exam for:
 Flow of response
 ECIT standards from the broad perspective -- looking for a balance in
understanding the five different areas.
Electronic Portfolio
As of fall 2005, the comprehensive exams have been replaced with the creation of an
electronic professional portfolio as the comprehensive exit requirement. Students
beginning in summer 2004 and on will be required to complete the portfolio. A rubric has
been developed to assess the quality of the portfolios and a student’s success in
addressing the five domains.
Students are asked to present their course products in a comprehensive portfolio. The
portfolio requires students to present their artifacts in a multimedia format and reflect on
how these artifacts are aligned with standards. The presentation format of the portfolio is
not defined. This allows students to use their creativity and technical skills to create a
personalized format. However, an infrastructure is identified that includes: a personal
philosophical statement, a matrix showing alignment of artifacts with standards, separate
pages for each of the artifacts created in a 240:xxx course, individual standards reflection
pages where the student reflects upon how the included artifacts address each standard,
and a comprehensive reflection referring to the entire masters experience.
Page 7 of 10
SOA - IT - IT Division
Masters Paper/Thesis
A final graduate requirement paper is written by each student to demonstrate the ability to
apply and synthesize knowledge and skills developed during the program of study. This
paper is designed to give the student the opportunity to support ideas with information
from current writing and research.
The student is allowed to select between writing a masters paper or a thesis. The
masters paper is a 2-credit assignment which involves the student writing under the
guidance of a single director and has a second reader review the final draft. The
thesis paper is a 6-credit venture, which requires the student to select a committee of
three faculty members. The student must submit and present a proposal to the
committee for approval. The student can begin the research process for writing the
paper when the proposal has passed the committee. Ultimately, the student must
submit and defend the final paper for the committee approval.
The departmental graduate research requirement for a non-thesis option may be met in
one of four paper formats: a literature review, original research, a graduate project, or a
journal article. Each of these options includes a literature review. The final determination
of the type of paper to be selected must be decided in consultation with the director(s) of
the study. This paper requires two faculty members to serve as first and second readers
for the paper. The first reader is a director for the project and works with the student in
topic selection, research, and writing the paper. When the student and faculty member
have refined the paper and are satisfied with its quality, the paper is submitted to the
second reader. The second reader provides a detached perspective to the paper. This
faculty member will make editing recommendations, or, if the paper still requires a great
deal of editing, the second reader can return the paper to the student and first reader for
further work.
Post-Graduation Assessment
The formal assessment of student performance in the program also involves contacting
students after graduation to determine if the program of study met their needs. This
follow-up information is primarily pursued through an on-line survey. In some situations,
there have been phone follow-ups where the surveyor actually asked each alumnus to
orally answer the questions from the on-line survey. The on-line survey can yield data for
developing insight into ways the program fostered the graduates’ knowledge and skills
within their present employment placement.
V. Analysis – Interpretation and Reporting Results
1. Outcomes Assessment
All five required courses in the Instructional Technology program and four elective
courses are evaluated for alignment to standards. The instructor for each course collates
Page 8 of 10
SOA - IT - IT Division
the student outcomes from the assessment rubric for that course’s representative
assignment. The alignment of the elements on each rubric with ECIT standards enables
the instructor to quantify the students’ success at exhibiting mastery of individual
indicators. These results are then compiled in a program matrix to demonstrate student
outcome success program wide.
Authentic Experiences
Application is an important part of any program. It supplies the proving ground for the
theoretical instruction found in coursework. As noted earlier, the practical application of
skills and knowledge is experienced as a practicum or in practicum-like courses. Some of
our students with graduate assistantships have practicum-like experiences where they
serve as laboratory instructors in large courses. The undergraduate course, 240:031,
Educational Media, is such an example where graduate students are taught how to teach
and then closely monitored and supported as they develop their skills.
Electronic Portfolios
Electronic portfolios were initiated in 2005 and are evaluated based on the division
developed rubric.
Post-Graduation Assessment
Exit surveys have been available since 1999 for our graduates. The surveys are
comprised of 14 statements, which indicate the level of preparation for their current
positions. The survey employs a Likert scale of 1 through 5 (not at all, somewhat,
adequate, good, and excellent).
Overall, the better than average ratings on the assessments traced to date indicate that
students feel they were well prepared through the Instructional Technology curriculum
for their professional responsibilities. Even though shifts in perspectives were noted in
the follow-up surveys of alumni, they still indicated a positive attitude towards their
professional preparation.
2. Methods for Sharing Findings
Findings derived from the data collection process are shared among program faculty
through various methods. The Instructional Technology Division faculty meets as a
division 4 – 5 times per month. During these meetings, faculty members share program
assessment data, student achievement reports, and curricular planning proposals. Issues
from the meetings are discussed through email. Pertinent materials and documents are
shared via the web, email and a common data server. Products of this analysis that are
relevant to students are shared through the Instructional Technology website and email.
Other interested parties can also access information from the program website. The
results of our program alignment process and student outcomes have been shared at an
international and state conference for the last 3 years with other practitioners in the
Page 9 of 10
SOA - IT - IT Division
discipline.
Page 10 of 10