NOCTI Overview
Amie Birdsall and Patricia Kelley
February 23, 2012
Discussion Topics
 NOCTI Overview
 Products and Services
 Selecting an Assessment
 CTEDDI
 Q&A
NOCTI Overview
 Over 40 years serving the CTE community

Non-profit organization

Owned and operated by the 50 states

Each state is a member of NOCTI’s Consortium of
States

Works with both secondary and post-secondary
populations

Nocti Business Solutions (formerly The Whitener
Group) provides services to industry
NOCTI Overview - Educational Partners
NOCTI Products & Services
Assessments

Over 80 Job Ready tests
offered

30 Pathway tests in 10
clusters available

Over 90 private industry
employment and/or
advancement tests
delivered

20 teacher-level
assessments
Supplemental Products
 Study guides for all Job Ready
and Pathway assessments
 Countdown Kits available for
all Job Ready assessments
 Successful Schools-Successful
Programs Guide
Professional Learning
Opportunities Also
Offered!
Selecting an Assessment
 Determine purpose of assessment
 Determine audience
 Determine results desired
 Research available options
Selecting an Assessment
Selecting an Assessment
Assessment
blueprints are
available on
NOCTI’s website
at no cost!
www.nocti.org
Assessment Blueprints
• Blueprints include:
• Competency
List
• Content
Distribution
• Sample
items/jobs
Assessment Blueprints
 Content distribution
for written and
performance
 Sample multiplechoice questions
 Sample performance
job
Job Ready Assessments

Assess technical skills at the occupation level
(e.g., Accounting, Carpentry, Pre-Engineering)

Measure aspects of occupational competence
such as factual and theoretical knowledge

Target secondary and post-secondary
program completers
NOCTI Job Ready Assessment Blueprint
Plumbing
8%
Safety
10%
Plumbing Math
24%
General Plumbing Practices
9%
Joining Pipes and Fittings
8%
Drainage
Vent Installation
Water Supply and Distribution
Hot Water Systems
Traps
Installing Fixtures
Plumbing Service and Repair
4%
7%
6%
8%
7%
9%
NOCTI Job Ready Assessment Blueprint
Number of Questions: 222
Areas Covered:
Accounting
NOCTI Pathway Assessment Blueprint
Construction Pathway
General Architecture & Construction Technical Skills
15%
Construction
15%
Academic Foundations
15%
Systems
6%
Ethics and Legal Responsibilities
6%
8%
Communication
Information Technology Applications
Problem Solving, Critical Thinking, and Decision Making
Leadership and Teamwork
Safety, Health, and Environment
Employability and Career Development
5%
7%
6%
8%
9%
Assessment Results
 NOCTI’s Standard Score Reporting Package:


Individual Score Reports

Group Score Reports

Comparative Data
Other Reports Available:

Task-Link Reports

Integrated Academic Reports
What data do I receive from NOCTI?
Score Reports
Online Test Administration
 Immediate Score
Reports
Paper/Pencil Administration
 Aggregate Reports:
10 business days
after receipt of
 Aggregate Reports:
correctly
2 business days
completed answer
after user codes are
sheets
released from
Client Services
Center
What data do I receive from NOCTI?
Immediate Score Reports

Immediate Student
Scores

Displayed to student
if option is
designated

Emailed to Site
Coordinator upon
completion of testing
(can be forwarded to
instructor)
NOCTI Standard Score Reports
Individual Score
Reports
• Provided for each
test taker
• Scores by duty and
total
• Pre-/post-test
comparisons
• Comparative data
NOCTI Standard Score Reports
Group Score Report
• Provided for the
testing group
• Duty and total
scores for each
participant
NOCTI Standard Score Reports
Analysis of Scores
Report
• Provided for the
testing group
• Comparative data
• Criterionreferenced cut
score
• Other statistical
information
NOCTI Additional Reports:
Task-Link Report
Task-Link Report
• Includes:
• Number of items
presented for each
task
• Average correct by
group
• Number correct
for each test taker
Project Goals
•
•
•
•
Understand data and assessments
Use data effectively
Improve learning and skill acquisition for students
Plan for continuous improvement of learning and
instruction
• Share ideas and collaborate with other educators
Instructional Improvement Cycle
Classroom Example
A Nursing Assisting
Class
Areas on a Nursing Assisting Test
Basic Nursing Assisting
8%
Infection Control
8%
28%
Communication Skills
Personal Care Skills
13%
10%
10%
Mental Health/Social
Needs
Client’s Rights
Basic Restorative Devices
13%
10%
Employability Skills
Individual Nursing Pretest
Pretest % Correct
Student 1
Posttest % Correct
Student 1
Basic Nursing Assisting
54.6
--
Infection Control
30.0
--
Communication Skills
41.1
--
Personal Care Skills
36.4
--
Mental Health/Social Needs
53.8
--
Client’s Rights
36.4
--
Basic Restorative Devices
50.0
--
Employability Skills
27.3
--
Nursing Assistant Pretest Areas
Total
43.4
Nursing Assisting Pretest
Nursing Assistant
Pretest Areas
Student 1
% Correct
Student 2 Student 3 Student 4
% Correct % Correct % Correct
Student 5 Student 6
% Correct % Correct
Class
Average
% Correct
Basic Nursing
Assisting
54.6
45.5
42.7
27.3
27.3
54.5
41.9
Infection Control
30.0
20.0
10.0
20.0
10.0
30.0
20.0
Communication
Skills
41.1
17.6
23.5
23.5
13.2
35.3
25.7
Personal Care Skills
36.4
54.5
45.5
45.5
36.4
54.5
45.5
Mental Health/Social
Needs
53.8
38.5
42.3
38.5
20.8
53.8
41.3
Client’s Rights
36.4
36.4
38.3
36.4
22.5
45.5
35.9
Basic Restorative
Devices
50.0
60.0
50.0
60.0
30.0
80.0
55.0
Employability Skills
27.3
18.2
18.2
27.3
9.1
27.3
21.2
43.4
38.4
35.9
38.5
22.8
48.8
38.0
Total
Nursing Assisting Posttest
Nursing Assistant
Pretest Areas
Student 1
% Correct
Student 3 Student 5
% Correct % Correct
Group
Site
State
Nation
Basic Nursing
Assisting
100.0
92.4
76.2
86.4
79.5
72.5
75.5
Infection Control
91.7
94.5
72.4
79.2
78.2
70.7
70.6
Communication
Skills
100.0
100.0
55.3
95.0
83.8
64.3
66.4
Personal Care Skills
92.3
54.5
52.5
69.2
65.8
51.6
52.9
Mental Health/Social
Needs
97.4
72.5
60.3
71.8
69.8
63.1
64.1
Client’s Rights
84.8
66.4
44.7
64.6
67.6
58.1
57.8
Basic Restorative
Devices
87.5
60.2
45.2
53.1
61.9
55.6
56.5
Employability Skills
87.5
78.5
37.4
79.7
75.6
66.6
67.0
93.9
79.4
58.4
76.8
73.7
64.1
65.2
Total
What the Data Reveal
• Achievement gaps
• Differences between subgroups
• Trends
• Progress
Collecting Data – More Than
NOCTI
• Includes process of collecting, analyzing, and
interpreting information
• Determines the degree to which students are
becoming competent and meeting standards
• Occurs at many levels (e.g., classroom and
program assessments to state and national
assessments)
What Data Exist?
• National
• State
• District/System
• School
• Program
• Teacher
• Class
• Student
• Standards
Icebergs
What lies beneath
the data scores
that influence
those scores?
What Questions Can the Data
Answer?
• What is the data designed to
answer?
• What other data may be
needed?
• What roadblocks are there to
interpreting the data?
Plan for Learning Improvement
• Start with results: Analyze the previous year’s
goals, comparing against local unit tests and
end-of-course tests, at the student level and
classroom level.
• Triangulate: Find relationships
between test data and
attendance and discipline
referrals.
Plan for Learning Improvement
Continued:
• Disaggregate data by subgroups (e.g.,
socioeconomic status or those receiving
free/reduced lunch, or Individualized Education
Program/IEP).
• Use information from advisory groups or parents.
• Set goals for the year as well as long-term goals.
Plan for Learning Improvement
Continued:
• A common denominator is to improve learning
through teamwork.
• Build school department and state data teams.
• Review annual assessment data with a faculty
team to isolate achievement gaps or groups of
students with high and low performance.
Plan for Learning Improvement
Continued:
• Schedule school and department data team
meetings to discuss data and review progress.
• Discuss and develop strategies, such as curricular
alignment with academic and industry standards,
curricular sequence, timing, and depth of
instruction.
Monitor Student Progress
• Use a progress monitoring system for all
students.
• Graph students’ curricula-based scores.
• Identify effective instructional components.
Thank You
Amie Birdsall
Amie.Birdsall@nocti.org
Pat Kelley
Patricia.Kelley@nocti.org