Sampling for the Follow-Up Surveys
Beginning in Program Year 2012, students are
identified for NRS post-program outcomes
according to their cohort membership. Cohorts are
determined based on pre-defined characteristics
rather than by goal setting. As a result, the number
of learners to follow for each outcome is likely to
increase in some states and programs.
precision.) To help states make these estimates, a
total survey sample size (ns) for each outcome has
been set for states. The size is dependent on the
number of students needed to produce accurate
estimates about the population. A state sample of
400 to 1,100 students will be required, depending
on the number of students in the cohort.
Although some states may prefer to do data
matching, this approach may not be possible for
some states because of issues such as high
proportions of students with missing social security
numbers. Instead, states may need to conduct
surveys to measure post-program outcomes. States
using the survey method of follow-up data
collection, especially those with large cohort sizes,
may want to consider using sampling rather than
collecting data from all students. This NRSTips
describes the benefits of sampling, the procedures
to follow when sampling, using sampling for state
and national estimates, and how to determine cohort
and sample sizes.
Although not required, a state may also choose to
use sampling to allow estimates of measure for
local programs. When doing sampling at the local
level, a substantially larger sample size maybe
required than the sample size needed for a state or
national estimate. The procedures to be followed to
create the estimate, however, are the same as those
listed above.
Why Sample?
The number of learners in each cohort may be much
larger than the groups states previously had to
follow for each post-program outcome. Sampling
can be used to reduce the burden of this increased
population size of cohorts because this approach
allows you to collect data from a smaller, random
sample that represents the total population. Based
on data obtained from a sample, an estimate of the
true number can be calculated, within a range of
error. Sampling allows you to focus on quality of
data collected rather than quantity. However, note
that the response rate requirement for sampling is
70% rather than the data match requirement of
50%; this is meant to ensure that the sample is
representative of the larger population.
How are the Cohort and Sample Sizes
Determined?
Although there may be changes in population size
from year to year, to estimate the size of your
state’s cohort for this year, look at last year’s data.
If you have the data available, you will be able to
use prior years’ reported cohort sizes to make the
estimates.
The estimates for the cohorts should be calculated
separately for each cohort:

Entered employment: The number of
unemployed in the labor force at entry who
exited the program

Retained employment: The number of
employed students at entry and number of
students in the entered employment cohort who
obtain employment one quarter after exit

Secondary credential attainment: The number
of students taking GED tests, adult high school
students in high ASE, and EDP assessment
phase students who exited the program

Entry into postsecondary education or training:
The number of students with a secondary
Using Sampling for Estimates
Sampling allows for state and national estimates to
be computed accurate enough to detect a difference
of 5%. (States can sample more if they need greater
credential and/or in transition classes who
exited the program
The sample size you need is based on the eligible
number of students in the cohort. (See table below.)
The sample may be greater, equal or nearly equal to
the entire cohort. Keep in mind:

When you have a very small cohort, rather than
sampling, you will need to survey everyone in
cohort.

When you have a very large cohort, you can
obtain a representative sample with a fairly
small sample relative to the population size.

A good response rate is more important than a
very large sample.
Number of Eligible Learners in Cohort
Number in Sample
1 to 200
All
201 to 400
All
401 to 600
400
601 to 1,000
500
1,001 to 1,200
550
1,201 to 1,400
630
1,401 to 1,600
660
1,601 to 1,800
715
15,001 to 16,000
1,030
16,001 to 25,000
1,035
25,001 to 35,000
1,065
35,001 to 40,000
1,070
40,001 to 100,000
1,085
100,001 and up
1,100
How Do I Sample?
To do the sampling, states should perform the
following steps:
1. Determine the total number of students in the
cohort for each outcome measure for which you
will be sampling. Remember that to be a
member of a cohort, a student must have exited
the program. For example, the entered
employment cohort includes the unemployed in
the labor force who exit, not simply the
unemployed.
2. Determine the overall sample size required for
each state cohort, based on the overall cohort
size, using the table.
3. Calculate the proportion of students in each
state cohort that come from each local program.
For example, if there were 1000 students total
in the state’s entered employment cohort, and
Program A had 100 students in the cohort, then
Program A represents 10% of the total cohort.
4. Calculate each program’s sample size, based on
steps 2 and 3 above. For example, if there were
1000 total in the state’s entered employment
cohort, the state sample size for that cohort
should be 500. If Program A represents 10% of
the overall state cohort, they should represent
10% of the sample as well. Therefore, the
sample size for Program A should be 10% of
500 (500 x .10), or 50.
5. Randomly sample the required number of
eligible students from each program. In
conducting the survey, states can distribute the
student sample list to each program and have
local programs conduct the survey OR the state
can both sample and conduct the survey.
Additional Resources
For more information on the new employment
reporting or for answers to questions, visit
http://www.nrsweb.org or e-mail: NRS@air.org.
Local program staff should first consult their state
office for their state policy.
NRS Tips is a quick reference tool for state staff,
program directors, and adult education teachers.
NRS Tips are written and produced by the staff at
the American Institutes for Research (AIR), a
nonprofit research and policy organization (see
http://www.air.org), under contract with the Division
of Adult Education and Literacy (DAEL) of the
U.S. Department of Education.