Respondent Generated Intervals (RGI)
S. James Press
Department of Statistics
University of California at Riverside
Riverside, CA 92521-0138
jpress@ucr.edu
Judith M. Tanur
Department of Sociology
State University of New York at Stony Brook
Stony Brook, NY 11794-4356
jtanur@notes.cc.sunysb.edu
1
Respondent-Generated Intervals (RGI)
Definition
The Respondent-Generated Intervals, or RGI, protocol for asking questions in
sample surveys involves asking respondents not only for a basic answer to a
recall-type question, but also, the respondent is asked for a smallest value his/her
true answer could be, and a largest value his/her true answer could be. We’ll refer
to these values as the lower and upper bounds provided (sometimes called
brackets). The result of the RGI protocol is that the respondents themselves
generate the intervals in which their true beliefs lie, instead of having their
quantitative beliefs forced into intervals pre-assigned by the survey designer.
Other survey protocols sometimes provide pre-assigned intervals for the response,
and the respondent must choose one of the intervals.
Interval-Response Surveys
Survey protocols that permit the respondent to give answers in intervals, selfdetermined, or pre-assigned by the survey designer, are often preferred by
respondents for sensitive questions because the respondent need not be specific
about the exact value being requested. Interval response protocols are also often
preferred by respondents for questions for which the answers are not very well
known. By responding in intervals for such questions, respondents need not be
precise about the exact answer. Respondents prefer the RGI technique because it
2
allows them to have control over their disclosures, and RGI allows respondents to
feel confident about the accuracy of the information they provide. The intervals
RGI respondents provide tend to be narrower than pre-defined intervals.
Genesis of RGI
The RGI protocol for questionnaire design has its origins in Bayesian assessment
procedures. In that context, for a specific individual, we might assess an entire
prior distribution about an unknown parameter. That prior distribution represents
the individual’s degrees of uncertainty about that unknown parameter. In certain
contexts, we might assess many points on the individual’s subjective probability
distribution for that parameter by means of a sequence of elicitation questions,
and then connect those points by a smooth curve that represents the underlying
distribution.
For example, using some purely hypothetical numbers, suppose an individual has
a normal subjective probability distribution representing “  0 ”, the true (but
unknown) change in the number of doctor visits he/she believes he/she made last
year, compared with the previous year, so that  0 ~ N (4,1). In such a case, the
individual believes that it is most likely that he/she visited a doctor 4 more times
last year than the previous year, with a standard deviation of 1. So this individual
equivalently believes that there is a 99.7% chance that he/she visited a doctor
between 1 and 7 more times last year, or that there is really almost no chance that
the true number of excess times was less than 1 or greater than 7. This
probability distribution is subjective, in that it represents a specific individual’s
degrees of belief about his/her uncertainty about the underlying quantity, in the
case of this example, the individual’s uncertainty about how many more visits
3
he/she believes he/she truly made to the doctor last year compared with the
previous year.
We postulate that: in a factual survey each respondent has a distinctive recall
distribution, and in an attitude or opinion survey he/she has an underlying
probability distribution for his/her opinion or attitude about some issue. In the
case of a recall-type question, we assume that the respondent knew the true value
at some point (or had enough information to construct it) but because of imperfect
recall, he/she is not certain of the true value. He/she may feel confident that
he/she knows the true value (but may be wrong in spite of high confidence), or
he/she may be quite uncertain of the true value (and conceivably could be correct
about the true value, but not realize it). We furthermore assume that the
respondent is not purposely trying to deceive. In the case of opinion or attitude
questions, the respondent may have a very fuzzy idea of his/her attitude about an
issue, or he/she may feel quite strongly and specifically about it.
Benefits of RGI to the Survey Administrator
The benefits of the RGI survey protocol to the respondents were discussed above.
Now we discuss its benefits for the survey administrator. There are issues of (1)
response rate, (2) measurement of range-of-belief, and (3) accuracy of estimating
the population mean (in the case of factual questions). We treat these issues in
turn.
(1)
Response Rate-- Interval protocol methods tend to reduce item nonresponse in that such techniques present a viable method for obtaining
4
some information from respondents who might otherwise provide none.
(See Item Non-Response, Item Non-Response Remedies.)
(2)
Measurement of Range-of-Belief—Suppose ai , bi denote the lower and
upper bounds, respectively, for respondent i, i = 1, …, n, where n
denotes the number of respondents in the sample. Then, if
a0  min  ai  , and b0  max  bi  , a measure of range of belief for the
i
i
sampled respondents is given by: (b0  a0 ).
(3) Accuracy—Data collected from a factual RGI survey can be used to
generate an estimate of the population mean that has smaller non-sampling
error (bias) than that associated with estimators obtained from more
conventional surveys. The proposed estimator is model-based and is
Bayesian. To obtain an estimate of the mean in a population we typically
just form the simple average of the opinions or attitudes or recalled values
from individuals who may have very different abilities to carry out the
task required for a response. But such a simple average may not
necessarily account well for typical unevenness in assessment of opinion
or attitude, or in recall ability. The RGI protocol can be used to improve
upon more traditional population estimates by its ability to learn more
about the different abilities respondents have to quantify their fuzziness
about their beliefs, and then take that fuzziness into account in the
estimation process. Ideally, we would ask the respondents many
additional questions about their beliefs. That would permit us to assess
many fractile points on each of their underlying opinion or attitude or
recall distributions. Owing to the respondent burden of a long
questionnaire, the sometimes heavy cost limitations of adding questions to
5
a survey, the cost of added interviewer time, etc., there may sometimes be
a heavy penalty imposed for each additional question posed in the survey
questionnaire. (See Respondent Burden.) In such cases, RGI might be
used for just a few questions. The RGI protocol obtains three points on
each respondent’s underlying distribution. A hierarchical Bayesian
procedure is suggested below for using the collection of triples of points
from the respondents for estimating the population mean. The resulting
RGI estimator can often improve upon the accuracy of the usual sample
mean.
Let the triple: ( yi , ai , bi ) denote the basic response, and, as above, the
lower bound response, and the upper bound response, respectively, for
respondent i, i = 1,…, n. Suppose that the yi ’s are all normally distributed
N ( i ,  i2 ) , that the  i ' s are exchangeable, and  i ~ N ( 0 ,  2 ) .  i
denotes the true (factual, or opinion, or attitude) value for respondent i. It
has been shown (see Press, 2003, Appendix) using a hierarchical Bayesian
model, that in such a situation, the conditional posterior distribution of the
population mean,  0 , is normal, and is given by:
(  0 data,  i2 , 2 ) ~ N ( ,  2 ),
(1)
where the posterior mean,  , conditional on the data and ( i2 ,  2 ) , is
expressible as a weighted average of the usage quantities, the yi ’s, and the
weights are expressible approximately as simple algebraic functions of the
6
bounds. The conditional posterior variance,  2 , drives the credibility
(Bayesian confidence) interval, and is discussed below.
We assess values for the  i2 parameters from: k1i  bi  ai  ri , the
respondent interval lengths. Analogously, we assess a value for  2 from:
k2  b  a , the average respondent interval length. For normally
distributed data it is commonly assumed that lower and upper bounds that
represent extreme possible values for the respondents can be associated
with 2 standard deviations below, and above, the mean, respectively.
We’ll generally assume therefore that k1  k2  k  4 (corresponding to
standard deviations above and below the mean).
The conditional posterior mean was shown (Press, 2004a) to be given by:
n
   i yi ,
(2)
1
where the i ’s are weights depending upon the interval lengths that are
given approximately by:


1
 2
2 
r  (b  a ) 
i  n i
.


1
1  r 2  (b  a )2 
 i

(3)
7
We note the following characteristics of this RGI (weighted average)
estimator:
1)
The weighted average is simple and quick to calculate, without
requiring any computer-intensive sampling techniques.
2)
It will be seen in the examples below that if the respondents who
give short intervals are also the more accurate ones, the RGI estimator will
tend to give an estimate of the population mean that has smaller bias than
does the sample mean. In the special case in which the interval lengths are
all the same, the weighted average reduces to the sample mean, y , where
the weights all equal (1/n). In any case, the lambda weights are all
positive (although some may be extremely small), and must sum to one.
3)
The longer the interval a respondent gives, the less weight is applied
to that respondent’s usage quantity in the weighted average. The length of
respondent i’s interval is a measure of his/her degree of confidence in the
basic response he/she gives, so that the shorter the interval, the greater
degree of confidence that respondent has in the basic response he/she
reports. Of course a high degree of confidence does not necessarily imply
an answer close to the true value.
4)
If we define the precision of a distribution as its reciprocal variance,
the quantity { [ri 2  (b  a ) 2 ] /(4) 2 } may be seen to be the conditional
variance in the posterior distribution corresponding to respondent i, and
8
therefore, its reciprocal represents the conditional precision corresponding
to respondent i. Summing over all respondents’ precisions gives:
total conditional posterior variance =  2 
1
.
n


16
1  r 2  (b  a )2 
 i

(4)
The number “16” is the square of the 4 that arises from 4 i  bi  ai  ri ,
and 4  b  a .
So another interpretation of i in equations (2) and (3) is that it is the
proportion of the total conditional posterior precision in the data
attributable to respondent i.
The variance of the conditional posterior distribution is given in eqn. (4).
The posterior variance is the reciprocal of the posterior total precision.
Because the posterior distribution of the population mean,  0 , is normal, it
is straightforward to find credibility intervals for  0 . For example, a 95%
credibility interval for  0 is given by:
(  1.96,   1.96 ) .
(5)
That is,
9
P{(  1.96   0    1.96 ) data}  95%.
(6)
An alternative interval estimate of the population mean that is more likely
than the credibility interval to cover the true population value is the
Average Respondent-Generated Interval, ARGI  b  a . The range-ofbelief for the sample of respondents is b0  a0 .
Examples
We next present some numerical “theoretical examples” of the use of RGI
estimates.
1) Example For Recall Questions
We examine an artificial extreme case. Suppose we have a sample survey of size
n = 100 in which the RGI protocol has been used. Suppose also that the true
population mean of interest that we are trying to estimate is given by  0  1000.
In this example we fix both the basic response quantities and the respondents’
bounds, (ai , bi ) , ri  bi  ai , i  1,..., n, artificially.
Assume the respondents have been ordered so that the first 30 respondents all
have excellent memories, are quite accurate, and have individual truths of 900..
Suppose the intervals these accurate respondents give are:
(a1, b1 ),...,(a30 , b30 )  (975,975),...,(975,975) .
10
That is, they are all not only pretty accurate, but they all believe that they are
accurate, so they respond to the bounds questions with degenerate intervals whose
lower and upper bounds are the same. Accordingly, these accurate respondents
all report intervals of length ri  0 , and usages of equal amounts, yi  975
(compared with the true population value of 1000.,
In addition, suppose the next 20 respondents also are quite accurate, but their
individual truths are above 1000, say 1100. Suppose the intervals these
respondents give are:
(a31, b31 ),...,(a50 , b50 )  (1125,1125),...,(1125,1125) .
Next suppose that the last 50 respondents all have poor memories and are
inaccurate. They report the intervals:
(a51 , b51 ),...,(a100 , b100 )  (500,1500),...,(500,1500) ,
that have lengths of ri  1000 , and they report unequal basic response quantities
of 550, for the first 30, and 1450 for the last 20. We now find: a  767.5, and
b  1267.5, so b  a  500. We may now calculate that the weights are given by:
.0167, i  1,...50
.0033, i  51,...,100
i  
11
100
It is easy to check that:

i
 1. We may now readily find that the conditional
1
posterior mean RGI estimator of the population mean,  0 , is given by:
100
   i yi  1014.17.
i 1
The corresponding sample mean is given by:
y  972.5.
The numerical error (bias) of the posterior mean is given by:
  1000  1014.17  1000 14.17.
The numerical error (bias) of the sample mean is given by
y  1000  972.5 1000  27.5.
The RGI estimator has reduced the bias error by
27.5 - 14.17 = 13.33, or about 48%,
compared with the error of the sample mean.
It is also interesting to compare interval estimates of the population mean by
comparing the standard error of y , with  , the standard deviation of the
12
conditional posterior distribution of  . These estimates give rise to the
corresponding confidence and credibility intervals for  0 , respectively.
We may readily find that for the data in our theoretical example,   16.14. It is
also easy to check that for our data, the standard deviation of the data is 323.80.
So the standard error for a sample of size 100 is 323.80/10 = 32.38. Thus, the
RGI estimate of standard deviation is about half the standard error of the sample
mean. Correspondingly, the length of the 95% credibility interval 2(1.96) ( ) =
63.27, while the length of the 95% confidence interval is 2(1.96)(32.38) = 126.93.
The 95% confidence interval is about twice as long as the 95% credibility
interval. The 95% credibility interval is given by: (982.53, 1045.80). The 95%
confidence interval is given by:
(909.04, 1035.96).
We note in this theoretical example that:
1) Both the RGI credibility interval and the confidence interval cover the true
value of 1000, but the credibility interval is smaller.
2) we expect to find many situations for which the bias error of the RGI
estimator is smaller than that of the sample mean; however, the
differences may be more, or less, dramatic compared with their values in
this example;
3) in large samples, we expect that both the sample mean and the RGI
estimator will converge to one another, and to the true population mean.
4) We also note that the success of the theoretical example depends on the
fact that the accurate respondents give short intervals and so have heavier
13
weights than the inaccurate respondents who give longer intervals. In
order to help assure that such conditions hold in operational studies, we
can ask respondents a preliminary question concerning their confidence in
their accuracy and use their confidence weightings to moderate their
responses.
2) Example for Opinions and Attitudes
Suppose there is a population of opinions about some issue, say, “Issue A”.
Perhaps the analyst would like to establish the mean of the opinions of all people
living in the City of New York about Issue A. This is “truth”. But it isn't an
absolute truth. It's a relative truth, a truth at this time, and under current
circumstances. It's not an absolute truth that will have the same value next month,
or next year, as would be the case for a factual or recall-type of question. The
analyst would like to estimate that relative truth. Relative true values are
generally different for each person in a sample, as is generally the case in a recall
question. But accuracy in an opinion or attitude context has no meaning, other
then with respect to sampling error, although respondents with fuzzy attitudes or
opinions who give intervals as well as point values provide additional information
with which their views may be sharpened. There is no “correct” answer for an
opinion or for an attitude for a given respondent, as there would be for a person
answering a recall-type of question. Similarly, response bias does not have the
same meaning as in recall. (With a recall-type of question, one of the reasons for
response bias arises out of faulty memory.)
When using RGI for attitudes or opinions we can find both point and interval
estimators, although we can't really say much about accuracy, per se. But we can
14
speak about the RGI point estimator coming closer to second-guessing the
opinions of New Yorkers in the above example than a mere sample mean that
includes some people with very fuzzy opinions, and some people who have very
firm opinions. RGI can provide a measure of average strength-of-opinion, as
measured by the average range supplied by all respondents , (b  a ) . It can also
supply a credibility interval measure of belief; a confidence interval can also
supply an interval measure of belief. The confidence interval, however, only
reflects sampling uncertainty, whereas the RGI credibility interval also reflects
fuzziness of opinion. The range-of-belief also available with RGI, (b0  a0 ) , is
somewhat different.
For further discussion of the use of RGI surveys with opinion questions, see Press
and Tanur, 2004a.
Future Research
Optimally efficient RGI used with factual questions requires that respondents who
give narrow intervals also be the more accurate ones, and respondents who give
wide intervals also be the more inaccurate ones. When RGI is used with opinion
or attitude questions, firm and fuzzy opinions or attitudes should be analogously
categorized. Future research with RGI should involve developing meta-cognitive
procedures that will achieve these objectives.
References
Press, S. James (2004). “Respondent-Generated Intervals (RGI) for
Recall in Sample Surveys”, Journal of Modern Applied Statistical
Methods, May, 2004, Vol. 3, No. 1, pp.104-116.
15

Press, S. James and Tanur, Judith M. (2004a). “An Overview of
the Respondent-Generated Intervals (RGI) Approach to Sample
Surveys”, Journal of Modern Applied Statistical Methods, Nov.
2004, Vol. 3, No. 2, pp. 288-304.
Additional Reading

Chu, LiPing, Press, S. James, and Tanur, Judith M. (2004).
“Confidence Eliciation and Anchoring in the Respondent-Generated
Intervals Protocol”, Journal of Modern Applied Statistical Methods,
Nov. 2004, Vol. 3, No. 2, pp. 417-431.

Press, S. James and Judith M. Tanur (2000). "Experimenting with
Respondent-generated Intervals in Sample Surveys" , with
discussions, in Survey Research at the Intersection of Statistics
and Cognitive Psychology, Working Paper Series #28, Monroe G.
Sirken, Editor, National Center for Health Statistics, Center for
Disease Control and Prevention, pp. 1-18. Available for
downloading at: http://www.statistics.ucr.edu/press.htm

Press, S. James and Kent H. Marquis(2002). "Bayesian Estimation
in a U.S. Government Survey of Income Recall using RespondentGenerated Intervals", Proceedings of the Conference of the
International Society of Bayesian Analysis, Crete, Eurostat.
Available for downloading at:http://www.statistics.ucr.edu/press.htm
16

Press, S. James, and Tanur, Judith M. (2003). “The Respondent-
Generated Intervals Approach to Sample Surveys: From Theory to
Experiment”, in New Developments on Psychometrics, by Yanai,
Haruo; Okada, Akinori; Shigemasu, Kazuo; Kano, Yutaka;
Meulman, Jacqueline J. (Eds.), Tokyo: Springer-Verlag.

Press, S. James and Tanur, Judith M. (2004b). “Relating
Respondent-Generated Intervals Questionnaire Design to Survey
Accuracy and Response Rate”, Journal of Official Statistics,
Statistics Sweden, June, 2004, Vol. 20, No. 2, pp. 265-287.
Special Issue on Questionnaire Development, Evaluation, and
Testing Methods.
S. James Press
Judith M. Tanur
Dr. S. James Press is Distinguished Professor in the Department of Statistics,
University of California at Riverside, Riverside, California.
Dr. Judith M. Tanur is Distinguished Teaching Professor in the Department
of Sociology, State University of New York at Stony Brook, Stony Brook,
New York.
17