Jonathon R.B. Halbesleben
University of Alabama
March 6, 2013


Erin Holve, Ph.D., M.P.H.,
M.P.P.
– Senior Director of
Research & Education,
AcademyHealth

AcademyHealth is a leading national organization serving the fields of health services and policy research and the professionals who produce and use this important work.
Together with our members , we offer programs and services that support the development and use of rigorous, relevant and timely evidence to:
1.
Increase the quality, accessibility and value of health care,
2.
Reduce disparities, and
3.
Improve health.
A trusted broker of information, AcademyHealth brings stakeholders together to address the current and future needs of an evolving health system, inform health policy, and translate evidence into action.

The audio and slide presentation will be delivered directly to your computer

Speakers or headphones are required to hear the audio portion of the webinar.


If you do not hear any audio now, check your computer’s speaker settings and volume.
If you need an alternate method of accessing audio, please submit a question through the Q&A pod.


Live technical assistance:
– Call Adobe Connect at (800) 422-3623

Refer to the ‘Technical Assistance’ box in the bottom left corner for tips to resolve common technical difficulties.

To submit a question:
1.
2.
Click in the Q&A box on the left side of your screen
Type your question into the dialog box and click the Send button





Receive an overview of appropriate strategies for assessing nonresponse bias
Learn how to identify the primary contributors to nonresponse bias
Understand six methods for assessing nonresponse bias, including the circumstances under which each is applicable to their data
Describe the advantages and disadvantages of six methods for assessing nonresponse bias.

+
Evaluating Survey Quality in
Health Services Research: A
Decision Framework for Assessing
Nonresponse Bias
Jonathon Halbesleben
University of Alabama

+
The story behind the paper

+
Ways of Thinking About
Nonresponse Bias

Response Rate

Nonresponse bias = P
NR
(Mean
Res
- Mean
Pop
)
OR

Nonresponse bias = P
NR
(Mean
Res
- Mean
NonRes
)

Ideally, one would be thinking about this BEFORE conducting the survey…but that’s not always possible

Related to, but not quite the same as, subject attrition

+
A Decision Framework

+
Do you have access to the population?

Follow-up analysis

Contact subset of non-respondents with survey to see if differences emerge

Likely to have high nonresponse on the follow-up

+
Are There Questions in the
Survey That Might Address
Reasons for Nonresponse?

Leverage-Salience theory

Interest-level analysis

Interest associated with responding

Questions as proxies for interest –look for variation, associations with substantive variables, control for interest

Passive nonresponse analysis

People who would have done it, but didn’t for some reason

Questions like workload can tap into this

Simply resending survey might address it

+
Were the Data Collected in
Multiple Waves or Reminders?

Continuum of resistance

Wave analysis

Respondents who complete survey only at Time
1 are compared to those who completed it all times

Respondents who responded to reminder are compared to those who responded earlier

+
Are Population Data Available?

Sample-Population comparison

Usually focused on demographics, which may not be helpful

Bebee et al. (2011)

Had access to health records of population of interest

Could compare health of respondents vs. nonrespondents

+
Have Others Published Similar
Data?

Benchmarking

Sometimes the whole point is to look at a different group or one we think might differ on substantive variables

+
What if my answer was “no” to everything?

Replication

File-Drawer analysis

+
Recommendations

Thinking ahead – how might one avoid nonresponse?

Reporting multiple assessments of nonresponse

+
Other considerations

“
Layers” of nonresponse

Variation on level of impact of nonresponse

Study of patient bill responsiveness

Dealing with potential nonresponse bias

Weighting

To submit a question:
1.
2.
Click in the Q&A box on the left side of your screen
Type your question into the dialog box and click the Send button

Please take a moment to fill out the brief evaluation which will appear in your browser.