The Research Design Checklist: A Guide for Developing a Research Project
Dr. Ken Mease
This document will help guide you through the development of your research design (RD). It outlines
what is necessary for a proper research design and summarizes many of the important concepts. It
does not replace critical thinking. There is no one size fits all when it comes to solving problems and
answering burning questions. Please read it very carefully; I suggest at least twice. For more in-depth
information, look at Russ Bernard's Social Research Methods, Sage Publications, 2000. This is a
good site for basic information on research methods and statistics: www.socialresearchmethods.net/

There are many other sources available to help you design a research project. However, if you
want my help, you must use this guide as a template; use the exact same section headings in
the same order in your RD. It is best to begin writing your RD in a copy of this file by removing
most of the explanatory text and just keep the headings and the Section Checklists. Start by
concentrating on Sections 1 through 10.

The Section Checklists, located at the end of most sections, remind you what you need to
address in each section. A research design is not complete until every item in the Section
Checklists is addressed. Please examine the FINAL CHECKLIST before sending me your RD.
What’s Inside

Helpful Hints

Sampling

Problem Statement

Methodology and Data Collection

Literature Review

Data Analysis

Hypotheses

Potential Problems

Variables and Data

Important Contacts

The Model

Research Schedule

Surveys

Budget

The TABLE

The FINAL CHECKLIST
Appendix: - Some Examples and Helpful Information.







Example of Hypotheses (Section 3)
Example of a Model (Section 5)l
Example of Survey Questions (Section 6)
Example of the TABLE (Section 7)
The RD Continuity Crosscheck Table (not required in your RD)
Writing better Questions and Items –Read this before working on your survey. (Section 6)
Quick Guide to Bivariate and Multivariate Statistical Testing (Section 9)
1
Helpful Hints
Types of RDs.
1. Traditional RDs – where you are trying to explain some behavior or outcome. Here there are usually a
set of independent variables (IV) and usually one dependent variable (DV), a classic Model (Section 5)
with IVs on the left and a single DV on the right. This type of RD also fits nicely into TABLE (Section 7).
2. Evaluation – this type of RD seeks to evaluate or determine if there is a need and or support for a
policy, program, process or approach. Often Best Practices (BP) are used to compare what exists to
what is considered Best Practice. In cases where you use a BP approach, you may end up developing
hypotheses based on various elements of what is included in BPs. Where needs or support are
involved, your RD may end up looking similar to a Traditional RD.
3. If you are examining more than one population – say regular staff and managers - you have two
populations, and may need two sets of hypotheses, models, tables, surveys, etc.
Section 1. Problem Statement and background to the problem – 1 to 3 pages.
Section 2. Literature Review (LR). Just show me the short versions (summaries) of the major research you will
use. I do not need the full length version, save that for your final paper (2 to 3 pages). Most important:
summarize the research reviewed; tell me which research will guide your project with specifics; and then
introduce your own research project (1 to 3 pages). Combined total: 6 pages max.
Section 3. Hypotheses. Whenever possible they should be short and with only one independent (IV) and
dependent variable (DV). Use the same words every time you mention your DV. Sometimes there is a need
from more, but not that often. I will talk about how you structure an evaluation and how it differs from RDs with
a single dependent variable and a traditional model. (1 page; Maybe 2 pages with more than one set of hyps)
Section 4. Variables, Data and Surveys. This is the most overlooked section, even though it is one of the most
important. The use of headers here helps me. Make sure you have addressed all the items in the Section 4
Checklist. If I don’t have a good understanding of what your variables are and how they are measured, it really
limits what I can do. (1 to 2 pages)
Section 5. The Model. This is graphic representation of your research. Whether you need one depends on
what type of research you are carrying out. (1 page; 2 pages with two models)
Section 6 . The Survey. Some RDs use semi-structured survey, others structured and some both types.
Section 7. The TABLE. This shows you and me that what you have in your hypotheses, variables, survey and
model are all in sync. (1 page; 2 pages if you need a second TABLE)
Section 8. Sampling. Follow the checklist for this section. You may want to do a census. I don’t need a long
discussion about sampling here. Please just answer the items on the checklist (1 – 2 pages).
Section 9. Methodology and Data collection. Many students do a Case Study; some use the Statistical Method
with a Case Study; and a few do Quasi-experimental designs. (1 to 2 pages)
Sections 10. Data Analysis. Shows you the options you will have in analyzing your quantitative data will be
determined by the level of measurement of your variables. Text data (words) must also be analyzed rigorously.
(1 to 2 pages)
Sections 11 & 12 are important to think about now. Sections 13 & 14 can wait.
2
Section 1. The Problem Statement. The first step is to clearly state the problem or objective of your
study. Discuss the problem and provide some background statistics and/or history with sources
(Citations Please!). Sources can include newspapers, studies, surveys, previous research, books,
company documents or reports, the internet, etc. A general discussion comes first, followed by one or
more (not too many) research questions aimed at addressing the problem in a meaningful way.
Section 2. Literature Review. It is always important to place your intended research within existing
research. With a little effort you will be able to find scholars and practitioners who have written about
and/or examined the same or similar topic, problem or research question. Theory from previous
research on most topics is not impossible to find - go see the Reference Librarian. While you will
need to spend some serious time completing your LR, you can often get started using Google. Do not
re-invent the wheel.

In this section, you review the literature, theory and research associated with your topic; pick
the literature that will guide your own work; and briefly introduce your research project. Think of
it as a funnel; from a broad discussion narrowed down to your topic and setting. Few research
questions have not already been asked.

Theory Defined - a set of interrelated propositions that suggest why events occur in the
manner in which they do. Think of theory as a conversation you want to join and add
something to. This is a conversation among social scientists or practitioners who share an
interest in your topic.
Literature Review (LR) Checklist
1. Keep things consistent and in context is critically important. Make sure you use the LR in
the rest of your research design - Sections 3&4&6 – Hypotheses, Variables and Sampling
2. The Last Section of the LR: Please use the three headers (in bold) below. Depending on
the amount of literature, this section should be between 1 and 3 pages.
a. Summary of all Literature Reviewed: Synthesize what you have introduced and
explain the relevance to your own research.
b. Literature that Guides this Research: Narrow the LR down and tell the reader
which literature/research will guide your research. Provide specifics about the
authors, years, main points, variables, hypotheses, etc.
c. This Research: Introduce your research project. Tell the reader a about where it will
take place, who will be involved. Restate your major research question and goals.
Section 3. Hypotheses. Now that you have placed your work in a theoretical context, you need to
develop some clearly stated hypotheses.

Hypothesis Defined. A hypothesis is used to organize a study; a hypothesis proposes a
relationship between two or more variables. Start building your hypotheses with just two
variables – an independent (IV) and dependent (DV) variable. Keep it Simple Please (KP). See
examples on Page 13. Cite the theory or previous research like this: (Mease, 2013, p. 1)
3

While many RDs have a single dependent variable and a number of hypothesized independent
variables, some RDs don’t. Sometimes, the evaluation of a program, policy or practice is the
focus of the research and may have several dependent variables. In this case, there may be
several hypotheses linked to best practices. Here are some examples of approaches used in
evaluations: comparing the policy/program to established best practices; investigating the level
of support for or need for a program/policy (a needs assessment.)
Hypotheses Checklist
1. Hypotheses should be linked to theory or an existing body of knowledge, and be testable. It
is OK to develop your own hypotheses, especially in the beginning.
2. Whenever you use information from another source, you must cite it: (author/source, year,
page#). Please cite the page number/s in all citations. Please, no footnotes.
3. A hypothesis is always statement, never a question!
4. It must clearly identify both an independent and dependent variable and make a prediction
about their relationship, as shown in the example above. The independent variable is
mentioned first, and then its predicted impact on the dependent variable
5. Keep your language simple and similar in each hypothesis. This is not a time for using
different words to describe your DV. Keep it boring and repetitive
6. Very Important If you have more than one population (group fo people to interview) , you
will likely multiple sets of hypotheses, models, surveys and sampling strategies. Think
about this now!
Section 4. Variables and Data. This is a long and very important section. Read it carefully and take
your time. Be sure to double-check the Section Checklist at the end of this section. Please use the
headers below: these are the only areas I need to see in this section. No tables please.


Variables – type, definition, level of measurement, significance to the study
Data Type/s - short explanation
Variables - A variable is something that takes on different values that can be measured or
counted. Variables represent the operationalized equivalent of the concepts in your hypotheses.

Independent variable (IV) causes or is associated with a change in the dependent variable.

Dependent variable (DV) – sometimes the toughest to measure. It is usually the focus of
your research - the process or behavior you are trying to explain. The best way to start is
with one dependent variable. Sometimes the DV is made up of several related elements.
Levels of Measurement. Always measure at the Highest Level Possible

Nominal – Yes/No; True False and other categories, such as male/female, religion, race,
marital status, occupation, location, etc. Lowest level of measurement
4

Ordinal - the attributes can be rank-ordered, but the distances between attributes do not
have a precise meaning. For example, the distance between Agree and Strongly agree in a
Likert scale may not be the same for everyone. The same can be said for other ordinal
scales.

Interval - the distance between attributes has meaning – for instance, distance can be
measure in miles and the distance between each mile year is known and equal.

Ratio - there is always an absolute zero that is meaningful. This means that you can
construct a meaningful fraction (or ratio) with a ratio variable. Distance traveled, which
starts at 0 is a good example of a ratio variable. Highest level of measurement.

Measurement Hierarchy – It is always best to use the highest level of measurement
possible. Example: Think of using a scale (ordinal) rather than a YES/NO question
(nominal). The lowest level is Nominal, highest is Ratio. As you move up, the current level
includes all of the qualities of the one below it and adds something new. Generally you can
move from a higher level of measurement to a lower one, but not the other way around.
Example - you can recode age measured intervally in years into ordinal categories: 18-25;
26-40; and over 40.
What is an Indicator?

An indicator is a device for providing specific information on the state or condition of
something. An indicator is also a measure, gauge, barometer, index, sign, signal, standard,
touchstone, yardstick, benchmark, criterion and point of reference. Source: Oxford Dictionary

An indicator can be numerical or text based (qualitative). It measures the quality of life in a
country, such as governance, democracy or human rights. Indicators can be used to illustrate
the progress of a country or an organization in meeting a range of economic, social, political or
environmental goals.
Types of Indicators

Objective indicators can be developed from archival or secondary data sources. Some de
jure examples include the existence of an integrity commission, existence of a law
protecting human rights, freedom of speech or some other civil liberty. De facto (practice)
examples include the number of corruption cases prosecuted or the number of human
rights violations reported.

Reported behavior or events-based indicators are often used in surveys, but can also be
developed using data from other sources. These indicators can come from sources such as
government statistics, company documents, etc. Example: A citizen survey might ask
respondents if they have ever been asked to pay a bribe to a public official (de facto).

Subjective or Perception based indicators are often found in surveys of typical citizens or
smaller surveys of workers or key stakeholders (experts). These indicators rely on opinions
or perceptions of how things are (de facto). Even though people often say one thing and do
another, perception based data have proven to be very reliable over the years in many
5
different contexts and cultures. Moreover, in many cases the only data available are based
on people’s perceptions.

Proxy indicators measure the subject of interest indirectly, rather than directly. Sometimes
issues of time and money influence the need to use proxy indicators. Other times,
researchers use proxies for subjects that are difficult to measure directly such as income in
developing countries, so researchers often use proxies, such as the roofing material of the
house, ownership of livestock, ownership of land, etc.
The De facto and De Jure States (not critical, but important to consider)

The de facto state refers to what happens in practice. An example here you might want to
know - the degree to which there is freedom of the press in a country

The de jure state refers to the existence of formal rules. These formal written rules are
often found in policies, laws, regulations and constitutions. For instance, is there a law or
constitutional provision protecting freedom of the press?

A well designed project examines both the de jure and de facto states. This is because
sometimes rights or laws may exist on paper (de jure), but not in practice (de facto).
Using Multiple Indicators to Measure a Concept: An index or a scale?
Index: An index is a numerical measure that allows the user to compare results overtime. A
well-known scale is the UNDP Human Development Index (HDI). The HDI is a combination of
many different indicators from different sources, such as GDP, literacy rates, educational
enrollment, life expectancy, number of telephone lines, etc. Weighting of the individual
indicators is usually required and best left to experts. It is important to consider the sources
and quality of information used when interpreting an existing index.
Scales: A scale is a set of numerical values assigned to subjects, objects, attitudes or
behaviors for the purpose of quantifying the qualities. Scales measure the degree to which an
individual or object possesses the characteristic of interest. Scales usually rely on original
data, but it is possible to use secondary data to develop a scale. Scales are easier to
understand and construct than an index. Example: a job satisfaction scale. There are many
tried and tested scales available. Get started with a simple Google search.
Understanding the Quality of Indicators

Reliability: Can the results you get be reproduced with a similar sample? Even if the data and
indicators are not survey based, reliability is still important. For instance, in a desk study one
would expect to get similar results from a different expert examining the same information.

Validity: Does the indicator measure what it is supposed to measure? Sometimes, we think
(assume) an indicator is measures a construct accurately, but we find out later (not good) it
does so poorly or is measuring something completely different. Validity issues can arise in both
survey and non-survey-based indicators. Like reliability, there are tests available for validity.
6
Types of Data - Regardless of which type or types of data you choose to collect, you cannot
escape developing a complete RD. Sources of data include - surveys, administrative data,
documents, laws, policies, manifestos, national statistics, newspapers, etc.
Qualitative Data




A strictly qualitative approach usually does not involve discreet variables that lend
themselves to precise measurement (numbers, ratings, etc.).
Instead qualitative work may rely on more general concepts, and collects and interprets
words or text data.
These studies are sometimes referred to as ethnographic research.
Text data can be interpreted and analyzed with software. This software ranges from simple
shareware (free) like EZ-Text and AnSWR to more complex and expensive products such
as NVivo and ATLAS TI.
Quantitative Data




Quantitative data express a certain quantity, amount or range.
These data are numeric and lend themselves to statistical analysis
Popular statistical programs include SPSS, Stata, SAS and R (which is free)
Structured surveys, often used in larger projects, produce quantitative data
A Mixed Approach



Taking a mixed approach means you will have both text and numeric data.
Often a semi-structured approach to interviewing is employed.
This approach is useful for smaller populations or with elite respondents where there is a
need for a deeper understanding of context. However, it is also employed in larger projects
Unit of Analysis. It is the major unit of your research. If you start out with individuals as your unit of
analysis (focus), don’t switch or draw conclusions about organizations. If you start examining
characteristics of organizations, don’t start talking about individuals in the organization. This is
usually not a problem. I’ll let you know if you have this problem. This also applies to Section 3 –
Hypotheses.
Section 4 Checklist: - Please make sure each area is complete. No Tables in this section

Variables:

Identify the type of variable – independent or dependent

Provide a clear definition of each variable

Explain precisely how you intend to measure it – i.e. years, levels, with a scale or index.
If doing a survey, try to craft a question with response codes. Also provide the level of
measurement – nominal, ordinal, interval, or ratio

Briefly explain the significance of each variable to the study - look to theory,
hypotheses or your research question

Type of Data - Discuss the type/s you will collect – quantitative, qualitative, mixed.
7
Section 5. The Model. It is a representation of theory - a graphic illustration of how your variables are
related to each other (IVs to the DV). Everything in your model should be found in Section 4’s
variables and Section 3’s Hypotheses. Please use the format shown in the example on page 13.
**Note: If you are evaluating a policy or program, you may not have a traditional set of independent
and dependent variables and therefore you may not have a Model.
Section 6 The Survey. Most RDs have at least one, but sometimes you don’t need to talk to anyone
because your data comes from other sources. Discuss the type/s of survey/s you will use. Will you
use unstructured, semi -structured or structured? Briefly explain your choice
Types of Surveys
If you are going ask people questions, develop a survey (maybe more than one) and be sure to link
every question to a variable or concept (independent and/or dependent), and also to one or more of
your research hypotheses. See example (without response categories) on Page 13..



Fully structured - all respondents get an identical instrument, that usually comes with explicit
instructions to and from the interviewer. Generates primarily numeric data – quantitative
approach
Semi-structured – You are in control of the interview, and usually have a set of questions that
every respondent receives. The open-ended sections seek the context and meaning behind
the answers given to the structured items. There may be other open-ended items not directly
connected to a structured item. Generates text and numeric data – mixed approach
Unstructured The informant is someone you may have known for a period of time and knows
something about particular subject. Not often used alone and only generates text data qualitative approach
Section 6 Checklist:



Make sure each variable in this section is in your survey, unless you are using other source of
data.
Be sure to examine the Guide to writing better questions. Look here for a variety of tried and
tested response scales: dataguru.org/ref/survey/responseoptions.asp
Very Important - Be sure to link each survey question to a Variable and a Hypothesis. Ex.
What is your age? (IV Age - Interval) ( H1). See example on page 13. You must have
response categories in your survey.
Section 7. The TABLE. It shows me (and you) that Sections 3, 4, 5 & 6 (hyps, variables, model and
survey or source of data) are in sync. See an example on page 14.
Section 8. Sampling. Almost every research project needs a sampling strategy. This goes for both
the qualitative and quantitative work. Social scientists rely upon sampling to make inferences about a
population. The population is the entire group of elements about which we would like to know
something. A sample is a subset of these items. If you have more than one group/population, you
should handle each separately. No tables please unless you have a large number of groups.
8
Key Terms


Census - If your population is small you may want to skip sampling and do a census. Or if you
can access the population using an internet survey, you may want to do a census. Even if your
population is between 300 and 500 you should do them all – i.e. a census. The main incentive
is that there is no sampling error in a census.
The sampling frame contains all the eligible elements for your study. Examples of sampling
frames include a voter’s list, list of employees, membership list, or a telephone book.

A Statistic – is a characteristic of a sample, used to estimate parameters of a population.
Subject to error as it is an estimate – such as the mean age of a sample that is used to
estimate the average age of the population

A Parameter – is a characteristic of a population – such as the average age. We use statistics
to estimate parameters
Two Types of Sampling – Probability and Non-probability
1. Probability Sampling - each element in the sampling frame has a known chance of ending up
in the sample. Preferred if you are planning on doing statistical analysis.

Simple Random - Everyone in the population has an equal chance of being selected.

Systematic random sampling – Great if you have a list of those in the population. You enter
the sampling frame (list) using a random number and then interview every Nth person. The
“Nth” is determined by the size of the sample in relation to the size of the population.
Example – population of 2000, desired sample of 200, “Nth” would be every 10 th person.

Stratified sampling - you select a criterion (strata), such as sex, religion, urban, rural, etc.
Beware of stratifying; once you start it is hard to stop. You should only stratify if the variable
is critically important to the research and a random sample will not provide enough
observations to test your hypotheses. Do not go overboard – more than 2 or strata can get
very complicated and hard to do! Usually only used in large N (sample) studies.

Multistage sampling - first you randomly select a set of geographic regions; then a subset
of the geographic area (town) is sampled within each of the regions and so on down to a
street, or small section of a village. More complicated.

Cluster samples - rather than travel to each and every neighborhood, or village, the
researcher clusters groups of interviews. The representativeness of this type of sampling is
only as good as the clusters selected. Clusters need to be representative of the population
of interest. Can be very complicated and difficult to do well.
2. Non-probability Sampling implies that personal judgment has somehow been involved in the
decision about which elements to include in the sample. Where units of the population do not
have an a priori known and nonzero probability of being included. Statistical analysis is still
possible.

Purposive or Judgment samples are used for their supposed representativeness of the
desired population as determined by 'experts', or because the elements in the sample are
capable of offering the desired information.
9

Snowball sampling – when you ask a respondent to recommend someone else who is
knowledgeable about the topic you are interested in learning more about. Often used in
conjunction with purposeful or judgmental sampling to expand the sample of known experts
on a subject

Quota sampling – you seek a specific number of a certain type of respondent based on
some criteria, such as sex, region, race, education, etc.

Fixed or Sequential - Fixed samples have a pre-established sample size associated with
them, while sequential samples vary in size depending upon the conclusiveness of
previous samples. If results are not conclusive with a sample of 200, a researcher may
measure additional 200 elements to seek more conclusive results. Not often used.
Sampling Error/ Margin of Error (MOE)
The amount of error associated with a sample not representing the population on the measure of
interest is called sampling error. It is important that you know your sampling error or the margin of
error (MOE). The larger the sample size, the smaller the MOE. The bad news is that small samples
have large sampling errors. The size of the MOE can impact the conclusiveness of your findings. If
your population is small, using the finite population estimate will reduce the MOE.
Here is a link to a sample size calculator: http://www.surveysystem.com/sscalc.htm
Examples of Sampling Error Rates:
o
a sample size of 1000 has a MOE of around +/- 3%
o
a sample size of 500 has a MOE of around+/- 4.3
o
a sample size of 200 has a MOE of around +/- 7%
o
a sample size of 100 has a MOE of around +/- 10%
o
a sample size of 50 has a MOE of around +/- 14%
Sampling Checklist
1. If you have more than one group/population, you should handle each separately and address
the items in this list for each one. No tables please unless you have a large number of groups.
2. In plain English say who you want to talk to.
3. How many are in the population?
4. How many will be in your sample of the population/s (and possible subgroups in the
population)?
5. Why you chose them?
6. Where they are located - country, region, city, village, etc.?
7. The type of sampling you intend to use?
8. Explain exactly how you will choose them – and I mean exactly - you can’t just say ‘randomly’
7. Be sure to explain why you chose the type of sampling and point out any strengths or
weaknesses it might have compared to other sampling methods.
10
Section 9. Methods and Data Collection. Choose which type or types of data collection techniques
best suits your topic and your budget!
Major Social Science Research Methods:

Case Study Method - If you use a single country or case, the study will be interpretive. These
single case studies use theory but because it is related to a single case, there can be no
theory building. However, if you use multiple cases there can be theory confirming and
infirming.

Statistical Method - is an approximation of the experimental method and is facilitated by the
use of statistics. It entails the manipulation and testing of empirically observed data, often
collected from interviews/surveys.

Quasi-experimental method. First you find groups or geographic areas where something has
occurred or something implemented, like a policy or development program. This is called the
treatment. Then, you locate other groups or areas where there has been no treatment. The
idea is to compare the areas that receive the treatment to the areas that have not. The areas
or groups, except for the treatment, should be as similar as possible

Experimental Method uses a control and an experimental group. Only the experimental group
is exposed to the treatment. Rarely used.
Data Collection Options:

If you are doing a survey, how will you conduct it? You can choose from face-to-face, Internet,
telephone or self-administered.

If your population has access to the internet and you can get their email address, I strongly
suggest doing an Internet survey using Survey Gizmo – account is free for students

How will the respondent be contacted – will you send a letter, email, phone call, etc.?
Methods and Data Collection Checklist
1. Discuss the method you will use and justify it
2. Explain how you will collect your data and how will contact your respondents
Section 10. Data Analysis. It is never too early to think about how you will analyze your data. Think
about how you will present your findings (frequencies, means, tables, sub-groups (like men and
women)

Talk specifically about your dependent and independent variables and the tests you intend to
use to test your hypotheses – see pages 17 and 18. The types of statistical testing you will use
are directly related to your sample size and the level of measurement of your variables –
nominal, ordinal, interval or ratio.

In the case of qualitative work, you will need to discuss how you will analyze your text data.
There is a method for doing this type of analysis, just like quantitative studies. For help, see
Bernard’s Social Research Methods, Sage Publications, 2000, or search on Google.

If you are creating a new measurement tool, you’ll need to talk about how you will test it for
reliability and validity. Another good reason for using tried and tested indicators.
11
Section 11. Potential Problems. Think ahead and try to identify potential problems before they
happen and work out solutions.
Section 11. Important Contacts. Discuss your plans to make contacts - who and why. These
contacts may be critical to the success of your research project!
Section 12. Budget. There should be a detailed list of all expected expenses




Materials
Transport
Living expenses
Research assistants or other labor
Section 13. Schedule. This schedule should cover all aspects of the research process




From preparing to go into the field, to survey development, pre-testing training, etc.
How long you will be in the field?
How much time needed for the analysis?
How much time to write up, defend and re-edit?
The Final Checklist for Sections 1-10
1. Make sure the literature and research you use in your LR get used on your hypotheses,
variable, model and survey sections.
2. Make sure you complete the Last Section of your LR. It is critical for me to be able to help you.
3. Think about how many different groups (populations) will be in your study – this can impact the
hypotheses, model, sampling, and survey – i.e. how many of each you will need.
4. I need to know about your variables. Make sure you all address items under Variables in
Section 4’s checklist. Do not put this in a table.
5. In your survey (Section 6), please make sure you identify the variable type IV or DV, the level
of measurement and the hypotheses that each question is linked to (you may not be able to
link all of them, but if none are linked, you have a problem. That said, you may have questions
that you want to ask that are outside your hypotheses and your model, but important to your
study.
12
Examples of Hypotheses (Section 3) Based on a Study of the Quality of Governance, with
Citations linked to theory/research presented in the Literature Review (LR)
H1 Older people will rate governance higher (Mease,2004, p.1)
H2: People with higher levels of education will rate governance lower (Mease,2004, p.20)
H3: Those with higher incomes will rate governance higher (Mease,2004, p.45)
H4: Those identifying with the PNM will rate governance lower (Mease,2004, p.65)
H5: Women will rate governance higher (Mease,2004, p.88)
H6: Single people will rate governance higher (Mease,2004, p.104)
H7 Hindus will rate governance higher (Mease,2004, p.145)
(H8) Indians will rate governance higher (Mease,2004, p.175)
**********************************************************************************************************
Example of a Model (Section 5)- Study of the Quality Of Governance (Mease, 2004, p.22)
INDEPENDENT VARIABLES
DEPENDENT VARIABLE
AGE
EDUCATION
INCOME
PARTY ID
GENDER
QUALITY OF GOVERNANCE
MARITAL STATUS
AT THE NATIONAL LEVEL
RELIGION
ETHNICITY
**********************************************************************************************************
Examples of Survey Questions without response categories (Section 6) ; Study of the Quality
of Governance. Showing links to the Variables (Section 4) and Hypotheses (Section 3)
Q1. What is your age? ____years (IV Age) (H1)
Q2. What is your highest level of education?(IV ED) (H2)
Q3. What was your total income last year in dollars?(IV Income) (H3)
Q4. Which political party do you feel closest to? (IV PID) (H4)
Q5, What is your gender? (IV Gender) (H5)
Q6. What is your marital status? (IV Marital) (H6)
Q7. What religion, if any do you practice? (IV Religion) (H7)
Q8. Which ethnic or racial group do you feel closest to?(IV Race) (H8)
Q9. How do you rate the quality of governance at the national level. (DV Governance) (H1-8)
13
Example of the TABLE’ (Section 7) Based on a study of the Quality of Governance- Connecting the Dots: Sections 4,5,6,&7 Hypothesis, Survey Question, IV or DV, the Variable, How it is Measured, Level of Measurement and Data Source
Hypotheses
Question #
Independent, or
Dependent
variable
Variable
How measured - Response
Categories
Level of
measurement
Source of
Data
H1
Q1
Independent
Age
Years
Interval
Survey
H2
Q2
Independent
Education
Level of education
Ordinal
Survey
H3
Q3
Independent
Income
Income from all sources
Ordinal
Survey
H4
Q4
Independent
Party Identification
PNM, UNC, COP, Other
Nominal
Survey
H5
Q5
Independent
Gender
Male/female
Nominal
Survey
H6
Q6
Independent
Marital status
Married, Single, Divorced
Nominal
Survey
H7
Q7
Independent
Religion
Hindu, Christian, Muslim
Nominal
Survey
H8
Q8
Independent
Ethnicity
Afro, Indo, Mixed, Other
Nominal
Survey
H1-H8
Q9
Dependent
Quality of
Governance
5 point Scale- Very Low,
Low, Moderate, High, Very
High
Ordinal
Survey
14
RD Continuity Crosscheck Table: X = Should definitely be in this section. See note below on LR*
Research
Questions
*Literature
Review
(LR)*
Model
Sampling
Strategy
Survey or
Other Source
of Data
Hypotheses
Age
Maybe
X
X
X
Maybe
X
Education
Maybe
X
X
X
Maybe
X
Income
Maybe
X
X
X
Maybe
X
Party ID
Maybe
X
X
X
Maybe
X
Gender
Maybe
X
X
X
Maybe
X
Marital Stratus
Maybe
X
X
X
Maybe
X
Religion
Maybe
X
X
X
Maybe
X
Ethnicity
Maybe
X
X
X
Maybe
X
X
X
X
X
Maybe
X
Variable
Quality of
Governance
* Almost always, but sometimes there are factors that previous researchers have not taken into account. Of course, there may be
hypotheses and/or variables outside your model that are important. This continuity check does not take them into account.
15
Surveys: Writing Better Questions and Likert type Items – Read this before you develop your Survey
2. Make sure the question addresses your research: Always refer back to your theory and hypotheses. Make sure the question is
reasonable. Avoid asking people what other people think. It is best to have them report on their own beliefs and behaviors.
3. Avoid Yes/No questions; try to use scales such as Likert (statements) and others, which will increase variance and provide more
option when you do analysis. Likert Scale mid-points – There have been many studies on offering a neither agree nor disagree
option. There are no conclusive answers. I prefer to use 4 points and skip the mid-point – no neutral.
4. Avoid double-barreled questions, meaning there are actually two or more questions in one question. This is a common problem
3. Avoid using jargon. This is an error frequently made by those experienced with and/or very close to the topic of interest
4. Be aware of transitioning between topics. Some common transition phrases are, “Next I would like to ask you about” or “Now I
have a few questions about”. This alerts the respondent that the topic is going to change.
5. Be aware of the length of your survey. A good time for most surveys is no more than 10 to 15 minutes. Anything beyond 30
minutes runs the risk of refusals or getting less than thoughtful answers. However, if the topic is of great interest to the
respondent, longer surveys are possible with good results.
5. Avoid loaded questions. Loaded questions are those that indicate a position in the wording of the question. Consider measuring
behavior, rather than opinion. It is usually preferable to get reports of past behavior to predict future behavior, rather than just
relying on opinion questions to predict behavior.
6. Beware of problems associated with respondent recall. A respondent’s memory is not infallible. Aided versus unaided recall:
Sometimes respondent recall can be improved by cueing them with potential response categories that they wouldn’t remember
on their own. Be aware that with long lists the items at the top will be listened to more carefully than those farther down. One
solution is to randomize the list order.
7. Open-ended questions are often used in small studies using a semi-structured approach. Close-ended questions are nearly
always preferable in large N (sample) studies.
9. Be aware of the placement of sensitive questions. Most people don’t refuse to finish a survey once they have started it. It is
always a good idea to put potentially sensitive questions at the end. Example - income questions.
12. Avoid lengthy explanations and never make the answer to a question required.
13. Question order: In many cases, question order can have a big effect on responses to future questions.
14. Make sure you have an intro stating that participation is voluntary and that responses will be kept confidential.
16
Quick Guide to Statistical Testing
Bivariate - Conducting a Statistical Test with
One Dependent and One Independent Variable
Dependent Variable: Level of Independent Variable: Level of
Measurement
Measurement
Appropriate Statistic or Test
Nominal
Nominal
Chi Square
Interval, Ratio
*possibly Ordinal
Nominal
Kruskal-Wallis - good for non-normal dependent variable
Interval, Ratio
*possibly Ordinal
Nominal
T-Test – Independent var or Grouping variable must have no more
than 2 values
Interval or Ratio
Nominal
One–Way ANOVA
Ordinal, Interval or Ratio
Ordinal, Interval, Ratio
Spearman Correlation
Interval or Ratio
Interval or Ratio
Pearson Correlation
* Stretching the rules of statistics - many researchers treat ordinal measures, such as a Likert Scale item, as interval.
17
Multivariate - Conducting a Statistical Tests with
One Dependent and Multiple Independent Variables
Dependent Variable:
Level of Measurement
Independent Variables :Level of
Measurement
Appropriate Statistic or Test
Nominal
Nominal, Interval, Ratio
Logistic Regression – nominal dependent variable: may only have 2
values.
Nom. indep. vars must be coded as ‘dummy ‘vars.
Nominal
Interval and Ratio
Discriminate Analysis
Ordinal
Nominal, Interval, Ratio
*possibly Ordinal
Probit – Nominal Independent vars. must be coded as ‘dummy‘ vars.
Interval or Ratio
Nominal
Factor Analysis – using ANOVA
Interval, Ratio or
*Possibly Ordinal too
Nominal, Interval, Ratio
Multiple Regression - Nominal Independent vars. must be coded as
‘dummy ‘variables
Possibly Ordinal too
* Stretching the rules of statistics - many researchers treat ordinal measures, such as a Likert Scale item, as interval.
18