The Research Process: An Overview


◦
◦
◦
◦
Research is a systematic investigation designed to develop or contribute to generalizable knowledge a planned investigation of gathering and analyzing data from a sample to draw conclusions about a research question that can be applied to a larger population.
From a discipline perspective, research is like diagnosis, particularly differential diagnosis, a processes of asking and answering questions a process of formulating “answerable” questions or forming
“testable” hypotheses, and then making the observations needed to answer them.

The Research Plan

 o o o o o
Most research plans evolve from the following factors:
Observation of a problem that leads to a question;
Development of a problem statement in the form of a testable hypothesis;
Use of appropriate methods for testing the hypothesis;
Statement and interpretation of results; and
Discussion and evaluation of the results.
Guiding the research plan are the rules of the scientific method that o o o recognize that a problem can be studied objectively; data can be collected through observation or experiment; and conclusions can be drawn based on an analysis of the data that have been gathered.

Clinical Research and Evidence



Like the research plan, the scientific method can be used to answer clinical questions and gather clinical evidence as well.
The challenge for all practitioners is to answer the question:
“ How do you know that what you do works?”
Answering clinically relevant questions or testing clinically relevant hypotheses for clinical purposes should be no different than for research purposes.
ASHA’s Code of Ethics (2003) states that “individuals shall evaluate services rendered to determine effectiveness.
”

Clinical Research and Evidence


In order for clinicians to function ethically, they must systematically evaluate the impact of the services they render.
They must move away from decisions based on opinion, past practice, and past teaching towards clinical decision-making that is guided by science, research and evidence (Elliott, 2004).

Evidence-Based Practice



Evidence-based practice is more than clinical problem-solving.
It requires SLPs to integrate their individual clinical expertise with the best available evidence from systematic (scientific) research to demonstrate that what they do works.
Like the scientific method, evidenced-based practice involves the following steps:

Evidence-Based Practice
 Converting a clinical need into an answerable question;
 Searching for and finding the best evidence to answer the question;
 Critically evaluating the evidence you find for its validity, strength, and applicability to the individual client;
 Applying the results of the search and appraisal to clinical practice; and
 Evaluating or auditing your performance.




Empirical research is based on observed and measured phenomena.
It involves questioning, observing, experimenting, and eliciting behavior to: o o o define relationships demonstrate cause and effect and spark our own minds to begin thinking of other possibilities to be tested and studied.
Empirical research leads to knowledge from actual experience rather than from theory or belief.

Empirical Research Designs: Purpose




Empirical research can be classified by its purpose or by the type of knowledge that will be produced.
The three levels of classification are exploratory, descriptive, and explanatory.
Exploratory (preliminary) research is typically conducted in the field and/or in natural settings or in non-natural settings with the purpose of discovering phenomena and/or theory.
Exploratory studies require no active manipulation or alteration of the research context/condition by the researcher.

Exploratory Research



It is typically conducted early in a research problem to uncover basic viewpoints, perceptions, behaviors, attitudes.
It can be quite informal, relying on secondary research, such as literature or data reviews, or qualitative approaches, such as informal discussions with clients, employees, management or competitors.
More formal approaches would include in-depth interviews, focus groups, case studies or pilot studies.

Exploratory Research
 Liebow (1967) conducted exploratory research through extensive observation and interview techniques to obtain stories of men’s lives on Tally’s
Corner, an urban street corner in a black neighborhood to explore the meanings of their experiences.

Liebow wanted to illuminate the richness of the lives of these men as well as struggles that each man encountered when attempting to fill varied life roles.
 His purpose in conducting an exploratory study was to reveal theory, not to support or test existing theory.

Exploratory Research



The results of exploratory research are not usually useful for decision-making by themselves.
The results can not be generalized because they are not representative of the whole population being studied.
But they can provide significant insight into the best research design, data collection method or subject selection procedure.

Descriptive/Ethnographic Research



Descriptive/ethnographic research involves the study of human behavior through the description or reconstruction of events that are observed as they naturally occur.
The ethnographer collects naturalistic data which is then arranged to render complete descriptions of activities and interactions.
Ethnography is an appropriate methodology for studying the customs, social patterns, speechlanguage behaviors, and rule-governed interactions of a culture or group of individuals.

Descriptive/Ethnographic Research



With descriptive research, data are merely described and information is explained in relationship to the individual studied.
Categories of analysis are generated and the relationship between the categories are explained.
No inferences to a larger group or population can be derived from the analysis.

Descriptive/Ethnographic Research



A classic example of naturalistic research is embodied in the work of Piaget (1932).
Piaget studied language development by primarily observing and recording children's questions, reflections, and conversations.
From such qualitative methods, he categorized various stages of what he termed “egocentric” and
“sociocentric” speech development that, in turn, stimulated much additional research.

Explanatory (Experimental) Research



Explanatory or experimental research is designed to reveal causal relationships and to predict outcomes.
Such studies are founded on an accepted theoretical framework of reference and seek to support theory through hypothesis testing and prediction.
It essentially involves comparisons of groups or individuals and relies on measurement and statistical analysis of quantitative data.

Explanatory (Experimental) Research



Let’s say for example you wanted to know if a hospital-based group socialization program was effective in reducing depression in young adults with a spinal cord injury.
You might structure the study so that these patients were randomly assigned to either an experimental group, in which the socialization program was conducted, or to a control group, which received individual counseling.
You would use a standardized depression scale that fit your theoretical framework.

Explanatory (Experimental) Research


All participants would be tested on the scale before their participation in the experimental and control group programs and after the conclusion of the programs.
A statistical comparison of the changes in depression scores between the two groups would be used to determine the extent to which there is a causal link between the socialization program and lowered depression.

Empirical Research Designs: Structure



Empirical research designs can also be classified by the way in which data gathering and analysis is organized and structured.
The first classification system is used to examine and analyze the structure of experimental-type designs.
There are three levels: experimental, quasiexperimental, and non-experimental.

Empirical Research Designs: Structure





The second classification system is based on the nature of the data collected and the type of analysis conducted.
Using this scheme, studies are classified as either
“qualitative” or “quantitative” in their structure.
Those that use numerical data and statistical analysis are quantitative.
Those that rely on narrative and categorical analysis are qualitative.
Let’s start with looking at the structure of experimental-type designs.

Experimental Designs


Experimental research is the appropriate method for investigating cause-and-effect relationships among variables, such as the effects of treatment on speech or language behavior.
True experimental designs have the following characteristics: o
Subjects are randomly assigned to at least two or more groups; o
Some type of active manipulation is performed; and o
One group of subjects (experimental group) is then compared with another non-manipulated group (control group) (Maxwell & Satake,1997).

Experimental Designs



Typically, quantitative measurements and methods of data analysis are used to compare the experimental and control groups.
The results are presented as quantities or numbers
(e.g., statistics).
Based on the results, the researcher is able to support or refute the original hypothesis.

Experimental Designs


Additionally, because experimental designs control for extraneous variables, the research is able to generalize findings to the larger population from which the sample is drawn.
Nonetheless, for many practical and ethical reasons, it is sometimes impossible for an investigator to assign subjects randomly to treatment groups or to indiscriminately apply a particular treatment to one group while withholding it from another.

Experimental Designs


Moreover, it could be argued that withholding a treatment from a target population or administering an alternative treatment with unknown effect rather than one with established benefits is unfair or perhaps illegal.
This is often the case in many clinical studies in which an insufficient number of appropriate subjects may preclude the use of randomization procedures.

Quasi-Experimental Designs



Quasi-experimental research designs are generally selected when true experimentation is impractical or impossible to perform.
Typically, subjects are assigned to groups on the basis of preexisting conditions or circumstances.
Suppose you work in a hospital clinic where you treat many adult patients for hoarseness accompanied by vocal nodules.

Quasi-Experimental Designs


Following the diagnosis, the availability of therapy is on a “first-come, first-served” basis so that many patients are on a waiting list for three months or more.
Although the use of randomization procedures may not be possible, you still wish to draw some conclusions about the efficacy of your treatment program.

Quasi-Experimental Designs



To estimate treatment effectiveness, a constructed control group must be established for comparison with a treated group of subjects.
The two groups would be matched on a number of variables prior to treatment.
Matching is done to control as many extraneous variables as possible so that any positive betweengroup differences can be attributed to your treatment.

Quasi-Experimental Designs


These variables would possibly include factors such as the degree of hoarseness, size of the nodules, duration of illness, occupation, age, sex, alcohol/tobacco consumption, etc.
Quasi-experimental methods often necessitate the use of more control procedures.

Nonexperimental Designs




Nonexperimental designs are those in which the three criteria for true experimentation do not exist.
In such research, there is no attempt to achieve randomization, nor is any purposeful effort made to manipulate the variables under study.
These designs examine naturally occurring phenomena and the researcher sets out to test and describe a concept or construct or the relationships among constructs.
The researcher does not manipulate the independent variable but instead examines it in relation to one or more variables for descriptive or predictive purposes.

Nonexperimental Designs



Any manipulation of variables is done post hoc through statistical analysis.
Because random selection, manipulation, and control are not present in these designs, investigators must use caution when make causal claims from the findings.
In health and human services, the three most common types of nonexperimental research include surveys, passive observation, and ex post facto designs.

Survey




Survey designs are used primarily to measure characteristics of a population and are typically conducted with large samples.
There is relatively minimal expenditure and numerous variables can be measured.
Perhaps the most well-known survey is the U.S. Census which seeks to develop a descriptive picture of the characteristics of the population of the U.S.
Statistical manipulation can permit multiple uses of the data set.

Passive Observation



Passive observation designs are used to examine phenomena as they naturally occur and to discern the relationship between two or more variables.
These designs are often referred to as
“correlational designs.
”
Passive observation can be as simple as examining the relationship between two variables, for example, height and weight.

Passive Observation


They can be as complex as predicting scores on one or more variables from knowledge of scores on other variables.
Like surveys, variables are not manipulated but are measured and then examined for patterns.

Ex Post Facto Studies




Ex post facto designs are a type of passive observation literally performed “after the fact.
”
The phenomena of interest have already occurred and cannot be manipulated in any way.
For example, an ex post facto design could be used to examine differing career patterns of male and female social workers after graduation.
One could look at patterns of gender differences in salary, career opportunity, geographical preference, etc.

Quantitative Methods



Quantitative research methods are rooted in the tradition of physical sciences.
With quantitative research, the researcher explores relationships using numeric data.
Statistical methods are used to analyze the data and results are summarized in impersonal, objective reports.





Qualitative Methods
Qualitative research methods enable researchers to study social and cultural phenomena using textual, rather than, numerical data.
Case study, observation, and ethnography are considered forms of qualitative research.
Results are used to understand and explain social phenomena in interpretive reports that reflect the researchers’ constructions of the data.
As such, results are not usually considered generalizable, but are often transferable in the sense that readers will form their own constructions of what is reported.

Empirical Research Design: Tim e




Time is an important factor in the design and execution of research along both the experimental and naturalistic continua.
There are two basic design classifications by time: retrospective and prospective research.
Retrospective research examines phenomena after the phenomena have occurred.
Prospective research searches for cause and effect relationships or examines change in the present as the event unfolds over time.

Retrospective Research
 Retrospective experimental-type designs include those that use chart extraction and are classified as passive observation or correlational.

A retrospective approach is useful when a reasonable data set, such as medical records, is available, if the researcher has limited time and funding resources, and if random assignment is not appropriate.
 Although there is no manipulation of a phenomenon or random assignment or control, the purpose is to describe the occurrence of phenomenon by

Retrospective Research o
Examining the relationships among variables; and o
Examining possible causative relationships.
 For example, Davidoff et al. (1991) conducted a retrospective study to determine the efficacy of rehabilitation programs to facility recovery after acute stroke.
 They reviewed the medical records of 139 acute stroke patients at admission, discharge, and oneyear follow-up from hospitalization.

Retrospective Research




They examined such information as neurologic status, functional status, and the quantity and reasons of outpatient physical and occupational therapy.
Another retrospective strategy is the life history approach.
This approach to qualitative research uses the interview or informant technique to gather information or reminiscences about personal experiences, feelings, and events.
This retelling and recasting of life events, feelings, and experiences can be used as a basis from which to interpret the sense of self as people age.

Prospective Research




There are two basic types of prospective designs: cross-sectional and longitudinal.
In a cross-sectional study, the researcher examines a phenomenon at one point in time.
There is only one single time frame in which data are collected.
Longitudinal studies, on the other hand, involve data collection over extended periods of time.

Cross-sectional Study



Suppose you wanted to describe attitudes toward aging and how individuals of varying ages perceive the aging process.
You could design a cross-sectional study that sampled individuals in different age groups by conducting a survey or interview at one point in time.
The analysis would involve a comparison among the age groups on the dependent variables, attitudes toward aging.

Longitudinal Studies



Longitudinal studies can be used to examine such phenomena as the long-term effects of health care programs or interventions, the natural course of human development and adaptation, the trajectory of illness, or the sequelae of various diagnoses.
There are three types of longitudinal studies: trend, cohort, and panel studies.
Trend studies involve examining a general population over time to see changes or trends that emerge as a consequence of time.

Longitudinal Studies




Suppose you wanted to see if there were any trends in the way in which different age groups perceived the aging process over time.
You would interview individuals who fit into the age brackets you are interested at one point in time, as in the cross-sectional design.
Five years later, let us say, you would interview other individuals who fit the same age brackets.
In this way, you would be able to compare and contrast attitudes toward aging of persons 41 to 50 years old in 2000 with person of the same age range in 2005.

Longitudinal Studies




Cohort studies involve examining a specific group or one particular generational grouping as they change over time.
In a study on attitudes toward aging, we would examine aging perceptions of one age group at one point in time, such as 2000.
Then, at a later point in time, let us say, 2005, we would sample different individuals but those who are from the same cohort as in the first data collection effort.
The individuals in the sample would be five years older.

Longitudinal Studies




Although we would be sampling from the same cohort of individuals we would not be interviewing the same individuals at both time periods.
Panel studies are another longitudinal design strategy.
Similar to a cohort design, the same set of people are studied over time.
For example, the subjects we interviewed in 2000 who were 41 to 50 years of age would be the same individuals we would attempt to interview five years later in 2005.

Research




When conducting empirical research, the notion of what constitutes a variable is important to understand.
Variables are concepts or constructs that change under different circumstances rather than remain constant.
Variables don’t have to be numerical or quantitative; they can be categorical, like gender.
In the behavioral sciences, examples of variables would include: o stimulus characteristics (e.g., tone, intensity or frequency) o o environmental conditions (background noise level) speech behavior (rate of speech or number of nonfluencies)

Research Variables



 o o o language performance (MLU in a language sample) hearing ability (speech reception threshold) gender (male, female, intergender)
Variables can be categorized as independent or
dependent.
The behavior that the experimenter controls is typically referred to as the independent variable.
Independent variables can be thought of as conditions that cause changes in behavior (B).
In other words, the independent variable is the antecedent (A) in the equation ABC.

Independent vs. Dependent Variables




The behavior that the experimenter measures is referred to as the dependent variable.
Dependent variables can be seen as the behavior that is changed.
In other words, the dependent variable is the consequent (C).
For example, delay auditory feedback (the independent variable) may cause a change in speech rate (the dependent variable).

Independent vs. Dependent Variables



In experimental research, the examiner manipulates the independent variable (while holding other potential independent variables constant) to examine the effect the manipulation of the independent variable has on the dependent variable.
For example, MLU is sometimes used instead of chronological age to classify children into groups that vary in degree of language development.
In this case, MLU is the independent variable.

Independent vs. Dependent Variables


If, however, a researcher is looking at the effect of manipulating the number of communicative partners on children’s MLU, MLU becomes the dependent variable .
The distinction between independent and dependent variables is really a distinction based on the use of variables rather than some inherent property of a variable.

Active vs. Attribute Variables



A variable that can be manipulated is an active variable.
The independent variable is an active variable when it can be manipulated in some way by the experimenter to see what effect it has on a dependent variable.
For example, an experimenter can change the intensity of a tone presented to a listener by manipulating the hearing-level dial on an audiometer.

Active vs. Attribute Variables



A variable that cannot be manipulated by a researcher is an attribute variable.
Subject characteristics, such as age, gender, intelligence, type of speech disorder, degree of hearing loss, or history, have been manipulated by nature, but they cannot be changed by the experimenter.
Some variables may be either active or attribute variables depending upon the circumstances of the research or on how the researcher uses the variable.

Active vs. Attribute Variables o o o
Anxiety is an example of a variable that can be active or attribute.
Anxiety can be an attribute of subjects.
Anxiety can also be manipulated by the experimenter by varying degrees to see what effect the manipulation of anxiety has on some dependent variable.

Continuous and Categorical Variables



A continuous variable is one that may be measured along some continuum or dimension that reflects at least the rank ordering of variable values.
The intensity of a tone, for example, can be measured along a numerical continuum from low to high values of sound pressure level.
Stuttering frequency can vary from zero nonfluencies to a high number of nonfluencies.

Continuous and Categorical Variables




A categorical variable cannot be measured and can only be categorized or named.
Tones could be presented to a listener binaurally or monaurally.
Subjects could be classified as “stutterers” or
“nonstutterers.
”
Continuous and categorical variables are displayed differently graphically.

Continuous and Categorical Variables
 Change in a dependent variable as a function of changes in a continuous variable are commonly displayed in a line graph.

Continuous and Categorical Variables
 Change in a dependent variable as a function of change in a categorical variable is customarily displayed in a bar graph.