8
Best Practices in Problem Analysis
This is a pre-print. I apologize if there are a few errors/typos.
There are corrections in line throughout.
This is now published in Best Practice VI.
Theodore J. Christ
Yvette Anne Arañas
University of Minnesota
OVERVIEW
The purpose of this chapter is to describe the theoretical
foundations and practical significance of problem
analysis. While a problem is defined as an unacceptable
discrepancy between expected and observed performance, problem analysis is the systematic process of
assessment and evaluation to better understand the
nature and possible solution for the problem. Problem
analysis includes the collection, summary, and use of
information to verify or reject relevant hypotheses
related to both the cause and solution of a problem.
Problem analysis contributes to the Data-Based
Decision Making and Accountability domain in the
National Association of School Psychologists (NASP)
Model for Comprehensive and Integrated School Psychological
Services (NASP, 2010), which requires that practitioners
competently use assessments and data to assess and meet
the needs of students.
This chapter describes the theoretical foundation of
problem analysis and describes its relevance at the
individual, group, and systems level. School psychologists who are not experts in a specific content area will
learn how to use a content-specific hypothesis-testing
framework as a guide to formulate intervention
recommendations. This chapter encourages school
psychologists to seek or develop additional contentspecific frameworks to organize hypothesis testing and
make concrete connections to the relevant research.
BASIC CONSIDERATIONS
Problem analysis is best guided by context- and contentspecific expertise. The context is often the school and
classroom setting. The content is often related to
academic or social-emotional skills and performance.
A school psychologist who lacks expertise in either the
context or content might still function as a successful
problem analyst if he or she relies on science and
evidence to make decisions. This section establishes that
problem analysis is scientific, relies on low-level
inferences, and focuses on alterable variables.
Scientific Method
An understanding and appreciation for science is
inherent to problem analysis. Both the scientific method
and scientific body of evidence inform problem analysis.
In brief, the scientific method is the process of inquiry
and discovery. The scientific body of evidence is the
knowledge and interventions that emerge for systematic
research.
The steps for problem analysis are consistent with that
of the scientific method. They are (a) identify a problem;
(b) hypothesize likely causes; (c) select methods for
assessment such as reviews, interviews, observations, and
tests; (d) collect data; (e) review data; and (f) revise
hypotheses regarding likely causes or, if the cause is
isolated, form a hypothesized solution. Table 8.1
presents an application of the scientific method to an
example of a student with a reading problem.
There are two types of hypotheses associated with
problem analysis: analytic and intervention (see
Table 8.2). An analytic hypothesis is developed in the
second step of the scientific method and relates to the
likely causes of the problem. Analytic hypotheses guide
analysis with improved articulation of the purpose for
data collection. That is, data are collected for a reason
and analytic hypotheses define those reasons throughout
problem analysis. An example of an analytic hypothesis
1
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
Best Practices in School Psychology
Table 8.1. The Six Steps of the Scientific Method and Problem Analysis
Scientific Method
Problem Analysis
Example
Observe and identify the
problem of interest
Develop and identify
relevant hypotheses
Identify the problem
Teacher observes that a student is struggling in reading
Hypothesize likely causes and
maintaining conditions
(analytic hypotheses)
Design and select
procedures to test
relevant hypotheses
Select methods for
assessment such as
reviews, interviews,
observations, and tests
Collect data
Analyze and synthesize
data
Collect the data
Review data
Devise a tentative
conclusion
Revise hypotheses regarding
likely causes or form an
hypothesized solution
(intervention hypotheses)
General:
. Poor instructional match
. Low exposure to instruction
. Poor curricular match
. Need for more practice
. Inaccurate critical skills
. Low rate critical skills
. Low motivation/incentive
Specific:
. Inaccurate performance oral reading performance
. Low rate or automaticity on oral reading
General:
. Review educational records
. Interview the student and teacher
. Observe student performance within instructional
conditions and interacting with the curriculum
. Test with oral reading assessments to calculate the
percent of words read correctly (accuracy) and the
number of correct words per minute (rate); use
vocabulary and comprehension measures as necessary
. Test the student response to a targeted intervention
aimed at the maintaining variable (e.g., incentivized
performance, repeated practice, timed practice,
instruction and curricular match)
Review, interview, observe, test
Synthesize the data to evaluate and address possible
causes and maintain conditions so to inform intervention
actions
. Intervention: The student requires targeted practice of
vowel and consonant diagraphs to establish accuracy
and automaticity for work attached and word
identification
. Goal: The student identifies four words correctly in
1 minute (35% accuracy) when presented with a list of
words with vowel and consonant diagraphs; if the
intervention is effective, then the student will gain three
words per minute per week with increased levels of
accuracy until the student reaches 65 words read
correctly in 1 minute (95% accuracy)
is, ‘‘The student cannot read because he or she struggles
to decode words.’’ The second type of hypothesis is an
intervention hypothesis, which is developed during the
final step of the scientific method. This hypothesis
relates to the likely solutions for a problem. An example
of an intervention hypothesis is, ‘‘A phonics interview
will teach the student how to match sounds to letters and
subsequently improve reading performance.’’
Intervention recommendations are defined as hypotheses because problem solutions are tentative until
demonstrated as effective. Intervention hypotheses
emerge from analytic hypotheses. A common error in
2
problem analysis is omitting analytical hypothesis testing
and immediately performing intervention hypothesis
testing. This error frequently leads to failed intervention
attempts, wasted resources, and frustrated staff.
Analytic hypotheses function to formulate and test the
supposed cause or causes of a problem. For example,
there are common analytic hypotheses that should be
tested when there are reading problems in a particular
grade. These analytic hypotheses may address prior
exposure to effective instruction, instructional match,
and the achievement of relevant foundational skills.
When formulating analytic hypotheses for academic
National Association of School Psychologists
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
Best Practices in Problem Analysis
Table 8.2. Example Analytic and Intervention Hypotheses for Reading
Analytic Hypotheses
Intervention Hypotheses
There is a poor instructional match (e.g., pacing, feedback)
that contributes to insufficient growth. For example: the level
of the material is too difficult.
There is a poor curricular match that contributes to insufficient
growth. For example: level of the material is too difficult.
There are inaccurate critical skills (e.g., does not know letter
sounds, diagraphs, or sight words) that contribute to
insufficient growth.
There is a low performance rate on critical skills that
contributes to insufficient growth. For example: accurate
and slow decoding and word identification.
There is a low motivation or lack of incentives that contribute
to insufficient growth. For example: the student can perform
when provided sufficient incentives, which might be more
interesting materials, activities, or tangibles.
problems, it may be helpful for school psychologists
and educators to refer to a set of research- and
evidence-based standards such as the Common Core
State Standards (National Governors Association
Center for Best Practices & Council of Chief State
School Officers, 2010) for reading, mathematics, and
other subjects. Finally, it might is also useful to
reference hierarchies of skill development so to discern
the likely sequence of skill acquisition and instruction.
Such hierarchies can help to conceptualize a sequence
of analytical hypotheses.
Poor instruction might have caused the problem.
Deficit skills that are inconsistent with the task demands
for third-grade reading might maintain the problem.
The process that begins with analytic hypotheses must
progress to intervention hypotheses so that problem
analysis concludes with useful recommendations. Some
examples are presented in Table 8.2, which illustrate
how analytic hypotheses (supposed causes) are associated with intervention hypotheses (possible solutions).
It is more helpful to organize hypothesis statements
into a hypothesis-testing framework, which can effectively guide the process. Figure 8.1 presents an example
of a hypothesis-testing framework for reading, which
incorporates some information about reading interventions from Scammaca, Vaughn, Roberts, Wanzek, and
Torgesen (2007) and unpublished meetings with K.
Bollman, M. C. Coolong-Chaffin, and D. Wagner
(personal communication, March 21, 2013). This
example is discussed in more detail below and is only
intended to serve as an illustration. We strongly
encourage researchers and practitioners to develop,
evaluate, and refine their own hypothesis-testing
If provided instruction at a slower pace with increased
feedback, then the student will demonstrate more rapid
growth.
If provided with curriculum materials that match or more
closely approximate the student’s skills, then the student
will demonstrate more rapid growth.
If provided with demonstration and slow deliberate practice,
then the student will demonstrate more rapid growth
toward performance at 95% or greater accuracy.
If provided with incrementally faster paced practice with
feedback, then the student will demonstrate more rapid
growth toward automatic performance.
If provided with an incentive to improve performance, then
the student will demonstrate more rapid growth toward
work completion and higher levels of performance on
classroom assessments.
frameworks. They have the potential to both refine
the analytic process for professionals and document the
process, which requires substantial expertise in the
content and context of school-based problems.
Levels of Inference
An inference is a tentative conclusion or assumption that
lacks explicit support from available data. Hypotheses
are inferences by definition. It is critical that problem
analysts make distinctions between what is known and
what is inferred, or hypothesized. There are two types of
approaches in making inferences: high inference and
low inference. A high inference approach typically relies
on theoretical, within-person constructs such as personality, psychopathology, or aptitude profiles. Such
constructs are difficult to observe directly and they tend
to rely on the assumption that outward behaviors are
merely symptoms of an internal trait. Evidently, this
approach requires a number of assumptions whose
veracity is often unknown.
A low inference approach relies on direct observation
of explicit behaviors and testable hypotheses. In contrast
to the high inference approach, this requires relatively
few untested assumptions. Examples of low inference
hypotheses may include a student’s strengths and
weaknesses as measured by a standardized test,
intervention recommendations based on multisourced
data, and the relevance of a published assessment to a
school. These hypotheses are preferred over high
inference hypotheses because they are based on testable
assumptions. Thus, low-level inferences should take
precedence and should be exhausted prior to the use of
Data-Based and Collaborative Decision Making, Ch. 8
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
3
Best Practices in School Psychology
Figure 8.1. An Example of a Problem Analysis Hypothesis Framework for Reading
high-level inferences. Moreover, the most relevant
analytic hypotheses and approaches will emphasize
those variables that can be altered to have an impact
4
in solving a problem. Low-level inferences and alterable
hypotheses are most closely linked with successful
problem solutions.
National Association of School Psychologists
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
Best Practices in Problem Analysis
Alterable Causal and Maintaining Variables
Problem analysis aims to identify variables that both
cause and maintain problems. Causal variables are usually
historical, such as insufficient nutrition, disadvantaged
experiences, or lack of prior exposure to effective
instruction. Maintaining variables persist in the present
context to influence and sustain the problem. Although
there are both biological and ecological variables that
interact to cause and maintain problems, the goal of
analysis is to promote problem solutions rather than
ruminate about unalterable conditions. Problem analysis
aims to examine causal and maintaining variables that
can be manipulated by an intervention team. Variables
such as instruction, classroom rules, and seating
arrangements are considered to be alterable variables. In
contrast, inalterable variables are irrelevant unless they
interact with the alterable variables. For example,
physical impairments such as blindness or deafness are
inalterable. However, such impairments might help
explain why previous instruction was ineffective and
consequently guide future strategies, thus interacting
with alterable variables.
Alterable variables are usually in the ecology (e.g.,
class instruction and curriculum) and not within the
learner. The learner’s environment is composed of a
large number of alterable variables that influence
performance, such as seating arrangements and classroom rules. A problem analyst can hypothesize that the
arrangement of the ecology will promote desirable
outcomes and can monitor student’s progress to test this
hypothesis.
Characteristics of Novice and Expert Analysts
In school psychology, an effective and efficient problem
analyst is expected to have context-specific expertise in
instruction, curriculum, and ecological variables that
interact with learner variables. A problem analyst needs
content-specific expertise in reading, writing, mathematics, classroom behavior, and social development.
Unfortunately, many school psychologists develop
substantial expertise in test interpretation with modest
knowledge of the context and content of education. For
that reason, many school psychologists are novices at
problem analysis because they lack domain specific
expertise (i.e., context and content knowledge).
Those with area-specific knowledge and experience
are likely to employ a top-down approach to problem
analysis. Experts use their content- and context-specific
knowledge and experiences for problem analysis. They
are more likely than novices to sample information
about a problem strategically and search for familiar
patterns, principles, and concepts associated with
possible problem solutions. Familiarity with area-specific
patterns and procedures enables experts to identify the
most relevant variables, ignore trivial details, and
generally combine complex patterns into meaningful
chunks of information. For example, when a student is
disruptive in the classroom, an expert behavior analyst is
more likely to focus on the causal mechanisms and
functions of the problem behavior. The expert may find
that the behaviors function to access peer attention,
access teacher attention, or escape task demands. When
addressing a problem, the expert is likely to extract the
most relevant information to analyze and represent a
problem using a goal-directed, top-down approach.
In contrast, a novice is likely to rely on a bottom-up
approach for problem analysis because he or she has
limited knowledge and experience. The novice often
focuses on the extraneous or surface-level variables
associated with a problem. It is more difficult for the
novice to strategically identify and attend to the most
relevant information. A novice behavior analyst might
focus on the topography of a target behavior or
irrelevant antecedents rather than on the function of
the behavior. Despite these difficulties, novice school
psychologists can solve problems effectively without
having content- and context-specific knowledge. They
can use systematic hypothesis-testing frameworks to
solve a given problem. In the example of the disruptive
student, a novice problem analyst can develop a
functionally based set of analytic hypotheses to guide
problem analysis. These hypotheses also can yield
intervention hypotheses. For example, if problem
analysis supported the hypothesis that disruption functioned to escape from task demands (e.g., difficult math
work), then the likely intervention hypothesis might
include a replacement behavior for escape (e.g., escape
cards) with modified task demands (e.g., ensure
instructional match).
BEST PRACTICES IN PROBLEM ANALYSIS
The prior section established that problem analysis by
school psychologists incorporates the scientific method,
low levels of inference, alterable variables, and expert
knowledge of both the context and content. In practice,
it takes time for school psychologists to develop these
skills and an appreciation of their value. Therefore, best
practices in problem analysis is facilitated by selected
implementation of problem analysis with a multitiered
Data-Based and Collaborative Decision Making, Ch. 8
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
5
Best Practices in School Psychology
system of support. It is also facilitated by selective use of
systematic hypothesis testing frameworks, which make
the process explicit and enhances the potential for widescale implementation.
Problem Solving and Multitiered Systems of
Support
A school psychologist must identify the problem, classify
it, and analyze it at the systems, group, or individual
level before selecting or implementing an intervention.
That is, it is necessary to have information about a
problem to understand why it occurs. It is a common
error to select an evidenced-based or a well-marketed
intervention. However, this well-marketed intervention
might be improperly aligned with a local problem.
Evidence-based interventions work for particular populations with specified deficits. No intervention works
generically to solve all problems. It is problem analysis
that helps align problem solutions with the problem at
hand.
Problem Analysis for the System, Group, and
Individual
A multitiered system of support is highly inefficient if all
problems are analyzed at the individual student level. It
is much more efficient to evaluate the prevalence of a
particular problem to align the scope of the intervention
with the scope of the problem. Therefore, problem
solving and problem analysis occur at the systems level
so that core curriculum, instruction, and support are
assessed, evaluated, and refined. It is very common for
school psychologists to observe reading performance,
math performance, or behavior as a prevalent problem
that is best addressed with systemic changes. Those
systemic changes are guided by careful analysis.
Regardless of the quality of systemic supports, some
groups and individuals will require supplemental and
intensive supports. Most problems in education are not
individual student problems. Rather, they are typically
more pervasive. Because problem analysis on the
individual student level is the most inefficient use of
resources, it is worthwhile to consider systemic and
group analysis first and reserve individualized problem
analysis for uncommon cases. The value of efficiency is
consistent with the goals of working smarter, not harder;
doing less, but doing it better; and doing it once, but for
extended amounts of time. Problem analysis should
enhance decisions about what is done, how it is done, for
whom it is done, and for what reasons it is done.
Ultimately, problem analysis is about an alignment of
6
problem characteristics with problem solutions or
interventions.
Problem Analysis Precedes Standard Protocol
It is common for school psychologists to employ a
standard protocol approach as part of a multitiered
system of supports. This standard protocol provides a
default approach to intervention for common problems.
This approach is relatively easy to implement because it
uses the same intervention or treatment for all students
who have similar problems in a particular area (Fuchs,
Mock, Morgan, & Young, 2003). Standard protocols,
however, do not address individual differences and do
not emphasize problem analysis. To address this gap,
school psychologists can use problem analysis to refine
standard protocols so that they understand the type of
problems and solutions before selecting a standard
intervention program. For example, students with
routine deficits in early reading are likely to benefit
from an evidence-based phonics intervention. Initially,
that intervention must be selected or developed with
problem-specific knowledge that emerges through
problem analysis. After implementation and evaluation,
a high quality solution is readily available in the
education system. That intervention becomes standard
protocol for use in the future. Thereafter, problem
identification and problem analysis function to match
identified problems efficiently with standardized interventions.
Theoretical Orientations Guide Systematic
Hypothesis-Testing Frameworks
A variety of theoretical orientations can support
hypothesis testing and support problem analysis. For
instance, developmental theories provide information about
typical development among children and can inform
what kind of instruction and interventions are needed to
promote learning and growth in students who are
developing differently from these trajectories. Other
theoretical orientations can inform interventions specifically for academic skill deficits. The instructional hierarchy,
for example, provides a framework that postulates that
skills progress in four stages: acquisition, fluency,
generalization, and adaptation (Daly, Lentz, & Boyer,
1996; Haring & Eaton, 1978). An applied behavioral
analysis orientation can be useful to develop hypotheses
about which environmental events predict and maintain
a target behavior, especially when a student has a
performance deficit. It is also useful to guide task
analysis and the construction of skills hierarchies.
National Association of School Psychologists
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
Best Practices in Problem Analysis
Curriculum-based assessment and curriculum-based
evaluation can inform problem analysis. Curriculumbased assessment is particularly relevant to problem
analysis because it is used to assess students within the
context of their curriculum content and learning needs
(Hintze, Christ, & Methe, 2006). Curriculum-based
evaluation provides an intricate hypothesis-testing
framework across both academic and social-skill
domains such as reading, mathematics, written expression, language, social skills, and task-related behavior.
The curriculum-based evaluation hypothesis-testing
framework usually is presented in a series of flowcharts
with corresponding decision-making rules and assumes
that performance discrepancies exist because prior
knowledge or skills have yet to be established. For more
details about curriculum-based evaluation, see Hosp,
Hosp, Howell, and Allison (2014).
Understanding the limitations of applying certain
theories is important to effective problem analysis. For
instance, hypotheses related to general intelligence are
usually limited to diagnosing developmental disabilities,
but the diagnoses are not necessarily useful in selecting
instruction and interventions. The traditional framework of using test-based trait and aptitude profiles relies
on high inference approaches and distal measurements
and thus is not very consistent with effective problem
analysis.
Development and Use of Systematic
Hypothesis-Testing Frameworks
School psychologists are expected to use context- and
content-specific expertise to solve academic and social
behavior problems. Unfortunately, many school psychologists lack training and expertise across the many
context and content domains within educational systems. The breadth and depth of academic, behavioral,
and socioemotional problems that school psychologists
address are substantial. Because it is not always possible
to be independent experts for all the problems that
might arise in schools, practitioners can develop and use
content-specific systematic hypothesis-testing frameworks to address problems. Such frameworks can
provide explicit guidance, particularly when a school
psychologist lacks expertise in a specific content area. In
problem analysis, an effective framework should (a)
derive from evidence-based practices; (b) be embedded
within the scientific method; (c) emphasize low inference
methods of assessment and evaluation; (d) evaluate
alterable causal and maintaining variables; and (e)
integrate expert approaches of analysis, which embed
navigational aids so that the sequence and targets of
evaluation are optimized.
Effective problem analysis seeks to establish knowledge in assessment and evaluation to optimize contentspecific problem analysis. Identifying and developing
explicit hypothesis-testing frameworks will contribute to
establish the school psychologist as a problem analyst.
There are some early examples that might inform future
development, including curriculum-based evaluation
and curriculum-based assessment for instructional
design and applied behavior analysis. Each provides
some systematic approach to assessment and evaluation
to declare relevant analytic hypotheses. When a school
psychologist lacks expertise, hypothesis-testing frameworks provide scaffolds and a system of support to guide
analysis. Likewise, hypothesis-testing frameworks guide
the identification of problem solutions. For example,
deficits in a critical foundational skill area might disable
learning and, thereby, establish and maintain learning
problems.
Examples of Systematic Hypothesis-Testing
Frameworks
There are two examples of systematic hypothesis-testing
frameworks. The first was presented in Figure 8.1 and it
uses oral reading assessments to illustrate a simple and
familiar hypothesis-testing framework. This example
begins with an oral reading assessment, such as
curriculum-based measurement of oral reading (CBMR). This is a useful starting point because many schools
use CBM-R for universal screening, which is useful to
identify reading problems. It is at that point that
problem analysis begins with the development and
consideration of relevant analytic hypotheses related to
causal and maintaining variables. Some examples are
listed in Figure 8.1. Data subsequently are collected
from interviews, observations, tests, and reviews of
extant data and evaluated to inform the use of analytic
hypotheses. The figure illustrates tests of skills-based oral
reading deficits. The bottom left section of that figure is
intended to illustrate how the analytic hypotheses and
intervention hypotheses/recommendations generally
correspond, such that analytics inform interventions.
The bottom right section of that figure illustrates the
way in which ongoing progress monitoring functions to
continually test and update the intervention hypothesis.
Although this particular example lends itself to problem
analysis of individual students or small groups, problem
analysis often will apply to larger groups of students.
Data-Based and Collaborative Decision Making, Ch. 8
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
7
Best Practices in School Psychology
Figure 8.2. Problem Analysis at the System, Group, and Individual Level
Figure 8.2 represents problem analysis for a multitiered system of supports. Neglecting a systems-level or
group-level problem can overwhelm both problem
solving and the system of supports. Problem analysis is
often resource intensive and cannot be sustained if an
excessive number of problems are analyzed at the
individual level. It is when specific groups and learners
have distinct needs relative to the local population that
resources are allocated to analyze those problems. If
data support the first hypothesis in Figure 8.2, then
analysis will progress to test this hypothesis: The
magnitude of the problem and need for services are
greater for some learners. This hypothesis is designed
to provide support for groups and individuals with
substantial needs. The needs of the system are
established by the discrepancy between the performance of the population and some external criterion
such as proficiency on a large-scale assessment in
reading. The need for interventions at the group or
individual level is established by the discrepancy
between a learner’s performance and that of the local
norms for the school, district, or classroom population.
Those students who are discrepant from local
standards are at even greater risk for academic failure.
8
This is consistent with the 80-15-5 model for multilevel systems of support.
Once the scope of the problem within the population
is defined at the system, group, and individual level, then
the school psychologist, or problem analyst, may begin
to develop hypothesis statements. Because poorly
defined expectations are often the root of school-based
behavior problems, the first hypothesis statement a
problem analyst would ask is, ‘‘Do students know what is
expected of them?’’ If the students clearly are aware of
expectations, then the problem analyst can progress to
evaluate instructional match or ecological contingencies.
If expectations were not taught and are not clearly
established, then the problem analyst might recommend
a remediation strategy. Expectations can be taught
through explicit instruction, modeling, practice, or
direct feedback. The implicit hypothesis is: If expectations are taught, then the magnitude of the problem will
be reduced. This is an example of an if/then hypothesis.
An if/then hypothesis establishes a tentative belief that a
particular manipulation in the ecology will result in a
predicted change in learner behavior. The hypothesistesting framework is directly linked to interventions
because the framework terminates with a testable
National Association of School Psychologists
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
Best Practices in Problem Analysis
hypothesis for intervention, the intervention hypothesis.
The intervention itself is a hypothesis because it is a
putative and testable solution to the problem. It is best
practices to recognize and define recommendations for
instruction and intervention as hypothetic statements.
The effect of an intervention is unknown until after data
are collected and evaluated.
Assessment for Problem Analysis
Assessment is a basic competency for school psychologists (NASP, 2010). Appropriate assessment data are
collected using a multimethod, multidomain, and
multisource approach. Figure 8.3 features a matrix that
illustrates this approach. Reviews, interviews, observations, and tests serve as different assessment methods,
while the instruction, curriculum, environment, and the
learner represent the multiple domains (Heartland Area
Education Agency 11, 2006).
It is necessary to note that problem analysis does not
require that school psychologists use an exhaustive set of
assessments across all methods, domains, and sources of
information. Instead, assessment should generate the
necessary information for answering a set of wellspecified questions. The school psychologist should
readily be able to respond to the question, ‘‘How will
the assessment procedure answer the assessment question?’’ For instance, if mathematical calculation skills are
a concern, then a school psychologist may ask the
following two questions: (a) ‘‘Is this a skill deficit, a
generalization problem, or a performance deficit?’’ (b)
‘‘If this is a skill deficit, then which specific skills are
Figure 8.3. Assessment: Multidomain, Multimethod, Multisource Matrix
Data-Based and Collaborative Decision Making, Ch. 8
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:51
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
9
Best Practices in School Psychology
affected?’’ The intensity and thoroughness of assessment
will be determined, in part, by the severity and
characteristics of the problem. For instance, those
influences maintaining minor problems may be relatively easy to identify. Consequently, assessment and
evaluation for problem analysis might be brief and
relatively few cells in the matrix would be completed. In
contrast, the influences maintaining severe problems—
or problems that are resistant to standard protocol
solutions—may be difficult to identify. In those cases,
assessment for problem analysis would be more
extensive and more cells in the matrix would be
completed. More severe problems that establish high
levels of risk or are resistant to intervention require a
more extensive dataset to support problem analysis.
SUMMARY
School psychologists establish a unique and necessary
niche when they engage in effective problem analysis.
The field will thrive and school psychologists will
become invaluable to multitiered systems of supports,
which depend on effective and efficient problem solving
and problem analysis. The dynamic process of problem
analysis is the essential link between assessment and
intervention. In this sense, problem analysts function as
applied scientists in that they seek to discover the
relationship between independent variables (instruction,
curriculum, environment) and dependent variables
(skills and behavior of the learner). The goal is to
modify the conditions of the independent variables to
have an impact on the state of the learner.
Problem analysis progresses from (a) a well-identified
problem to (b) possible causes to (c) possible solutions to
(d) a validated solution. As discussed, successful problem
analysis depends on knowledge and appreciation of
science as both a method and resulting body of
evidence. Problem analysis progresses through steps
similar to the scientific method, which includes problem
identification, hypothesis development, hypothesis testing, and the generation of tentative conclusions in the
form of intervention recommendations. Problem analysis also depends on the body of evidence that emerges
from science. Published research and evidence-based
practices is relied on to inform analytic hypothesis and
intervention hypothesis development.
Problem analysis is most effective when low inference
methods of assessment and interpretation that focus on
the causal and maintaining variables are relied on.
There are many theories and perspectives on student
development. Those that are most relevant to problem
10
analysis are immediately testable through observation of
skills and behavior in the school-based setting. School
psychologists and other educators often have expertise in
one or more content areas, but rarely have broad and
deep expertise in all domains. As discussed, the ability to
identify and organize the most relevant information is
an important aspect to problem analysis. Together, and
for those reasons, we recommend the ongoing development and evaluation of content-specific hypothesistesting frameworks. Although there are a few simple
examples presented in this chapter, researchers and
content experts must develop, evaluate, and refine
frameworks that are content and context specific.
These will assist the nonexpert problem analysts as they
pursue intervention recommendations through problem
analysis.
In practice, problem analysis is fundamental to
multitiered systems of support. It helps to understand
the causal and maintaining features of problems before
solutions are proposed. As discussed, it also helps to
understand the prevalence of problems. This understanding helps to determine whether the intervention
should target the system, core supports, identifiable
groups, or individuals. Multitiered systems of support
are rarely sustainable if each problem is resolved at the
individual student level. Many problems are common to
systems and can be either prevented or resolved more
efficiently at the group level.
REFERENCES
Daly, E. J., Lentz, F. E., & Boyer, J. (1996). The instructional
hierarchy: A conceptual model of understanding the effective
components of reading interventions. School Psychology Quarterly, 11,
369–386.
Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003).
Responsiveness-to-intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities
Research & Practice, 18, 157–171.
Haring, N. G., & Eaton, M. D. (1978). Systematic instructional
procedures: An instructional hierarchy. In N. G. Haring, T. C.
Lovitt, M. D. Eaton, & C. L. Hansen (Eds.), The fourth R: Research in
the classroom. (pp. 23–39). Columbus, OH: Merrill.
Heartland Area Education Agency 11. (2006). Program manual for
special education. Johnston, IA: Author. Retrieved from http://www.
iowaideainfo.org/vimages/shared/vnews/stories/4a8b1534597fd/
Special%20Education%20Procedures%20Manual%20January%
2015%202013%20final.pdf
Hintze, J. M., Christ, T. J., & Methe, S. A. (2006). Curriculum-based
assessment. Psychology in the Schools, 43, 45–56.
Hosp, J., Hosp, M., Howell, K., & Allison, R. (2014). The ABCs of
curriculum-based evaluation: A practical guide to effective decision making.
New York, NY: Guilford Press.
National Association of School Psychologists
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:52
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
Best Practices in Problem Analysis
National Association of School Psychologists. (2010). Model for Comprehensive
and Integrated School Psychological Services. Retrieved from http://www.
nasponline.org/standards/2010standards/2_PracticeModel.pdf
National Governors Association Center for Best Practices & Council
of Chief State School Officers. (2010). Common Core State Standards for
English language arts and literacy in history/social studies, science, and
technical subjects. Washington, DC: Author.
Scammaca, N., Vaughn, S., Roberts, G., Wanzek, J., & Torgesen, J.
K. (2007). Extensive reading interventions in grades K–3: From research to
practice. Portsmouth, NH: RMC Research.
Data-Based and Collaborative Decision Making, Ch. 8
Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:52
The Charlesworth Group, Wakefield +44(0)1924 369598 -
Rev 7.51n/W (Jan 20 2003)
11