Introduction to Data
An Introduction to Data-based Decision Making
What is data-based decision making?
As teachers we have a lot of experience gathering information about our students and using it to
make decisions. For instance, we review grades to assess student progress on academic tasks
and revise our lesson plans to ensure that students are successful at least 70% of the time in
initial practice sessions and 90% or more during independent activities. When we observe
students are more disruptive at certain times of the day, we look closely at the setting to find
clues that explain this misbehaviour. Sometimes we find that a task requiring more concentration
needs to be scheduled in the morning instead of after lunch when students are more easily
distracted. Decisions that are based on data help us to create the ideal learning conditions for our
students. Many of our decisions can be made without collecting data systematically. However, at
times a strategic data-based decision making process may be needed. A formal data-based
decision making process can help us identify important variables related to our teaching faster
and more efficiently than relying on our memory to recall important events. This is especially true
for teachers who often deal with many challenges that require their attention simultaneously.
How can teachers use data-based decision making?
Quick, time-efficient strategies can be used when a teacher wants to collect data on one or more
students. For instance, placing a handful of paper clips in one pocket and transferring one paper
clip to another pocket every time a behaviour is observed is a relatively simple strategy that can
be used during instruction. Another strategy may include analyzing assignment scores after class
by summarizing these data. In other words, the student records the data for you.
In more complicated situations, assistance from an EA or Marlene or myself, or other school
personnel who can spend time observing and recording information may be necessary for data
collection purposes.
Data-based decision making can be used in many different ways with individual students or
classrooms. Questions about student academic performance and both problematic and
appropriate behaviour can be answered by collecting information systematically. Examples of
questions to investigate might include the following:







Is the lesson plan too difficult for most students?
Are students experiencing enough success on independent work tasks?
Does the number of homework assignments completed increase when students receive
reinforcement (extra free time, bonus points, etc.)?
Are the new classroom management strategies related to decreases in student
misbehaviour?
Are there times of day or specific activities when the students are more likely to be ontask?
Is there a difference in academic scores on in-class exams when students complete a
review activity that summarizes major points the day before?
Has the student engaged in more social interactions with her friends and this month since
the peer-mentoring plan started?
In addition to student behaviour, you can collect data on your own behaviour as well. Master
teachers are constantly reviewing their performance and trying new strategies. Self-monitoring
and self-management strategies can help you improve your instructional and classroom
management skills. Data can be collected on your own behaviour by using a small counting
device that you can click every time you engage in a behaviour. For instance, you could keep
-1-
Introduction to Data
track of the frequency of verbal praise given to students. Other adults can come observe your
class and collect observational data while you are busy teaching. Teaching students to collect
data is a valuable learning activity and asking for their feedback about lessons provides valuable
information that can be used to improve your teaching. Types of questions to investigate about
your own work might include the following:










Am I providing reinforcement to students on a ratio of 4 positives for every demand or
correction?
Which lesson did the students prefer and find more interesting?
Am I moving around the room and using proximity to decrease problem behaviour?
Am I using pre-correction on a regular basis in math class?
Am I attending to students when they raise their hand to answer a question?
How often do I praise students for correct answers when they yell out the answer instead
of raising their hand?
Am I providing examples throughout the lecture?
Are my classrooms expectations clear to the students? How many of them can tell me
these class rules?
How predictable is my classroom schedule? How often do I let the students know ahead
of time that we will be changing activities?
How much time do I allow for students to work independently? Is this too much or too
little?
How do you decide what to measure?
The decision to measure something is made when you are concerned about some event (for
example, individual student or class academic progress, student conduct in class) or when you
want to evaluate a new strategy or intervention. The first step is to define the behaviour of
interest. It is always very important to create a clear definition, especially when asking someone
else to assist in collecting data. If the behaviour is defined well, it will include:




A brief description of the behaviour,
Information about what the behaviour looks like (topography), and possibly what it does
not look like (so as to not get confused with similar behaviours),
Details about the frequency, length or duration of the behaviour, and/or
Information about the behaviour’s intensity.
If the concern is about an academic behaviour, it is important to establish a certain criterion that
describes the behaviour of concern. For example, you may be interested in any score that is
below 50% on a student's test. You can then count the number of times a student scores below
50% on the tests (note that this excludes all other scores, such as homework assignments). Good
definitions will be written in such a way that someone who hasn't seen the behaviour will be able
to understand and observe it.
Behaviour
Good definition
Bad definition
On task
During class time, Jack is on task every time
he is looking at the teacher when she is
talking, when he is answering her, or when
he is looking at a paper on his desk.
Jack is on task when
he is working.
On time
Mimi is on time to class if her feet are in the
classroom at the time that the bell rings. If
Mimi is at the door when the bell rings (i.e.
Mimi is on time if
she gets to class by
the time that the bell
-2-
Introduction to Data
her feet are partly out of the doorway) she is
not on time.
rings.
Noncompliant
James is non-compliant every time that the
teacher asks him to begin working on an
assignment and he does not begin looking at
it within five seconds of the request.
James is noncompliant when he
does not do what the
teacher says.
Rudeness
Clark is exhibiting rudeness when a teacher
is talking to him and Charlie rolls his eyeballs
upwards while closing his eyes. This
excludes any time that a teacher is not
directly addressing him.
Charlie is rude to the
teacher.
Once the target behaviour has been defined, you can choose a measurement strategy that
best fits the type of behaviour you are observing.
Which measurement strategy should I use?
Some measurement systems provide an exact measure of a behaviour's occurrence, while others
provide a general estimate or proportion of the behaviour's occurrence. The type of measurement
system chosen depends upon a number of different issues including what the behaviour looks
like (topography), the frequency of the behaviour, and the time and energy the person observing
can dedicate to observing and recording the data. In general, elaborate and complicated
measurement methods for collecting data can yield greater accuracy. Nevertheless, this is only
true when the method is used as intended. We can achieve greater accuracy with a method that
is not as rigorous but is a better fit for the time, energy, and r of those responsible for using it. In
addition if data are not used on a regular basis to make decisions, the quality of the recording
often decreases and the process is viewed as a waste of time.
Each measurement strategy has advantages and disadvantages. Some measurement strategies
are more accurate but are difficult to implement. In some cases, you may choose to use variety of
methods to measure different aspects of the same behaviour depending upon how much time
you have to invest. Follow the guidelines below to identify the best measurement strategy for
different types of behaviours. Please note that the methods have been placed in order of least to
most difficult to implement.
Choose a measurement strategy by asking the following questions:
Permanent Product Recording. Does the behaviour generate a product (for example, a written
assignment, a clean table, or papers on the floor)? If the answer is yes, use Permanent Product
recording.
Event Recording. Can you easily count every time that the behaviour occurs (for example, raising
your hand)? Can you easily identify when the behaviour starts and when it ends? Would this
behaviour be easy to count (or does it occur so frequently that it would become complicated to
track)? If the behaviour is easy to count, use Event Recording.
Momentary Sample recording. Does the behaviour occur so often that it may be difficult to count
each occurrence (for example, blinking), or is it difficult to tell exactly when the behaviour starts or
when it ends (for example, reading)? If the answer is yes, use Momentary Sample recording.
-3-
Introduction to Data
How do I begin collecting data?
If you are collecting data in a team setting, the next step is to identify:



Who will be recording the behaviour
Exactly when and where they are to be recording or observing the behaviour, and
How often observations will occur.
The person observing should be in fairly close proximity to the student they are observing to
make sure all instances of the behaviour can be seen. However, the person observing should
also try to be as discrete as possible while observing the student, so as to not influence the
occurrence of the behaviour. It can be helpful to discuss types of student behaviour or times
when a behaviour will not be counted. For instance, a teacher and an EA may be collecting data
on how often a student places his head on his desk. The teacher may not think that it is a
problem for the student to put his head on his desk during breaks. However, if the EA counts all
occurrences of the student's behaviour while the teacher only records occurrences during
lessons, there may be problems later interpreting the data. The number of times the student's
head was on his desk is in part reflective of the way in which each observer recorded the data.
Deciding how often observations will occur depends upon a number of factors including the time
available to collect data, the concern or strategy being evaluated, and the type of behaviour. A
common mistake is to assume that behaviour must be collected all the time, every day. Creating
a highly intensive data collection schedule is not usually necessary. However, it is crucial for
everyone measuring the behaviour to be clear on when the behaviour is to be measured so that
everyone's numbers are comparable. It is also important to think about how long observations will
be conducted. You will need to observe long enough to identify a clear pattern of behaviour over
time and to feel confident that you have gathered enough information to answer your questions.
How are data summarized to evaluate new strategies and interventions?
The way to summarize data efficiently for decision-making is to create a visual summary using a
graph. A graph is a visual representation of the occurrence of behaviour over time. After
instances of behaviour are recorded on a measurement form, the information is summarized and
then transferred to a graph. Graphing the data you are collecting helps to organize the
information and identify important patterns related to behaviour. The information collected before
any interventions have been implemented is referred to as a baseline. The information collected
during the implementation of interventions is referred to as intervention data. Baseline data are
compared to intervention data to determine whether the intervention you are conducting is
resulting in positive outcomes (for example, increases in academic achievement, increases in
positive social skills, and decreases in problem behaviour).
Developed by: Rachel Freeman, Ph.D., Marie Tieghi-Benet, M.S., University of Kansas
-4-
Introduction to Data
Graphing
Why is it important to use a graph?
Once you have collected data from observation sessions, it is important to organize the
information in such a way that it is easy to interpret. It can be difficult to see patterns by simply
looking at long lists of numbers or reading data collection sheets across different days. Graphs
can provide quick and easy visual summaries that allow teachers to determine patterns of
behaviour, evaluate the results of new teaching strategies, and establish whether or not
interventions are having the desired effects. This information can then be used to provide
students with feedback on their performance.
What type of graph should be used?
There are several different types of graphs that can be used to represent data including line
graphs, bar graphs, pie charts, or scatter plots. The most common type of graph used to evaluate
behavioural data is the line graph. A line graph shows individual data points connected by line,
creating a path. Over time, this path can show a visual pattern that helps you evaluate the overall
directions of a behaviour.
-5-
Introduction to Data
Another common graph used is referred to as a bar graph. A bar graph is often used when
portions of a whole are being represented or when reporting a percentage. The bar graph focuses
on the height of the data rather than the trend in the data, and is most often used when
nonconsecutive data points are being evaluated. This is a particularly useful method when
comparing information across individuals, settings, or situations.
Pie charts may be useful when representing portions of a whole. For instance, it might be helpful
to create a pie chart indicating the amount of time a student spends actively engaged in activities.
Finally, scatter plots are used when a variety of observations or measures have been taken that
are not necessarily collected consecutively. For example, a scatter plot may be used to represent
the scores obtained by a class on a standardized achievement test. In this type of graph, each
-6-
Introduction to Data
data point is independent. However, depicting the data in this fashion may allow one to see the
performance of each person compared to the rest of the group.
Example of a scatter plot showing Mrs. Jones's class grades on a standardized academic
achievement test:
What are the important elements of a line graph?
It is important to know the basic elements of a line graph because it is the most common type of
graph used to evaluate behavioural data.
The Horizontal Axis (X-Axis) and Vertical Axis (Y-Axis)
Data are presented in a graph within a boundary containing a horizontal line and a vertical line
that are referred to as axes. The horizontal axis is called the x-axis, and the vertical axis is
referred to as the y-axis. These two axes meet at the bottom left side of the page. The horizontal
axis represents the passage of time. The vertical axis represents the numerical property of the
behaviour being measured. The numbers on both axes are usually divided into equal intervals.
The scale of the y-axis can be an important variable when interpreting graphs. If the scale is set
too high or too low, the changes in behaviour will look much bigger or smaller in appearance, and
this might be misleading. In most graphs, the x-axis (representing time) is longer than the y-axis,
especially if repeated observations of the behaviour have been made.
Points on a graph
Points are usually plotted on a graph by placing a mark where the lines of the behaviour's value
(y-axis) and that of the behaviour occurrence (x-axis) intersect. Each time an observation is
conducted, a point can be plotted on the graph. Points are often connected to each other by lines.
Condition Lines
Each time there is a change that may have an impact on behaviour, a vertical line is drawn
beginning on the x-axis, passing between the data points represented on the graph. Data points
-7-
Introduction to Data
on either side of the condition line are not connected to each other. A condition change line can
denote the move from baseline to intervention or from one intervention to another. Condition lines
can also be used to denote other changes that may impact the behaviour (e.g., sickness, a
change in classroom, a change in teacher or supervisor). However, if the changes are temporary
(e.g., presence of a substitute teacher, illness, father gone on a trip), arrows rather than condition
lines, may be used to mark the beginning and end of these temporary factors.
Condition Labels
Each condition in a graph must be labeled with a short descriptive phrase or word placed at the
top of the graph above the data. This descriptive phase or word represents a condition (for
instance, the baseline or intervention) that is implemented during the time period represented in
the graph.
How do you use a graph to inspect the data gathered?
A visual analysis of the data in a line graph helps to answer two types of questions:


Are there meaningful changes in the behaviour over time?
To what extent can that change in behaviour be attributed to the teaching strategy or
behavioural intervention that was introduced?
Although there are no formal rules for the visual analysis of graphs, there are certain properties
that are common to all behavioural data. The properties within and across conditions that are
examined visually include variability, level, and trends in the data.
Variablility. Variability is the extent to which a behaviour changes from one data point to the next.
If the behaviour does not show much variability, it may not be necessary to collect as much data
since the behaviour is considered more stable and chances are that the behaviour will remain at
this level is high. On the other hand, if a behaviour shows a lot of variability, additional data
-8-
Introduction to Data
should be collected before making any changes. This will allow one to better determine whether
or not the changes in behaviour are due to the intervention.
Levels of Behaviour. The level of a behaviour is the increase or decrease in a behaviour from the
beginning to the end of a condition--the bigger the level of change, the more powerful the effect of
the intervention. For instance, the greater the magnitude and direction of change that has
occurred from baseline to intervention, the more likely that the intervention is effective.
Sometimes a line representing the average of the data points within a condition is drawn on the
graph to help show the change in level. This means line can be useful when the data are
somewhat variable. In the figure below, the mean level line for the duration of tantrums shows
that there isn't much difference between baseline and treatment, indicating that the treatment
may not be too effective.
Trend. Trend refers to the direction the data points on a graph are heading. A steep slant
upwards shows a strong increasing trend while a slant downward indicates the behaviour is
decreasing. Looking at the steepness and direction of the data points can also helps you make
-9-
Introduction to Data
decisions about the effectiveness of an intervention. Before moving to a new condition, the trend
in each phase is evaluated. It is important to make sure that the trend is stable before moving
from baseline to intervention or from intervention to a new intervention. For example, if the
baseline trend is steadily decreasing or increasing it is considered to be in the process of
changing. If the intervention is begun during an increasing or decreasing trend, it is more difficult
to know whether the change in behaviour is due to the intervention since the behaviour was in the
process of changing prior to the intervention.
Developed by:
Rachel Freeman, Ph.D., Marie Tieghi-Benet, M.S., University of Kansas
- 10 -