Objectives
1. To demonstrate the use of naturalistic observation methods
2. To compare different ways of operationally defining observed behavior.
Type of observational Research Method of collecting data
Naturalistic observation: A method of scientific inquiry in which the researcher unobtrusively observes the subject in the natural environment.
Running Record: A written description of behavior and events as they occur.
Systematic Observation: The quantifiable observation of a few, carefully defined behaviors in a particular setting. This approach is less global than naturalistic observation.
Event Recording: Recording a specific behavior each time it occurs during an interval of time.
Issues to consider in writing observational definitions
•
Inter-rater (or inter-observer) reliability: A reliability index indicating the degree of agreement between two observers coding the same behavior. It is often expressed as the percent agreement between two coders. A general rule of thumb is that 80% agreement or better is needed for a reliable coding system. An observational coding system must be reliable (so that different coders are all recording the information in the same way) in order for it to be valid (measuring what you think it measures). If the coding system does not have acceptable inter-rater reliability, it cannot be valid because at least one of the coders is not coding what it intended.
•
Sensitivity: In addition to being reliable, a measure must be able to capture differences between individual children and situations. If the behavioral codes don't pick up the differences in behavior effectively, then the measure is not likely to be a valid reflection of the variable you are interested in. It is fine for a measure not to reflect differences when they aren't there, but it needs to pick up any differences that are there.
Background information
Observation has been one of the most frequently used methods of studying children's behavior.
The advantage of a naturalistic observation is that a lot of information can be obtained and, since it does not constrain the behavior of the children, the results are generalizable to other similar situations. A more systematic observation approach may provide a narrower range of information but because the information is quantified it is easier to use for comparisons. In this laboratory you will be examining the behavior of preschool children enrolled in a preschool program at Bates Hall or attending the Kindercare daycare center. These are natural preschool/daycare settings so your results are generalizable to other similar settings. During the time that you will be observing the preschoolers they may be involved in structured, adult directed activities so the behaviors you see will not be generalizable to less structured settings.
This exercise will illustrate some of the advantages and disadvantages of two observational techniques. You will be doing a naturalistic observation using the running record technique and a more systematic observation using event recording.
IMPORTANT: The class will go to Bates Hall as a group. The observation room at Bates Hall is to remain quiet so as not to disturb the children or other researchers. Refrain from making comments about the children as parents or other relatives may also be observing. Keep all your observations confidential and protect the anonymity of the children. No results may be reported outside the course without written permission from the OSU Institutional Review Board for the
Protection of Human Subjects.

Procedure
You will be working in groups of 4, which will be divided into subgroups (pairs) to do the two types of behavioral coding. Each subgroup will watch a different child using each technique. Each person will record their own set of data.
(1) Running record
Each subgroup member will write down everything one child does during a five-minute period.
Record information about the situation(s) the child is reacting to. In your record try to include as much specific detail as possible.
Good record
• the child frowned
• his/her face got red
• s/he stamped his/her foot and then yelled
Bad record
• the child got mad
• the child was happy
• the child heard the adult
The good description details the behaviors of the child this allows it to convey the anger and indicate how you reached your conclusions about anger. Often different people reach different conclusions about the meaning of a behavior. By recording the behavior itself you have a record that is more reliable. The second description just states the conclusion without any reasons. All that is recorded here is your interpretation of the behavior you observed. Different observers may reach different interpretations making the measure unreliable.
Remember each subgroup is will be observing a different child, with all members
INDEPENDENTLY keeping their own records for comparison.
(2) Event Recording
Identify a set of behaviors that cover most of the reactions that children might display in a particular situation. For example, you might look for these types of behaviors for the different types of activities.
Adult directed group activity
•
•
"on task" walking away from group
• eating or drinking
• looking away
• fidgeting
Reactions to adult instructions
• being restrained by an adult
• comply
• agree but not comply
• ignore the adult
• verbally refuse
• physically aggress against the adult
Interactions with other children
• sharing materials
• talking together while
• putting hand in mouth
• looks at other children
• working independently
• cooperation on a joint project
• hitting another child
• yelling at another child
Each group needs to prepare definitions for a set of behaviors you want to code. Then prepare a coding sheet with your behaviors listed along one side and room to make tally marks as you count the occurrences of your target behaviors. Remember each person will maintain their own records to compare with your other teammates. You will count the number of times each of the behaviors occurs during a ten to fifteen minute interval.

BEFORE THE OBSERVATION
1. What are the operational definitions of the behaviors you want to observe for the event recording?
AFTER THE OBSERVATION
2a.
Take your group's 4 running record sheets and exchange them with another group.
Compare two running records made on the same child. Rate their similarity (poor, good, very good, excellent) and explain your answer. What are the differences in amount of information and in content of the information conveyed by the 2 records? What kind of reliability are you examining in this comparison?
2b. Compare the running records made for the 2 different children. Can you say anything about how the 2 children are reacting differently? What principle of measurement are you examining in this comparison?

3a. Take your group's 4 event recording sheets and exchange them with another group (it can be with the same group who you traded the running records). Compare 2 sets of event recording data from the same child. Rate the amount of agreement there is (poor, good, very good, excellent) and explain your answer. What proportion of the categories have complete agreement (# of categories with same number of events recorded/total number of categories)? What kind of reliability is this?
3b. Compare the event recording data from the two different children. What similarities and differences are there between the two children?
4.
What caused the differences between data recorded from the same child? (Consider each method of observation)
5. In general, why is it easier for you to see differences between different observers' records of the same child when looking at another group's data than it would be with your own groups data?
6.
Give an advantage and a disadvantage for each of the two coding systems.

Running Record (5 minutes):

Event Recording (10-15 minutes):
Behavior Frequency