Auckland Mathematics Association 2011
Experiments at Level 7 workshop material
Anna Martin
Avondale College
mar@avcol.school.nz
Auckland Mathematics Association – Experiments at Level 7 workshop material
Workshop summary
This workshop will look at the new Level 2 standard on experiments, and discuss ideas for teaching
activities and assessment. Different types of basic experiments will be explained, and how these link to
the problem, plan, data, analysis and conclusion phases of the statistical enquiry cycle. A teaching unit for
this topic will be shared, as well as some advice about the placement/timing of the standard within a Year 12
course.
Synthesising available information
New Zealand Curriculum
(Level 7)
Teaching and learning guides
Carry out investigations of phenomena, using the statistical enquiry cycle:
conducting experiments
•
evaluating the choice of measures for variables and data collection methods used
•
using relevant contextual knowledge, exploratory data analysis, and statistical
inference.
Conducts experiments to find solutions to problems:
•
Poses investigative questions about an experimental situation
•
Plans experiments:
•
Considers sources of variation, for example, what are the variables to be
collected, how each variable will be measured
•
Evaluates the choice of variables and measures used in the experiment
•
Selects and uses appropriate data collection and recording methods
•
Conducts the experiment and collects data
•
Uses exploratory data analysis to explore features of the data:
•
Uses appropriate statistical plots and tables to explore the data and
communicates relevant detail and overall distributions
•
Uses appropriate measures to communicate features of the data
•
Uses relevant contextual knowledge when communicating findings
•
Makes statistical inferences
Communicates findings in a report which includes:
•
relevant summary statistics, graphs and tables to support the findings of the
survey
•
quantitative and qualitative statements
•
statistical inferences
•
justified conclusions
From the glossary:

Experiment
In its simplest meaning, a process or study that results in the collection of data,
the outcome of which is unknown.
In the statistical literacy thread at level 8, experiment has a more specific
meaning. Here an experiment is a study in which a researcher attempts to
understand the effect that a variable (an explanatory variable) may have on some
phenomenon (the response) by controlling the conditions of the study.
In an experiment the researcher controls the conditions by allocating individuals
to groups and allocating the value of the explanatory variable to be received by
each group. A value of the explanatory variable is called a treatment.
In a well-designed experiment, the allocation of subjects to groups is done using
randomisation. Randomisation attempts to make the characteristics of each
group very similar so that if each group was given the same treatment, the groups
should respond in a similar way, on average.
Experiments usually have a control group, a group that receives no treatment or
receives an existing or established treatment. This allows any differences in the
response, on average, between the control group and the other group(s) to be
visible.
1
Auckland Mathematics Association – Experiments at Level 7 workshop material
When the groups are similar in all ways apart from the treatment received, then
any observed differences in the response (if large enough) among the groups, on
average, is said to be caused by the treatment.
Example
In the 1980s, the Physicians’ Health Study investigated whether a low dose of
aspirin had an effect on the risk of a first heart attack for males. The study
participants, about 22 000 healthy male physicians from the United States, were
randomly allocated to receive aspirin or a placebo. About 11 000 were allocated
to each group.
This is an experiment because the researchers allocated individuals to two groups
and decided that one group would receive a low dose of aspirin and the other
group would receive a placebo. The treatments are aspirin and placebo. The
response was whether the individual had a heart attack during the study period of
about five years.
See: causal-relationship claim, placebo, randomisation

Experimental design principles
Issues that need to be considered when planning an experiment.
The following issues are the most important:
Comparison and control: Most experiments are carried out to see whether a
treatment causes an effect on a phenomenon (response). In order to see the
effect of a treatment, the treatment group needs to be able to be compared fairly
to a group that receives no treatment (control group). If an experiment is
designed to test a new treatment then a control group can be a group that
receives an existing or established treatment.
Randomisation: A randomising method should be used to allocate individuals to
groups to try to ensure that all groups are similar in all characteristics apart from
the treatment received. The larger the group sizes, the better the balancing of the
characteristics, through randomisation, is likely to be.
Variability: A well-designed experiment attempts to minimise unnecessary
variability. The use of random allocation of individuals to groups reduces
variability, as does larger group sizes. Keeping experimental conditions as
constant as possible also restricts variability.
Replication: For some experiments, it may be appropriate to carry out repeated
measurements. Taking repeated measurements of the response variable for each
selected value of the explanatory variable is good experimental practice because
it provides insight into the variability of the response variable.

Exploratory data analysis
The process of identifying patterns and features within a data set by using a wide
range of graphs and summary statistics. Exploratory data analysis usually starts
with graphs and summary statistics of single variables and then extends to pairs
of variables and further combinations of variables.
Exploratory data analysis is an essential part of the statistical enquiry cycle. It is
important at the cleaning data stage because graphs may reveal data that need
checking with regard to quality of the data set.
For data sets about populations, exploratory data analysis will reveal important
features of the population, and for data sets from samples, it will reveal features
of the sample that may suggest features in the population from which the sample
was taken.
For bivariate numerical data, exploratory data analysis will indicate whether it is
appropriate to fit a linear regression model to the data.
For time-series data, exploratory data analysis will indicate whether it is
appropriate to fit an additive model to the time-series data.
2
Auckland Mathematics Association – Experiments at Level 7 workshop material
Draft standard
Students are expected to be familiar with the process of investigating a situation by
experiment which involves:
•
posing a question about a given experimental situation
•
planning the experiment

determining appropriate variables and measures

determining data collection and recording methods
•
conducting the experiment and collecting data
•
selecting appropriate displays and measures
•
discussing displays and measures
•
communicating findings in a conclusion.
Conduct an experiment to investigate a situation using statistical methods requires the
student to show evidence of using each component of the investigation process.
Conduct an experiment to investigate a situation using statistical methods, with justification
involves linking components of the process of investigating a situation by experiment to the
context, explaining relevant considerations in the investigation process and supporting
findings with statements which refer to evidence gained from the experiment.
Exemplar tasks on TKI
Matrix – where does it sit?
Conduct an experiment to investigate a situation using statistical methods, with statistical
insight involves integrating statistical and contextual knowledge throughout the
investigation process, which may involve reflecting about the process, considering other
variables.
The plan for the experiment should be of a simple design. Simple experimental designs will
involve one explanatory variable and one response variable. Possible designs could involve:

Measuring the change in the response variable between two dependent values of
the explanatory variable (paired comparison)

Comparing the response variable across two (or more) independent values of the
explanatory variable (categorical)

Exploring the relationship between the response variable and independent values
of the explanatory variable (numerical)
More detail in the assessment schedules
NCEA Level One standards

Multivariate data (comparison methods)

Bivariate data (basic experiments to get data)

Probability investigation (experiments with chance)
NCEA Level Two standards

Inference (comparison methods)

Statistical literacy (risk/relative risk)
NCEA Level Three standards

Experimental design principles

Comparison methods

Statistical literacy (causal relationships)

Bivariate data
3
Auckland Mathematics Association – Experiments at Level 7 workshop material
Development of assessment of experiments over curriculum levels 6 – 8
At Level 1 (bivariate), the experimental situation, the variables, and the investigative question are given
(relationship). The variables are defined and students have to explain how they will be measured . The
analysis of the experimental data is based on “looking for a relationship” through a scatterplot. In terms of
contextual knowledge, the focus is on the participation in an experiment or the data collection process to
provide context for any discussion.
At Level 2 (experiments), the experimental situation is given with possible variables. Students have to write
their own investigative question which means they have to choose the design for the experiment, define the
variables, and explain how the variables will be measured. Students need to use exploratory data analysis
that allows them to “find what they need” or seek explanations through a variety of displays and summary
statistics. In terms of contextual knowledge, the focus is using relevant contextual knowledge about the
situation to be able to explain decisions made in the design process and to link with what is seen in the data,
and to acknowledge sources of variability.
At Level 3 (experimental design principles), only the experimental situtaion is given. Students have to to write
their own investigative question, which involves choosing their own design for the experiment, choosing and
defining the variables, including how they will be measured. Students have a wider range of analysis
methods which they can use with the data generated from the experiment, and again the type of analysis
used will depend on the design of the experiment. In terms of contextual knowledge, the focus is on students
informing themsleves of the situation so they can choose appropriate variables, including how to measure
them. There is also a greater demand on the plan to specify how variables have been controlled or balanced
as part of the experiment design, which requires informed decisions.
In general:




Questions for experiments will be less defined, as the variables will be defined in the plan
There is a continued focus on dealing with variability and seeking explanations for variability
The scaffolding for the development of the plan decreases as you move up the curriculum levels
The demands for the “accountability” in the plan increases as you move up the curriculum levels
4
Structured learning outcomes
Key concept: Conducting
experiments and analysing
experimental data
Identify the key requirements of an experiment
Identify whether situations are experimental or observational
Identify what is being manipulated in an experimental situation
Collect data from an experiment and record in a table
Construct a given display using the experimental data
Identify key features of the experimental data (centre, spread, unusual)
Calculate summary statistics for experimental data
Explain the steps taken in an experiment to try to control variation from other
sources/factors
Describe the type of experiment being conducted
Select an appropriate display for data generated from an experiment by
considering the nature of the data
Link the investigative question and type of experiment with the appropriate
analysis to perform
Use exploratory data analysis to interrogate the data generated by the
experiment
Students need to know that an experiment must enable them to manipulate something and then be able to record the
Resources:
results of this. This is different from sampling or observational data. Use the power point on “Experiment or not experiment”
to get them discussing what situations are experiments, and what is the factor being manipulated. You can also use this to
Experiment or not experiment power point
talk about how you would carry out the experiment, focusing on the need to try to control other factors that might influence
the results.
Heart rate experiment
Get the class to participate in a range of experiments led by you. Try to focus on “two independent group comparison” type
experiments, “paired differences” type experiments, or simple bivariate experiments (changing values of the explanatory
that are independent and seeing what happens to the response). Make it clear to the class the analysis that goes with each,
and how it is different, as you are looking for different things. See the reference sheet included about types of experiments.
Practice effect experiment
General knowledge quiz experiment
Toy gun/myth busters experiment
For the analysis, we need students to explore the data and look for patterns. They should use the mean to discuss the
“typical” result for the experiment, and use the standard deviation to discuss how “consistent” or “similar” the results were.
They should also be looking closely at the shape of the data and for any unusual values. Exploratory data analysis is looser
than comparison-based analysis, and needs to be encouraged e.g. general knowledge quiz, explore if students tend to
estimate above or below the “anchor” given in the question, not just what they estimated, explore how far away they were
prepared to estimate from the “anchor” For the word recall experiment, did students tend to remember some words more
than others? Construct some bar graphs or work out proportions and see. Did students remember words at the top of the
list more than at the bottom of the list? The experimental data should be explored beyond just making obvious inferences.
Get students to think about how experiments need to have a control to allow for us to see if the changes/variability
observed is because of what we changed (the explanatory variable) or because of some other reason (other related factor).
Discuss other sources of variation that might affect the results. Get the students to research experiments that have been
completed to try to prove that something causes something (drug companies) OR that something exists (e.g. like being
psychic)
Explore the experimental data to discover any effects from the experiment
Discuss the variability in the experimental data and consider other related factors that might affect variation in
the data
Consider how experiments are used to establish causal relationships and find examples of interesting
experiments through research
Word recall experiment
Re-drafting experiment
Taste test experiments (BYO food!)
http://www.phy.ntnu.edu.tw/ntnujava/index.
php?PHPSESSID=n3u00b69q2eai42bcanvnk1
t74&topic=387.0
Students can go to this website and try out different
travelling speeds to see how this changes brake time.
The idea is that they realise only to change one factor if
they want to see what happens.
Auckland Mathematics Association – Experiments at Level 7 workshop material
Key concept: Designing
experiments
1
Identify the explanatory and response variables from a given situation
Link the investigation question with the purpose for the investigation
Write a question that can be investigated using an experiment
Explain why the people/objects used for the experiment are suitable
Identify who and what will be used for the experiment
Explain why the explanatory variable will be changed in a certain way
Describe how the explanatory variable will be changed
Explain why the response variable will be measured in a certain way
Describe how the response variable will be measured
Explain how factors that might affect the response variable have been
Describe how the data will be collected and recorded
controlled in the experiment
Identify factors that might affect the results of the experiment
Justify key steps to be taken to carry out the experiment
List the steps to be taken to carry out the experiment
Focus on what things needs to be considered when planning an experiment. Students should be aware right from the start that they question they write
Resources:
will determine the design for their experiment, which will guide their analysis, so spend some time on matching questions with simple designs with types
of analysis e.g. Does eating chocolate make your heart rate increase? Match this with a “before and after” experiment – measure the heart rate before – Take an experiment
eat the chocolate – measure the heart rate after – calculate the increase (difference in heart rates). Match this with a scatterplot initially to see the
situation and work through
relationship, and then a dot plot of the differnces. [NB there could be a control group who does nothing in between, but we still measure the difference
the planning process with
in ther heart rates – the differences from both groups could then be compared]
students. Some possible
investigative questions…..
Model how to write a question for an experiment, which can be part of the overall purpose for the investigation. The question needs to identify (loosely)
what variables will be investigated and be one that will require an experiment to investigate. Find an interesting context (newspaper articles can be
“Does eating chocolate
good sources) and get students to come up with investigative questions. Discuss what the experiment would look like, and what you would need to look
increase your heart rate?”
for in the data.Predict the results of the experiment.
“Do people who are
Write a plan for the experiment. The plan should:
blindfolded find it harder to
describe the variables and measures chosen and why they have been chosen
balance than people who
explain how to collect the data and record the results (this will include who will be used and details of the sample if appropriate)
are not blindfolded?”
link to relevant knowledge about the situation
describe any related variables and the possible effects of these
“Does the surface you run
describe the experimental method
on affect how fast you can
run?”
When carrying out expreiments, stress the importance of making notes about the experiment (about the data collection and experimental process).
These notes will be useful in discussion and reflection of the process in the report write up. This standard is more about showing an understanding of
“Does the distance you
experimental ideas and process than about making inferences (there’s a whole other standard on this concept).Students need to think about what else
stand from the basketbball
the could investigate that would help them understand the situation better, or consider why their experiment did not give them the results they were
hoop affect theh proportion
expecting.They should be combining ideas of how a well run experiment should give them good data, good in the sense it will alllow them to answer
of shots you can make into
their question knowing the effects they observe can be attributed to the variable they manipulated.
the hoop?”
Predict the results of the experiment when discussing the purpose of the investigation
“Does the amount of time
Consider what other questions could be investigated that would give more insight into the experimental situation
you spend on homework
Discuss the impact of other sources of variation on the experimental data and explain how this can be seen in the data
affect your results in a test?”
Reflect on how well the experiment went and aspects that could be changed so that the question could be investigated better
Auckland Mathematics Association – Experiments at Level 7 workshop material
What makes an experiment an experiment?
Identify which of the below are experiments, and if so, what kind of experiment.
Describe what should you be looking for in the data.
Situation
Question
Before and after exercise pulse rates
What is the relationship between people’s “before exercise”
pulse rates and their “after exercise” pulse rates?
Left and right hand reaction times
What is the relationship between people’s left hand reaction
times and their right hand reaction times?
Height a ball is dropped from and the
height of the first bounce
What is the relationship between the height balls are dropped
from and the height of the first bounce?
Length and maximum circumference of
carrots
What is the relationship between the length and maximum
circumference of carrots (or other fruit or vegetable?)
Cubit length and height
What is the relationship between people’s cubit lengths and their
heights? (Cubit length is from elbow to tip of middle finger.)
Dominant and non-dominant hand writing
speed
What is the relationship between people’s dominant hand writing
speed and their non dominant hand writing speed?
Car age and price
What is the relationship between cars’ ages and their prices?
Jumping distance from left foot and right
foot.
What is the relationship between distances people jump using
their left foot and distances people jump using their right foot?
Survey
Version One
Is the number of assessment standards (for all qualifications not just NCEA) offered by NZQA greater or less
than 5000? Circle your answer
Greater than 5000
Less than 5000
How many assessment standards do you think are offered by NZQA?
__________________________
Version Two
Is the number of assessment standards (for all qualifications not just NCEA) offered by NZQA greater or less
than 15000? Circle your answer
Greater than 15000
Less than 15000
How many assessment standards do you think are offered by NZQA?
__________________________
2
Auckland Mathematics Association – Experiments at Level 7 workshop material
Letter spotting
How many Fs did you count on the first attempt?
__________
How many Fs did you count on the second attempt?
__________
Practice make perfect
Attempt 1 time
__________
Attempt 4 time
__________
Attempt 2 time
__________
Attempt 5 time
__________
Attempt 3 time
__________
Attempt 6 time
__________
Reaction time
http://www.phy.ntnu.edu.tw/ntnujava/index.php?PHPSESSID=n3u00b69q2eai42bcanvnk1t74&topic=387.0
[60 km/hr = 1km/min = 1000m/min ≈ 17 m/s]
Test the distance you would need to be travelling behind a car to avoid crashing into them at different speeds
(keep everything else the same)
3
Auckland Mathematics Association HOD Day – Experiments at Level 7 workshop material
4
Types of experiments
Type
Comparison
Sub-type
Paired comparison (dependence)
Questions
Conditions
What are you
manipulating?
Does doing something improve
something? What’s the
difference/change if I do something?
What’s the effect of doing this?
Same group, before and after,
measuring
change/difference/improvement in
variable [linking two measurements of
one variable from the same
unit/person]
The in between, the change in
conditions e.g. how long you exercise,
the fact that you exercise, drinking
caffeine, watching a scary movie
Exploratory data
analysis
Can visualise link on a scatterplot (if
both variables numerical)
Derive variable for increase/difference
Inference about change/improvement
What not to do?
Separate the two measurements and
compare (breaking the link)
Examples
Heart rate before compared to after
exercise, how much do heart rates
increase by?
Comparison of two (or more)
independent groups
Is this better than this? Does doing
this give better results than doing
this? Does it matter if I do this?
Two different groups, comparing one
variable across two (or more)
independent measures/conditions
[treatments]
What each group gets done to them
 (which can be a control so
nothing)
For numerical response data, can be
comparative dot plots and box plots,
with summary statistics. Can explore
data further at look at variation –
shape, distance. Can also look
further into other effects produced
as part of the experiment.
Compare two groups that are not
indpendent
One group given words to memorise,
one group given pictures (of the
same words) to memorise, who can
remember more?
Relationship
Independence within explanatory
variable
Can I use one thing to explain/predict the
other thing? How are these two things
linked? As one changes, how does the
other change?
Each measurement is indpendent
Able to manipulate the explanatory
variable to see change in response
variable
Able to use a wide range of values for
the explanatory variable
Numerical values of the explanatory
variable
Can visualise association on a scatterplot
Can fit models for the relationship
Dependence within
explanatory variable
As one changes, how does the
other change?
Repeated measurements from
same unit, increasing values of
explanatory, seeing what
happens to the response.
How many times something is
repeated/prolonged
Bar graphs, line graphs,
scatterplots
Assume a linear relationship
How does the height a ball bounces
change when you change the height at
which it is dropped?
How does performance on a
task change after repeated
attempts?