Science &
Scientific Method
Literature
[Wikipedia, 2005]
– Scientific Method.
[Carter, 1996]
– The Scientific Method.
[Zobel, 1997]
– Writing for Computer Science, Chap 8. SpringerVerlag 1997
DAIMI
Henrik Bærbak Christensen
2
Definition
Science is both a process of gaining knowledge,
and the organized body of knowledge gained by
this process.
The scientific process is the systematic
acquisition of new knowledge about a system.
This systematic acquisition is generally the
scientific method, and the system is generally
nature.
Science is also the scientific knowledge that has
been systematically acquired by this scientific
process.
DAIMI
Henrik Bærbak Christensen
3
Example
Science is the “opposite” of faith or belief. Often
observations of the world can lead to false
conclusions, or people generalize too quickly.
People mistake truth for belief.
Classical trick:
Which of the two horizontal lines is the longest?
DAIMI
Henrik Bærbak Christensen
4
Example
For centuries people believed in spontaneous generation. This
was the idea that non-living objects can give rise to living organisms.
It was common “knowledge” that simple organisms like worms,
beetles, frogs, and salamanders could come from dust, mud, etc.,
and food left out, quickly “swarmed” with life.
For example:
Observation: Every year in the spring, the Nile River flooded areas
of Egypt along the river, leaving behind nutrient-rich mud that
enabled the people to grow that year’s crop of food. However, along
with the muddy soil, large numbers of frogs appeared that weren’t
around in drier times.
“Conclusion”: It was perfectly obvious to people back then that
muddy soil gave rise to the frogs.
DAIMI
Henrik Bærbak Christensen
5
New ‘evidence’
Microscope in the 1600s: observation of
bacteria growth in soup seemed to support
spontaneous generation.
But – what exactly goes on?
“Life force” present in the molecules of all
inorganic matter, including air and the oxygen in
it, that could cause spontaneous generation to
occur?
DAIMI
Henrik Bærbak Christensen
6
Scientific question
Question:
– The scientist then raises a question about what (s)he
sees going on. The question raised must have a
“simple,” concrete answer that can be obtained by
performing an experiment.
– For example, “How many students came to school
today?” could be answered by counting the students
present on campus, but “Why did you come to school
today?” couldn’t really be answered by doing an
experiment.
DAIMI
Henrik Bærbak Christensen
7
Example question
Question:
– Is there indeed a “life force” present in air (or oxygen)
that can cause bacteria to develop by spontaneous
generation? Is there a means of allowing air to enter a
container, thus any life force, if such does exist, but
not the bacteria that are present in that air?
DAIMI
Henrik Bærbak Christensen
8
Hypothesis
Hypothesis
This is a tentative answer to the question: a
testable explanation for what was observed. The
scientist tries to explain what caused what was
observed.
DAIMI
Henrik Bærbak Christensen
9
Example
Hypothesis:
• There is no such life force in air, and a container of
sterilized broth will remain sterile, even if exposed to
the air, as long as bacteria cannot enter the flask.
DAIMI
Henrik Bærbak Christensen
10
Aspects of Hypotheses
In a cause and effect relationship, what you
observe is the effect, and hypotheses are
possible causes. A generalization based on
inductive reasoning is not a hypothesis. An
hypothesis is not an observation, rather, a
tentative explanation for the observation.
– For example, I might observe the effect that my throat
is sore. Then I might form hypotheses as to the cause
of that sore throat, including a bacterial infection, a
viral infection, or screaming too much at a ball game.
DAIMI
Henrik Bærbak Christensen
11
Aspects of Hypotheses
Hypotheses reflect past experience with similar
questions (“educated propositions” about cause)
and the work of others. Hypotheses are based
on previous knowledge, facts, and general
principles. Your answer to the question of what
caused the observed effect will be based on
your previous knowledge of what causes similar
effects in similar situations.
– For example, I know that colds are contagious, I don’t
know anyone with a cold, I was at the ball game
yesterday, and I was doing a lot of yelling while I was
there, so I think that caused my sore throat.
DAIMI
Henrik Bærbak Christensen
12
Aspects of Hypotheses
Multiple hypotheses should be proposed
whenever possible. One should think of
alternative causes that could explain the
observation (the correct one may not even be
one that was thought of!)
– For example, maybe somebody sitting near me at the
ball game had a sore throat and passed it on to me.
DAIMI
Henrik Bærbak Christensen
13
Aspects of Hypotheses
Hypotheses should be testable by
experimentation and deductive reasoning.
– For example, throat culture and other tests yielded no
signs of a bacterial or viral infection, I have no fever
or other signs/symptoms, and the doctor says my
vocal cords are “swollen” in a way that would indicate
overuse.
DAIMI
Henrik Bærbak Christensen
14
Aspects of Hypotheses
Hypotheses can be proven wrong/incorrect, but
can never be proven or confirmed with absolute
certainty. It is impossible to test all given
conditions, and someone with more knowledge,
sometime in the future, may find a condition
under which the hypothesis does not hold true.
– Example: Newtonian physics does not hold at
relativist speeds…
DAIMI
Henrik Bærbak Christensen
15
Aspects of hypothesis
That a hypothesis cannot be proven to be
correct only incorrect has some consequences.
First, you should actually try to falsify the
hypothesis and if this fails repeatedly only then
should your confidence in it grow.
It is generally difficult for people to be brutally
critical about their own work!
DAIMI
Henrik Bærbak Christensen
16
Types of reasoning
– Inductive reasoning goes from a set of specific
observations to general conclusions: I observed cells
in x, y, and z organisms, therefore all animals have
cells.
– Deductive reasoning flows from general to specific.
From general premises, a scientist would extrapolate
to specific results: if all organisms have cells and
humans are organisms, then humans should have
cells. This is a prediction about a specific case
based on the general premises.
DAIMI
Henrik Bærbak Christensen
17
Prediction
Prediction:
– Next, the experimenter uses deductive reasoning to test the
hypothesis.
Example:
Prediction:
– If there is no life force, broth in swan-neck flasks should remain
sterile, even if exposed to air, because any bacteria in the air will
settle on the walls of the initial portion of the neck. Broth in flasks
plugged with cotton should remain sterile because the cotton is
able to filter bacteria out of the air.
DAIMI
Henrik Bærbak Christensen
18
Experiment/testing
Testing:
– Then, the scientist performs the experiment to see if
the predicted results are obtained. If the expected
results are obtained, they support (but does not
prove) the hypothesis.
– In science when testing, when doing the experiment,
it must be a controlled experiment. The scientist
must contrast an “experimental group” with a “control
group”. The two groups are treated EXACTLY alike
except for the ONE variable being tested.
DAIMI
Henrik Bærbak Christensen
19
Example
Testing:
– Wide-mouth jars each containing a piece of meat
were subjected to several variations of “openness”
while all other variables were kept the same.
control group — These jars of meat were set out
without lids so the meat would be exposed to
whatever it might be in the butcher shop.
experimental group(s) — One group of jars were
sealed with lids, and another group of jars had gauze
placed over them.
DAIMI
Henrik Bærbak Christensen
20
Theory
A theory is a generalization based on many
observations and experiments; a well-tested,
verified hypothesis that fits existing data and
explains how processes or events are thought to
occur. It is a basis for predicting future events or
discoveries. Theories may be modified as new
information is gained.
– This definition of a theory is in sharp contrast to
colloquial usage, where people say something is “just
a theory,” thereby intending to imply a great deal of
uncertainty.
DAIMI
Henrik Bærbak Christensen
21
Summary
from Wikipedia
Process
The essential elements of the scientific method
are iterations and recursions of the following four
steps:
–
–
–
–
DAIMI
Characterization
Hypothesis (a theoretical, hypothetical explanation)
Prediction (logical deduction from the hypothesis)
Experiment (test of all of the above)
Henrik Bærbak Christensen
23
Characterization
The scientific method depends upon the careful
characterization of the subject of the investigation.
Observation demands careful measurement and the use
of operational definitions of relevant concepts.
– When the terms used are formally defining, they acquire exact
meanings which do not necessarily correspond with their use in
natural language:
– for example, mass and weight are quite distinct concepts. On the
surface of the Earth, the distinction may not be obvious, but in
orbit around Earth, we are weightless and not massless.
– Isaac Newton: "I do not define time, space, place and motion, as
being well known to all"
DAIMI
Henrik Bærbak Christensen
24
Hypothesis development
A hypothesis includes a suggested explanation of the
subject. It will generally provide a causal explanation or
propose some correlation.
Observations have the general form of existential
statements, stating that some particular instance of the
phenomena has some characteristic. Causal
explanations have the general form of universal
statements, stating that every instance of the
phenomena has a particular characteristic. It is not
deductively valid to infer a universal statement from any
series of particular observations.
DAIMI
Henrik Bærbak Christensen
25
Prediction from the hypothesis
A useful hypothesis will enable predictions to be
made, by deductive reasoning, that can be
experimentally assessed. If results contradictory
to the predictions are found, the hypothesis
under test is incorrect or incomplete, requiring
either revision or abandonment. If results
confirming the hypothesis are found, the
hypothesis might be correct, but is always
subject to further test.
DAIMI
Henrik Bærbak Christensen
26
Experiment
Once a prediction is made, an experiment is designed to
test it. The experiment may seek either confirmation or
falsification of the hypothesis. Yet an experiment is not
an absolute requirement. In observation-based fields of
science, like astronomy, actual experiments are never
performed, and control experiments are impossible.
Scientists assume an attitude of openness and
accountability on the part of those conducting an
experiment. Detailed recordkeeping is essential, to aid in
recording and reporting on the experimental results, and
providing evidence of the effectiveness and integrity of
the procedure. They will also assist in reproducing the
experimental results.
DAIMI
Henrik Bærbak Christensen
27
Evaluation and Iteration
The scientific process is iterative.
– Verification
• Science is a social enterprise, and scientific work will become
accepted by the community only if they can be verified.
• Scientific journals use a process of peer review, in which
scientists' papers describing experimental results and their
conclusions are submitted to a panel of fellow scientists for
evaluation
– Reproducibility
• Your experiments should be reproducible so other scientists
can validate your hypothesis and conclusions.
– Reevaluation
• All scientific knowledge is in a state of flux, for at any time
new evidence could be presented that contradicts a long-held
hypothesis.
DAIMI
Henrik Bærbak Christensen
28
Evidence and assumptions
Evidence comes in different forms and quality,
mostly due to underlying assumptions.
An underlying assumption that 'objects heavier
than air fall to the ground when dropped' is not
likely to incite much disagreement. An
underlying assumption like 'aliens abduct
humans' however is an extraordinary claim
which requires solid proof. Most extraordinary
claims also do not survive Occam's razor.
DAIMI
Henrik Bærbak Christensen
29
Scientifc Process case
Debugging as science 
Debugging…
The scientific method may sound heavy, but
often you can use the
observe  hypothesis  prediction 
experiment
– and detailed record keeping
...in fast iteration to analyze precisely
One good example is debugging...
DAIMI
Henrik Bærbak Christensen
31
Case: Haken-Blostein algoritm
Haken-Blostein is a complex algoritm for spacing
sheet music.
Basically it treats the graphical objects like a
system of connected springs with rods.
DAIMI
Henrik Bærbak Christensen
32
Observation
Observation: Each invokation of the algoritm
spaced the sheet music differently. Something
gets accumulated or changed between
invokations?
Hypothesis: The observed behaviour is due to
some kind of accumulating defect in the code.
Some variable(s) change value during
invocation
But - Too vague to be useful...
DAIMI
Henrik Bærbak Christensen
33
Iteration
Parameters:
– external algorithm parameters: a
– spring constants ci, rod sizes Rj
Hypothesis 1:
– The algorithm parameter, a, changes value
Experiment:
– Inspect the value of a before and after invokation
– Easy: there is a dialogue box where it can be
inspected.
Result: Hypothesis false.
DAIMI
Henrik Bærbak Christensen
34
Iteration
Hypothesis 2:
– Spring constants changes
Experiment:
– Inspect the values of a before and after invocation
– Difficult:
• There are a lot of springs and no user interface.
• Inserting debug output into code.
• Manually comparing long lists of double values.
Result: Hypothesis false.
DAIMI
Henrik Bærbak Christensen
35
Iteration
Hypothesis 3:
– Rod sizes changes
Experiment:
– Inspect the values of a before and after invocation
– Even more Difficult:
• There are a hundreds of rods and no user interface.
• Inserting debug output into code.
• Using diff-tool to compare very long lists of double values.
Result: Hypothesis accepted.
– a few rod sizes somehow accumulate width…
DAIMI
Henrik Bærbak Christensen
36
… and computer science ?
Software Engineering ?
Some observations…
Computer science / software engineering has
some twists…
Computer science deals more with artifacts than
nature itself.
Thus the material we make hypotheses about
and experiment with are constructed objects.
– i.e. there exists a relation between hypothesis and the
material in the experiment that we often do not see in
physics or biology…
– gives problems with reproducibility
• who else builds systems like ours?
– scope of hypothesis (our system versus all systems?)
DAIMI
Henrik Bærbak Christensen
38
Some observations…
Computer science/software engineering is a
creative discipline just as much as a descriptive
discipline.
– You do not invent nature in physics or biology
– but software engineers do!
Part of computer science is inventing stuff
– meaning there is room for presenting hypotheses
without experimentation, only convincing
argumentation.
Example: Inheritance in Simula
– started object-orientation
DAIMI
Henrik Bærbak Christensen
39
Some observation...
Another research tradition: Mathematics
– No experiments but proofs.
– [but proofs are subject to errors as well ]
DAIMI
Henrik Bærbak Christensen
40
Diploma Thesis
Science in the small…
Science in the small
From the studieordning:
– Ved udarbejdelsen af hovedopgaven skal den
studerende demonstrere fortrolighed med almindelige
principper for videnskabelig metode og færdighed i at
anvende metoder og teorier til selvstændigt at
afgrænse og behandle problemstillinger inden for
området softwarekonstruktion.
DAIMI
Henrik Bærbak Christensen
42
… Thus
This is the yardstick that we will use to evaluate
your thesis!
– softwarekonstruktion: the focus is on constructed
software objects (not necessarily constructing them!)
– selvstændigt at afgrænse og behandle
problemstillinger: you must define your focus yourself.
– videnskabelig metode, færdighed i at anvende
metoder og teorier : adherence to the process is an
important aspect.
DAIMI
Henrik Bærbak Christensen
43
Corollary
Note an important implication:
– building application X or Y is not interesting in
itself!
The focus is on
– A) the scientific process
– B) the product is knowledge
This is usually in contrast to engineering
degrees where the product is the focus…
DAIMI
Henrik Bærbak Christensen
44
Diploma considerations…
Now – you are not supposed to become world
leading scientists in software engineering :
– we do no expect ground-breaking new hypothesis
– comprehensive, large sample, double-blind
experiments
– large software artifacts being build
DAIMI
Henrik Bærbak Christensen
45
Diploma considerations…
But – we do expect
– Characterization
• that you master the terminology and precision and clarity in
statements and writing
• that you provide clear definitions in new concepts or ideas
you present or introduce
– Assumptions and scope
• that you state assumptions
• that you know what “variables” you control and what remain
fixed
• that you clearly scope your work (what do you not consider
part of the work, and argue how this influence your
conclusions).
DAIMI
Henrik Bærbak Christensen
46
Diploma considerations…
But – we do expect
– Scientific method
• that you either
– try hard to formulate your problem as a hypothesis to be
evaluated (prediction and experiment)
– perform an experiment to falsify/confirm an existing
hypothesis/claim in a ‘novel’ environment
• that software construction techniques are used as part of the
process
DAIMI
Henrik Bærbak Christensen
47
Diploma considerations…
But – we do expect
– Scientific progress
• that you not replicate well known facts in well known
situations
• that you consult the (scientific) literature for inspiration and
relations to your work (including web, products, etc).
– Scientific experiment
• “Detailed recordkeeping is essential, to aid in recording
and reporting on the experimental results”
• that you document both experiments and process in a clear
and unambiguous way so your observations are easy to
understand and reproduce
• record ‘open issues’ as you go along!
DAIMI
Henrik Bærbak Christensen
48
Diploma considerations…
But – we do expect
– Scientific evaluation and iteration
• that you present and discuss your work with your fellow
‘researchers’
• Short papers and presentations at seminars
– include as part of your final report
• Peer reviewing groups
• Consulting senior researchers (course teachers )
DAIMI
Henrik Bærbak Christensen
49
Record keeping
If you do an interview
– you analyse and conclude on the interview
– ... but the interview raw data must be present also
Not only the result but also the steps in
achieving the result is important (you may make
an error that a fellow scientist will correct!)
DAIMI
Henrik Bærbak Christensen
50
’we build a system type’
hypothesis: ”using our tool is better and easier”
– better: ”will achieve result in half the used time”
how to handle this:
– describe the current situation using a couple of
concrete scenarios
• inductive reasoning
– describe same output using the tool
• deductive reasoning confirms hypothesis
DAIMI
Henrik Bærbak Christensen
51
Rationale
Every time you make a decision you potentially
invalidate your analysis (because you may make
the wrong choice!)
Thus every decision you make must be
supplemented with a rationale. Why did you
choose as you did?
DAIMI
Henrik Bærbak Christensen
52
Designing a project
… or finding a problem…
At least two ways to go…
Question
– observer some interesting aspect
• from the work place; open source system; old project…
– turn the scientific machine
• formulate hypothesis; make predictions; experiment
Evaluation
– that is reproduce someone else’s results:
– take existing hypothesis/technique
– make predictions and experiment
DAIMI
Henrik Bærbak Christensen
54
Process of making hypothesis
Hypothesis making is difficult !
Example:
– General idea:
• “I would like to write something about vegetation in Africa”
– Specific topic
• “Trees in Africa”
– Problem
• “Imbalance between felling and re-growth”
– Hypothesis
• “Urban wood consumption is more harmful to tree cover than
rural consumption”.
DAIMI
Henrik Bærbak Christensen
55
Software example
– General idea:
• “I would like to work with frameworks”
– Specific topic
• “Frameworks for classic board games”
– Problem
• “Hot spots required to cover large set of board games”
– Hypothesis
• “It is possible to construct a framework for classic board
games with 90% code reuse that covers backgammon,
chess, and checkers”.
It is only the latter that we can objectively
measure…
DAIMI
Henrik Bærbak Christensen
56
Software example
– General idea:
• “We want to build software to plot weather data from the
school’s weather station”
– Specific topic
• “Plotting software for weather data (time series)”
– Problem
• “Adding hot spots to existing plot framework for weather
data”
– Hypothesis
• “It is possible to extend framework X with relevant hotspots
using patterns A, B and C to provide hot spots for weather
data manipulation”.
Less “objectively measurable” but adds a lot of
precision.
DAIMI
Henrik Bærbak Christensen
57
Advice
Even small questions take a lot of time!
– Better to answer small, specific, questions and add a
few more questions than
– … drown in work on a very broad hypothesis.
Problem X turns out to be
–
–
–
–
–
DAIMI
problem A
problem B
problem C
problem D
...
Henrik Bærbak Christensen
58
Advice
Important to be explicit about assumptions and
limitations:
Problem X consists of subproblems A..D. We will
here delimit us to subproblem C.
DAIMI
Henrik Bærbak Christensen
59
Example
Health care scenario
– [Nurse showing graph on TV using PDA]
Subproblems:
–
–
–
–
locating TV to use
combining ctrl and viewer for PDA and TV
steady-state interaction between PDA and TV
MFC combines ctrl and viewer in concrete code
... but most of these are independent problems
and can be abstracted away...
DAIMI
Henrik Bærbak Christensen
60
Open issues
’Open issues’ is an important product!!!
Scientific process leads to
– results (typically only on a single subproblem)
– open issues
• that is: all the unsolved subproblems that we encountered
during the process of trying to produce the result!
Scientists do not solve all the problems!
The usually identify more problems than they
solve! (We don’t want to become unemployed)
DAIMI
Henrik Bærbak Christensen
61
Example: Open Issues
Health care scenario
– [Nurse showing graph on TV using PDA]
Experiment
– .NET on Laptop and PDA
• Laptop keeps model and viewer; PDA keeps controller
Open issue
– MFC’s architecture merge ctrl and view making
experiment really troublesome
– analyze and propose alternative implementation.
DAIMI
Henrik Bærbak Christensen
62
Evaluation
Another path is to evaluate existing
hypotheses/techniques and report results
– Using Quality Attribute workshops to assess qualities
in an architecture
– Report experience on architectural documentation
using multiple view
– Use architectural prototyping to assess buildability of
new project
DAIMI
Henrik Bærbak Christensen
63
Acknowledgements
Parts of the presentation is work by J. Stein
Carter.
Parts are taken from Wiki pedia
DAIMI
Henrik Bærbak Christensen
64