Dr. Gail P.
Taylor
MBRS-RISE Coordinator
UT San Antonio
08/2006

• CRITICAL THINKING, THE SCIENTIFIC METHOD, AND
PAGE 25* OF GILBERT Dany S. Adams, Department of
Biology, Smith College, Northampton, MA 01063 http://www.sdbonline.org/SDBEduca/dany_adams/critical_thinki ng.html#blurb
• Validity: http://carbon.cudenver.edu/~lsherry/rem/validity.html
• At the Bench, A Laboratory Navigator; updated edition. Kathy
Barker, Cold Spring Harbor Press, 2005

• Observe phenomenon & conceive ideas
• Make predictions/develop a hypothesis
• Devise a test/formulate experiment
• Carry out experiments
• Draw conclusions from results
• Reject or support hypothesis

• Science does not generally deal with facts, but rather with evidence
• Each experiment weakens or strengthens a hypothesis
• All evidence is not equal
• Try to discern cause and effect!

• What ideas have you come up with?
• Why is your idea important?
• Have other people tested this idea before?
– http://www.pubmed.org
• What type of background information is available?
• Define question/Develop hypothesis
(and null hypothesis)

• Causality can be difficult to prove
• Different experimental designs impact differently
• Correlative Evidence (weak evidence)
Found together in time or space
• Loss of Function (stronger evidence)
– Blocked a phenomenon
• Gain of Function (strongest evidence)
– Initiation of event leads to second event (additional function)

• “Show it”
• Correlative evidence (time and space):
– Antibody used to detect:
• Found in particular microorganism when aggregating
(and not when free living)
• Found in area where cells are contacting one another during aggregation
– No causality; nothing beyond inference about function
• Clumping could cause the protein expression
• Clumping and protein expression could be induced by same causative agent
• Could be completely coincidental

• “Block it”
• Loss of Function - What does its loss do to clumping?
– Antibody to protein used to block it from functioning.
– Or knock out gene
– Clumping no longer takes place
• Need controls-
– Clumping specifically and only being inhibited
– cells not dying
– May support “real” clumping agent to function
– Therefore it is necessary for clumping

• “Move it”
• Gain of Function
– In organism that does not normally clump…
• Artificially introduce required protein
• Or artificially turn it on at all times (constituitively express)
– Aggregation now takes place
• Therefore is sufficient to induce clumping

• Often you will see this progression through Biological scientific papers
– What is it?
– Yes, it’s there
– Yes, it’s in the right place
– It’s loss produces this response
– It’s addition produces this response…

• Consider statistical methodologies during planning stages
• Look in prior papers for ideas about statistics.
• Statistical analysis will generally discern that likelihood that a result occurred by chance
• Consult mentor or statistician for confirmation
– Compare 1 treatment and control: t-test
• Decide on p (Probability value) p < 0.05 or 0.01
– Compare many treatment groups: ANOVA
– Many more…

• Variables
– Independent (manipulated)
– Dependent (outcome)
• # of samples (minimum 2, 3 better)
• # repetitions (minimum 2x)

• Cause and Effect- Did the experimental treatment, and only the experimental treatment, cause the effect!
– Controls (Be Careful!!!)
– Prevent additional variables from sneaking into your experiment
– Must control for:
• Selection: Anything that makes treatment and control groups different at beginning (random assignment)
• History : What different things may happen between expt. And control groups between initial treatment and measurement
• Maturation : Natural changes in subjects (aging)
• Instrumentation : All tests/equipment/reagents must stay same throughout experiment
• Testing : “incoming” may “teach” the subject
• Mortality : Subjects may leave or die (contamination)
•
Regression: If initial test scores were high, on average, will naturally move towards mean

• The extent to which the findings of the study can be applied, reproduced, or generalized to another setting or systems.
i.e., techniques to ensure that these groups correspond to general population
• Unrepresentative Sample: Sample members not representative of general population.
• Clear Description of the Treatment or Protocol (replicability)
• Hawthorne Effect: Subjects know that they are being studied and it influences behavior
• Novelty Effect : Particularly in humans…enjoy experiment, then possibly don’t.
• Pretest Sensitization : If the pretest is part of the treatment, it will obviously affect the results or findings.
• History and Treatment Interaction : something else happened which influenced results, for all participants
• Measurement of the Dependent Variable : Treatment and data collection must be the same every time!

• Experimental
– Standards/calibration
– Animal/Cell selection/care
– Positive controls
• See what a positive response looks like and that it can be obtained. (positively expressing cells…)
– Negative controls
• Shows what a zero response looks like
• Treatment controls
– All groups treated identically except for indep.
Variable
– If two treatments combined, show individual
– All time points must be covered
– Multiple samples

• Your mentor wants to look at the time course effects of a possible cancer suppressor on proliferation and mRNA expression in six breast cancer cell lines. Wants to look at 0, 12, 24, 36,
48, 72, 96h

• Mega Experiment
– Ex: 6 types of cells, 7 time points, treated and untreated (2), in triplicate (3).
– 6x7x2x3 = 252 plates
• Plan Strategically and Break it down…
– 1 cell line, treated and untreated, duplicate,
7 time points = 28. Or postpone duplicates.

• Low possibility for confusion
• Reasonable workload
• Reasonable use of resources
• Ability to “assess as progress”
– Easy to interpret
– Can change directions on fly
• Easy to create discrete graphs

• For a Publication…
– Need a Big Picture of what you are pursuing; tell a good story
– Start with correlation
– Get additional information
– Knock it out/Add it back/Overexpress
• Slight modifications, depending on field

• Instructions for carrying out a particular technique
• If followed, will produce desired results
• Best if it’s a proven protocol
– Designing your own is time-consuming
– Obtain from another investigator
• In lab, best
– A book of protocols, from web, from kit
• Will need fine-tuning for your local circumstances
– Methods section from published papers (least reliable)

• Read and do dry run-through
• May find logic gaps
• May find references to “common” procedures you do not know

• Rewrite (keeping same steps, etc) to make more sense to you.
• Add notes about own equipment required

• Buy all required materials
– Radioisotopes
• Make all solutions and buffers
• Reserve machine time if needed

• If it doesn’t work, you can assume it’s you.
• Do again. Not work?
• Can get help from person who provided

• Once protocol is working, modify.
• Make notations of changes
• Rewrite for next run though
• Good if type into computer…can record changes and re-print.

• Record which media, temperature or date-sensitive agents you used
• Record any procedural deviation
– Dropped something
– Delayed
– Calibration questions
• Put in lab notebook in a timely fashion

• Did the Experiment work?
– Examine procedural (markers, cells lived)
– Examine positive control (yes, antibody working)
– Examine negative control (No, did not have everything come up positive)

• What were the results?
– Compared to controls, did you see effect?
– Graph your data
– How big was effect?
– Did effect vary over time?

• What does the experiment mean?
– Do the results make sense?
– Was the result what you expected?
– Can you explain spurious results?
– What additional controls may you need?

• What do other investigators think?
– Talk to lab members
– Discuss results with someone versed in technique
– Run through background papers again
– Repeat results

• Are the results repeatable?
– Do experiment again
– Add any additional controls

1. Check notes.
2. Redo the experiment
3. Focus on individual parts of expt.
1. + and – controls…
4.
Do partial expt. to insure it’s fixed
5.
When you’ve done all…try again several times
6. If external protocol, may want to switch

• Never can reproduce data
• Project has little support from PI
• Direction of project has changed
• Not technically possible to do experiments well
• Project too difficult or involved