HOW TO OUTLINE AND EVALUATE RESEARCH Outline: (APPRC) A = Aim of the study P = Participants used in the study P = Procedures of the study R = Results of the study C = Conclusions of the study Evaluate: (REVISED) R = Reliability E = Ethics V = Validity I = Issues and debates S = Social implications E = Economic implications D = Different research The APPRC mnemonic is pretty straightforward. However, what does each term in the REVISED mnemonic mean? Reliability: Was the conducted in a controlled environment? Yes = high controls and standardised procedures enable easy replications to check the reliability of the results. No = low controls prevent easy replications. It is therefore difficult to check the reliability of the results. Have replications of the research produced consistent results? Yes = consistent results suggest that the results are reliable. No = inconsistent results suggest that the results are not reliable and therefore cannot be trusted. Reliability refers to whether the research produces consistent results. In order to be considered valid (accurate), results MUST be reliable (consistent). However, results can be reliable despite not being valid. It is therefore important to ensure both reliability and validity are demonstrated. Ethics: Informed consent: Were participants aware of what they would be required to do? Investigators should give participants sufficient details of the investigation so they can make an informed choice as to whether to participate. Deception: Were the participants deceived about any aspect of the study? Investigator(s) should not deliberately mislead/lie to the participants. Confidentiality: Were the participants’ personal details kept confidential? Investigators should keep personal details confidential unless agreed in advance. Debriefing: Were the participants adequately debriefed following the completion of the study? Investigators should explain all relevant details of a study to participants and ensure they leave the study in no worse state than when they started it. A debrief is especially important if participants have been deceived. Right to Were the participants given a right to withdraw from the study at any time without withdraw: incurring negative consequences? Investigators should make participants aware that they can leave a study at any time, and can even withdraw their data after the study has finished. Protection of Were the participants protected from physical and mental harm? participants: Investigators should protect participants from physical and mental harm during the investigation. Risk of harm must be no greater than in ordinary life. You should be able to discuss some of your strengths and limitations. With reference to ethical issues this could be: Informed consent: Could the researchers have predicted the outcome of the procedure in order to gain fully informed consent? This may not be possible in observational research. E.g. Zimbardo’s study lacked informed consent as participants were not made aware of the negative behavioural consequences of a set-up prison setting. However, Zimbardo could not have predicted these outcomes, and as such, it would have been highly difficult to gain fully informed consent. Deception: Was deception necessary? E.g. Asch deceived participants into thinking they were taking part in a line judgement task with other participants. Unknown to the naïve participant, these other ‘participants’ were in fact confederates who were giving obviously incorrect answers to investigate whether the naïve participant would conform. However, if participants had known that the others were confederates who were giving wrong answers deliberately, the study would have lacked internal validity. This is because they would have shown demand characteristics – conformed to the answers as it is what the researcher wanted. Protection of participants: could the researchers have predicted the physical and/or mental harm? How serious was the harm caused, and was it long-term? Did the practical applications of the study outweigh any short-term distress? E.g. Milgram exposed his participants to severe stress, which is supported by the extreme physical reactions that many participants exhibited (uncontrollable seizures, sweating, digging their nails into their flesh). However, Milgram conducted follow-up questionnaires that found that the majority of participants thought they had learned something and were glad to have taken part. A thorough debriefing was also carried out where participants met the unharmed learner and, a year later, all participants received psychiatric assessments with none showing any signs of long-term damage. The study can therefore be justified in terms of cost-benefit analysis, where the shortterm distress is outweighed by the practical applications of the study i.e. on understanding and preventing blind obedience. Validity: Validity concerns accuracy. It refers to how accurately a study investigates what it claims to (internal validity) and the extent to which findings can be generalised beyond the research settings (external validity. Validity Internal validity Concerns whether the results are due to the manipulation of the independent variable/IV (the variable the study manipulates). External validity Refers to the extent to which the results can be generalised to other settings (ecological validity), other people (population validity) and over time (temporal validity). Internal validity: Was the study conducted in a controlled environment? Yes = high controls limit extraneous variables from affecting the results. However, the risk of demand characteristics is increased. No = there is a reduced risk of demand characteristics as participants are unaware they are being studied. However, the risk of extraneous variables affecting the results are increased due to low control. Were the participants volunteers? Volunteers are typically more keen to participate, resulting in possible attempts to guess the research aims. If true, the findings would lack internal validity due to demand characteristics. External validity: 1. Population validity – consider the number, gender, culture and age of participants. Are they representative of the whole population? Can the results be generalised? 2. Ecological validity – consider the environment in which the study was conducted. Is it natural or artificial? What effect is this likely to have on participants’ behaviour? Can the results be generalised to everyday behaviour? 3. Temporal validity – consider when the study was conducted. Have there been societal changes since the research which may result in different findings if replicated today? Issues and debates: There are many issues and debates within psychology. Here are some examples: Gender: Androcentrism Does the research consider men’s behaviour as the standard against which women’s behaviour is compared? Alpha bias Does the research exaggerate the differences between the genders? Beta bias Does the research attempt to downplay the differences between the genders? Culture: Ethnocentrism Does the research assume that one ethnic group is superior to another or all others and that the behaviour in that group is the ‘norm’? Cultural Does the research consider the behaviour of the individual within their culture before relativism making a judgement. Social norms are culturally relative as what is considered acceptable in one culture may be unacceptable elsewhere. Social implications: Does the research have wider social implications, good or bad? Economic implications: Does the research enable practical applications that could be used for the betterment of the economy? Different research: You should be referring to supporting and opposing research in your evaluation where possible. PUTTING IT INTO PRACTICE: Using Milgram’s research as an example, let’s put APPRC and REVISED into practice: Milgram aimed to measure whether individuals would obey the orders of an authority figure that incurred negative consequences and went against one’ moral code. His sample consisted of American males who had responded to a newspaper advertisement to volunteer for a study of memory and learning at Yale University. Participants (who became teachers via a fixed lottery) were told by an experimenter wearing a grey lab coat that the study concerned the effects of punishment on learning. As a result, they were required to administer increasingly severe electric shocks from 15 (labelled ‘slight shock’) to 450 volts (labelled ‘XXX’) to a learner (confederate participant) every time they answered a memory question incorrectly. Although the learner was placed in another room out of sight, their verbal answers and complaints could still be heard. Aim Participants Procedure At 150 volts, the learner began to protest and demanded to be released. At 300 volts he refused to answer any more questions and said he had heart problems that were starting to bother him. Any time the teacher seemed reluctant to continue, the experimenter gave a series of verbal prods such as “the experiment requires you continue” and “you have no choice, you must go on”. 100% of participants obeyed up until 300 volts. 62.5% of participants fully obeyed the experimenter by administering electric shocks to the maximum of 450 volts. During this time, many participants showed distress, such as sweating, digging their nails into their flesh and some having uncontrollable seizures. Milgram therefore concluded that people obey those regarded as authority figures, even if such orders cause us distress and go against our moral code. Results Conclusion =========================================================================== As Milgram’s sample consisted only of American males, it can be argued that the results may not be representative of females and those from other cultures. However, a replication by Milgram found similar findings with female participants, suggesting the results are generalisable to both sexes. In contrast, research has found varying levels of obedience between different cultures. It may be that different cultures obey to different levels because of cultural differences regarding authority. As a result, Milgram’s findings are not representative of obedience levels across cultures. It has also been suggested that the high rate of obedience found in Milgram’s study was a product of American culture being very obedient during the early 1960s and, as such, doesn’t reflect obedience levels today. However, Burger (2009) used an adaptation of Milgram’s procedure, finding that obedience rates have not changed dramatically in the 50 years since Milgram’s study. As a result, Milgram’s research is considered to have temporal validity. Milgram’s research was conducted in a controlled laboratory environment, enabling standardised procedures. As a result, Milgram’s research can easily be replicated to check for the reliability of his findings. However, controlled laboratory environments are artificial, lacking mundane realism. People may therefore be less obedient to destructive orders in natural settings, where obedience to harmful orders is not considered to be for the betterment of society. If true, Milgram’s results would lack ecological validity as it would fail to reflect behaviour in everyday situations. Population validity Temporal validity + different research Reliability to reflect behaviour in everyday situations. However, Hofling et al. (1966) investigated whether nurses would obey destructive orders from an unknown doctor to administer double the maximum dosage of an unfamiliar drug to a patient. As similar obedience levels were obtained, and the study was conducted in the nurses’ natural setting, it could be argued that Milgram’s results are representative of obedience levels found in everyday situations. Ecological validity + Different research Milgram’s study raised several ethical issues. Firstly, he failed to gain fully informed consent; participants were not told that they would be required to administer electric shocks to another ‘participant’. This has ethical implications in that participant may have found it more difficult to withdraw from the study upon finding out what they were required to do because they had already agreed to take part. Milgram also deceived his participants into thinking they were administering real electric shocks to the learner. This deception caused participants significant distress (as evident by their physical reactions) and also effected the extent to which they had given informed consent. However, if participants had known that the electric shocks were not real, Milgram’s study would have lacked internal validity. This is because they would have figured out the aims of the research and so would have shown demand characteristics – obeying the researcher because that is what they wanted. Moreover, Milgram’s study can be criticised for failing to give participants a clear understanding of their ability to withdraw from his study at any time. For example, he used a series of standardised prompts (e.g. “the experiment requires that you continue”, “you have no choice, you must go on”) when participants expressed their unwillingness to continue. However, Milgram defended this, arguing that the study was measuring obedience and thus these prompts were necessary in order to investigate whether participants would obey. Furthermore, 35% of participants did withdraw from the study, demonstrating that they did, in fact, have a right to withdraw. Another criticism refers to Milgram’s failure to protect his participants from harm; he exposed his participants to severe distress which is supported by the extreme physical reactions many participants exhibited (e.g. uncontrollable seizures and digging their nails into their flesh). However, Milgram conducted follow-up questionnaire that found that the majority of participants thought they had learned something and were glad to have taken part. A thorough debriefing was also carried out where participants met the unharmed learner and, a year later, all participants received psychiatric assessments with none showing any signs of long-term damage. The study may therefore be justified in terms of cost-benefit analysis, where the short-term damage (stress) is outweighed by potential practical applications of the study i.e. on understanding and preventing blind obedience. Despite hoping that his findings would help from strategies to reduce destructive blind obedience, however, not much has changed. Horrendous crimes are still committed by people operating under the excuse of ‘simply following orders’. One such example is the abuse of Iraqi prisoners by US troops in the Abu Ghraib prison in 2004. Consequently, Milgram’s study has little practical applications for the betterment of society. Ethics Social implications This answer is more than enough for what you would be required to write in an exam, even for a 16 marker. However, it is a good example of how you could use APPRC and REVISED in order to write detailed outlines and evaluations of studies. Over to you: Go through the key studies you have learned so far, using APPRC and REVISED to outline and evaluate each piece of research.
