Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 Queensborough Community College/CUNY Department of Social Sciences Individual Course Assessment Report Date Submitted: June 18, 2015 Course No./Title: PSYC (Upper Level Courses) The courses assessed were: PSYC 215, Child Development, 1 section PSYC 220, Human Growth and Development, 14 sections PSYC 230, Abnormal Psychology, 4 sections PSYC 240, Social Psychology, 2 sections PSYC 245, Cross-Cultural Psychology, 1 section PSYC 250, Personality, 2 sections PSYC 255, The Psychology of Women, 1 section PSYC 270, The Psychology of Aging, 1 section PSYC 290, States of Consciousness, 1 section Pathways Learning Outcomes (Scientific World) Evaluate evidence and arguments critically or analytically. Participants No. Sections Assessed: 27 sections in total No. Students Assessed: 624 Course Assessment Method Students were given a short reading assignment (see Appendix 1) concerning research into eyewitness testimony which, in recent years, has garnered much interest and empirical investigation. The students were able to take the reading home and carefully consider it. Then, in the next class, students were given a written, 16 multiple-choice assessment (see Appendix 2). The questions were formed around the four categories on the critical reading rubric (see Appendix 4): constructing meaning, contextualizing, using others perspectives and positions, and evaluating evidence and drawing conclusions. There were four questions for each of the four categories. This work represents the second time the assessment was piloted. The first time was during the Spring, 2014 semester (Time 1) - 178 students participated, from 9, PSYC 101 sections. The second time was during the past semester of Spring, 2015 (Time 2) - 624 students participated, from 27, upper level PSYC sections. The procedure and instrument were virtually identical from Time 1 to Time 2. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 Two issues were of interest. • Across the four categories of the critical reading rubric, would the pattern of student performance be similar from Time 1 to Time 2? At Time 1, performance in the categories of constructing meaning and using other perspectives and positions outpaced contextualizing and evaluating evidence and drawing conclusions. • Would student performance in any of the four categories show an increase from Time 1 to Time 2? At Time 1, students in PSYC 101 sections participated but during Time 2, students in upper level PSYC courses participated. Assessment Results: Spring, 2015 (Time 2) Rating Criteria Emerging 1 correct Weak Development 2 correct Strong Development 3 correct Constructing Meaning (Questions 1-4) Contextualizing (Questions 5-8) X X Questions correct (percent) 77% 55% Using Other Perspectives and Positions (Questions 9-12) Evaluating Evidence and Drawing Conclusions (Questions 13-16) Mastery 4 correct X X 74% 67% Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 • As at Time 1, performance at Time 2 in the categories of constructing meaning and using other perspectives and positions outpaced contextualizing and evaluating evidence and drawing conclusions. • As can be seen in the Table above, students showed strong development in the category of ‘constructing meaning’. On average, 77% of the items were correct for a mean of 3.07 items. Performance was also strong in ‘using other perspectives and positions’; on average, 74% of the items were correct for a mean of 2.96 items. • Student performance showed weaker performance over the remaining two categories. In the category of ‘evaluating evidence and drawing conclusions’, 67% of the questions were answered correctly for a mean of 2.68 items. Last, only 55% of the questions were answered correctly in the category of ‘contextualizing’ for a mean of 2.21 items. Assessment Results: Performance during Spring, 2014 (Time 1) and Spring, 2015 (Time 2). Spring, 2014 (Time 1) Spring, 2015 (Time 2) Questions correct (M) Questions correct (%) Constructing Meaning (Questions 1-4) 2.88 72% Contextualizing (Questions 5-8) 1.99 51% 2.21 55% Using Other Perspectives and Positions (Questions 9-12) 2.78 70% 2.96 74% Evaluating Evidence and Drawing Conclusions (Questions 13-16) 2.51 64% 2.68 67% • Questions correct (M) 3.07 Questions correct (%) 77% As can be seen from the Table above, the pattern of results across the four categories of questions was similar from Spring, 2014, and Spring, 2015. As mentioned, students Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 showed strong development in the category of ‘constructing meaning’ and in ‘using other perspectives and positions’. Performance was weaker in the category of ‘evaluating evidence and drawing conclusions’ and weakest in ‘contextualizing’. • Compared to Time 1, performance was better during Time 2. The number of questions answered correctly in each category averaged about .2 points higher during Time 2 than Time 1 and was greater by 3-5 percentage points. Conclusions and Action Plan • The results from both time periods point to similar findings. Comparing across the four categories of critical reading, students showed stronger development in ‘constructing meaning’ (derive accurate meaning from texts) and ‘using other perspectives and positions’ (explore perspectives and assumptions contained within the text). Students had greater difficulty in ‘contextualizing’ (makes connections in a limited fashion between written language and contexts, such as prior experience or physical setting) and ‘evaluating evidence and drawing conclusions’ (draws conclusions that are logically tied to information). The emergence of the same trend, across the two time points may indicate an important characterization of critical reading ability in students. In order to examine if the findings are an artifact of the instrument, the results found on assessments carried out by other sub-disciplines within the Department of Social Sciences will be undertaken. Such an approach is beneficial because the assessment instruments varied across sub-disciplines but they were formed around the same critical reading rubric. • It is interesting to note that there was a modest increase in performance from the first administration of the assessment to the second one. It may be tempting to conclude that this reflects an increase in skill, but any conclusion should be drawn very cautiously. While the administration of the assessment appears to follow a longitudinal design, the same participants were not followed with the much larger sample during the second administration highlighting this fact. Ideally, the same participants would have been followed with much more methodological and statistical controls imposed. • While students did not have the reading at the time of taking the multiple-choice assessment, the issue has been carefully considered. By not allowing students to have the article during the assessment, it was thought that careful reading and study of the material would be fostered. Having the reading available at the time of the assessment, could allow students to procrastinate, attempt to read and evaluate the article in class, while attempting to answer the questions. This was not an ideal scenario because faculty members were asked to allot only 20 minutes for students to complete the questions on the assessment. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 • The Assessment Committee of the Department of Social Sciences will be carefully considering the instrument and findings and decide how to proceed. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 Appendix 1: Assigned Reading The Psychology and Power of False Confessions (Abbr., Rev.) Observer – Association for Psychological Science By Ian Herbert On July 8, 1997, Bill Bosko returned to his home in Norfolk, Virginia, after a week at sea to find his wife murdered in their bedroom. A few hours later, Bosko’s neighbor, Danial Williams was asked to answer questions at the police station. And after eight hours there, Williams confessed to the rape and murder of Michelle Moore-Bosko. Five months later, because of inconsistent physical evidence, the Norfolk police became convinced that Williams did not act alone and turned their attention to Joseph Dick, Williams’ roommate. Dick confessed as well. He later pled guilty, testified against two other co-defendants, named five more accomplices who were never tried, and publicly apologized to the victim’s family. “I know I shouldn’t have done it,” Dick said just before the judge gave him a double life sentence. “I have got no idea what went through my mind that night — and my soul.” Dick now says that all of that is untrue, and he has a team of lawyers who believe him. In 2005, the Innocence Project filed a petition on behalf of Williams, Dick, and the other two members of the group called the “Norfolk Four.” They petitioned Virginia Governor Tim Kaine for clemency on the basis of new physical evidence, and in August 2009, the outgoing governor issued conditional pardons, which set the men free but forced them to be on parole for the next 20 years. It was a decision that Kaine struggled with, and he granted conditional pardons because he said the men failed to fully prove their innocence. “They’re asking for a whole series of confessions … to all be discarded,” Kaine said on a radio show in the fall of 2008. “That is a huge request.” We know that false confessions do happen on a fairly regular basis. Because of advances in DNA evidence, the Innocence Project has been able to exonerate more than 200 people who had been wrongly convicted, 49 of whom had confessed to the crime we now know they didn’t commit. In a survey of 1,000 college students, four percent of those who had been interrogated by police said they gave a false confession. But Why? False confessions seem so illogical, especially for someone like Joseph Dick of the Norfolk Four, who got a double life sentence after confessing. Why do people confess to crimes they didn’t commit? Some do it for the chance at fame (more than 200 people confessed to kidnapping Charles Lindbergh’s baby), but many more do it for reasons that are far more puzzling to the average person. In the November 2004 issue of Psychological Science in the Public Interest, APS Fellow Saul Kassin looked at the body of research and described how the police are able to interrogate suspects until they confess to a crime they didn’t commit. Generally, it starts because people give up their Miranda rights. In fact, Richard A. Leo found that a majority of people give up the right to remain silent and the right to an attorney. In fact, according to self-report data, innocent suspects gave up their rights more often than guilty suspects (most told Leo either that this was because they felt that they didn’t have anything to hide because they were innocent or that they thought it would make them look guilty). Once a suspect starts talking, the police can use a variety of techniques to make the accused feel as though they are better off confessing than continuing to deny (these include promises of leniency and threats of harsher interrogation or sentences). If a suspect feels like a conviction is inevitable not matter what he or she says, confessing may seem like a good idea. But, in some cases, the accused comes to believe that he or she actually did commit the crime. It’s been shown repeatedly that memory is quite malleable and unreliable. Elizabeth Loftus has repeatedly shown that the human brain can create memories out of thin air with some prompting. In a famous series of experiments, Loftus, APS Past President, was able to help people create memories for events that never happened in their lives simply through Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 prompting. She helped them “remember” being lost in a shopping mall when they were children, and the longer the experiment went on, the more details they “remembered.” The longer police interrogate a suspect, emphatic about his guilt and peppering their interrogation with details of the crime, the more likely a suspect is to become convinced himself. Joseph Dick claims that this is what happened to him. His confession, testimony, and apology to the family were not lies, he maintains, but rather the product of a false memory. “It didn’t cross my mind that I was lying,” he said. “I believed what I was saying was true.” ‘Corrupting the Other Evidence’ Despite the evidence that false confessions are a regular occurrence, most jurors struggle with the concept just like Kaine did with the Norfolk Four. Confessions are difficult to discount, even if they appear to be coerced. Years ago, Kassin noticed that cases with confessions have an unusually high conviction rate, and since then he has dedicated his life to studying why that happens and what can be done about it. In a 1997 study, Kassin and colleague Katherine Neumann gave subjects case files with weak circumstantial evidence plus either a confession, an eyewitness account, a character witness, or no other evidence. Across the board, prospective jurors were more likely to vote guilty if a confession was included in the trial, even when they were told that the defendant was incoherent at the time of the confession and immediately recanted what he said. Kassin and Neumann also did two simultaneous studies to further explore the power of confessions. In one, they had people watch a trial and turn a dial to rate the extent to which evidence convinced them the defendant was guilty or innocent. The other asked potential jurors after the trial which evidence was most powerful. In both the mid-trial and post-trial ratings, jurors saw the confession as the most incriminating. Other studies have shown that conviction rates rise even when jurors see confessions as coerced and even when they say that the confession played no role in their judgment. “I don’t honestly think juries stand a chance in cases involving confessions,” Kassin says. “They’re bound to convict.” Kassin says he doesn’t blame jurors. He travels around the country lecturing on the psychology of false confessions and he says “the most common reaction I get from a lay audience is, ‘Well, I would never do that. I would never confess to something I didn’t do.’ And people apply that logic in the jury room. It’s just that basic belief that false confessions don’t occur.” What’s more, the evidence juries are given in conjunction with the false confessions is very damning, Kassin says. False confessions of guilt often include vivid details of how a crime was committed — and why. Confessions sometimes even come with an apology to the family. It’s no wonder jurors have trouble discounting them. What confessions rarely include is an explanation of why the person confessed. In most states, police are not required to videotape the interrogations, just the confessions. So juries don’t get to see any potential police coercion and they don’t get to see the police planting those vivid details in the minds of the suspects. And that may be just the tip of the iceberg. Kassin believes that confessions can have a dramatic impact on trials even if they never make it into a courtroom. They can influence potential eyewitnesses, for example, and taint other kinds of evidence. Kassin recently teamed up with psychologist Lisa Hasel to test the effect of confessions on eyewitnesses. They brought subjects in for what was supposed to be a study about persuasion techniques. The experimenter briefly left the room and, during that time, someone came in and stole a laptop off the desk. The subjects were then shown a lineup of six suspects, none of whom was the actual criminal, and they were asked to pick out which member of the lineup, if any, committed the crime. Two days later, the witnesses were brought back for more questioning. Those who identified a suspect were told that the person they identified (or someone else) had confessed, or all suspects denied their involvement. Those who had (correctly) said none of the people in the lineup committed the crime were told either that someone had confessed or all suspects denied their involvement. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 The results show that confessions can have a powerful effect on other evidence. Of the people who identified a subject from the lineup, 60 percent changed their decision when told that someone else had confessed. Also, 44 percent of the people who said no one in the lineup committed the crime changed their mind when told that someone had confessed. When asked about their decision, “about half of the people seemed to say, ‘Well, the investigator told me there was a confession, so that must be true.’ So they were just believing the investigator,” Hasel said. “But the other half really seemed to be changing their memory. So that memory can never really be regained once it’s been tainted.” Moreover, when people who identified someone in the lineup were told that someone had confessed, their confidence levels rose and they were more certain of the details of the crime even though no one in the lineup committed the crime. The implications for inside the courtroom are obvious. Whereas physical evidence is immutable (once collected, it can always be retested), an eyewitness’s decision cannot be revisited without contamination, Kassin and Hasel suggest. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 Appendix 2: Multiple-Choice Assessment Based on Assigned Reading 1. The main purpose of the article was to: a. Expose the unfairness of the judicial system b. Describe the Innocence Project c. Explain why jurors make wrong convictions d. Provide explanations of why some people may make false confessions and describe the impact of confessions to the judicial process 2. In one research study, Dr. Elizabeth Loftus helped people to create memories that never existed in order to: a. Learn about their childhood experiences. b. To show how police interrogation might impact confessions c. To show how memories are real and cannot be constructed d. To display the importance of Miranda Rights 3. Which of the following statements can be described as one of the author’s unstated assumptions? a. All of the accused are innocent b. False confessions rarely happen c. Confessions may be considered as part of evidence but jurors and eye witnesses should be told about how the confessions were obtained and, educated about the possible problems with confessions. d. People should not be convicted based on eye witness testimony alone 4. According to the article, which of the following is a true statement about false confessions? a. If a suspect confesses to a crime then it must be true. b. False confessions are considered strong evidence in the judicial system. c. False confessions rarely ever make it to the court room. d. A confession cannot be considered false if the person believes their own confession to be true. 5. The article opened with the case of Joseph Dick of the Norfolk Four in order to illustrate: a. The pressure by the police to get a confession b. The start of the legal process c. How false memories are related to confessions d. Why people give up their Miranda rights 6. This reading would most likely be found in what kind of text: a. Forensic Psychology b. Sociology c. Memory d. Behavioral Psychology Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 7. Based on the information in the article, it can be inferred that the Innocence Project a. Would support using repressed memories in court cases b. Would recommend that DNA evidence be given higher value in considering guilt or innocence than eyewitness testimony c. Is focused on releasing people from jail d. Would consider sociocultural issues like race and socioeconomic status when considering eye witness testimony and judicial processes 8. Which piece of evidence, would strengthen the author’s position about false confessions: a. A survey study of 1,000 college students found that 4% or 40 students admitted to giving a false confession. b. The Norfolk Four were given conditional pardons. c. The Innocence Project reports that ninety percent of defendants’ confessions are true. d. An expert forensic witness is able to get defendants to tell the truth. 9. In the article, the author focuses the reading around understanding the behavior of what people in a court case? a. judges b. the accused c. lawyers d. prosecution 10. The work of Dr. Elizabeth Loftus provides some very interesting and surprising features of human memory. Based on her research, what would Dr. Loftus tell you to do to help you remember an event accurately? a. Be sure to talk over the event with several people who experienced the event to get an accurate account. b. Be sure to read stories of people who experienced some similar event either in the newspaper or on the web. c. Carefully write down and record the event before talking to anyone about the event. d. Make no special effort to remember the event, but discuss the case with your family. 11. When a jury determines if an accused person is guilty or not, the jury considers all of the evidence around the alleged crime. What evidence below would the author of the article suggest as the most important to a jury when reaching a guilty verdict or outcome? a. A person testifies in court that was an eyewitness to the crime. b. The accused person has a history of mental illness and memory problems. c. If the alleged crime an accused person committed was a felony such as stealing, rape, or murder. d. If the accused person confessed to the crime. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 12. While the article focuses on the law and courtroom testimony, it also reinforces an idea about memory. What idea about human memory is found throughout the reading? a. Human memory is very accurate when it comes to remembering events. b. Encoding, storage, and retrieval are fundamental components of memory. c. Human memory is not always accurate and leads people to remember things incorrectly. d. Memory is worse for the accused because they make more mistakes remembering information. 13. In the article, a number of research findings are reported. What is the main goal of the research? a. To understand the way in which human memory functions. b. To better understand how juries interpret information presented by the lawyers. c. To gain knowledge regarding how judges interpret information. d. To learn how a confession can impact the outcome of a trial. 14. The author of the article presents information, results from studies, and interpretation to explore the case of the ‘Norfolk Four’. Based on the reading, does the author lead you to conclude that the accused in the case were: a. not guilty b. guilty c. unable to stand trial d. under the legal age and not able to testify 15. What is an important feature of the research summarized in the article? a. All of the studies took at least one year to complete. b. Most studies used the experimental method. c. Most studies used the correlational method. d. Many of the participants were not actual jurors or lawyers. 16. The research of Kassin and Hasel is discussed in the article. In their work, they have participants witness a laptop being stolen and then ask the participants to identify a suspect from a lineup. While none of the people in the lineup actually committed the crime, some witnesses identify a ‘suspect’ whereas others correctly do not. Then, they attempt to change opinions of the witnesses telling them that someone confessed to the crime. What does the study show about human memory? a. Once a person makes a decision about the guilt of a suspect, a confession cannot change his/her opinion. b. A person who confesses to a crime should always be believed. c. A confession can change the opinion of a person and contaminate the memory of an eyewitness. d. Research tells us very little about how memory operates in a courtroom. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 Appendix 3: Instructions to Faculty Department of Social Sciences Queensborough Community College/CUNY Medical Arts Bldg., Room M-104 718-631-6251 Memorandum To: Social Sciences (Psychology) Dept. Faculty From: J. Jankowski Date: April, 2015 Re: Course Assessment, Psychology, Upper Level Courses This semester, we will once again be piloting an assessment instrument from May 4-15. The creation of the instrument was prompted by the assessment initiative of the college and guided by the Assessment Committee of the department. To this end, the department committee has settled on examining critical reading across the social sciences disciplines. As you may know, in Spring 2014, we piloted the assessment with some PSYC 101 sections. In this semester, we will pilot the instrument with all upper level sections (not PSYC 101). While the goal is once again to test the usefulness of the assessment, we are also focused on comparing the performance from PSYC 101 sections to upper level ones, looking for any improvement in performance. The assessment should not take more than 20 minutes for students to complete. The assessment results from all upper level sections will be aggregated, and a general report on the results will be drafted. Please note that this course assessment WILL NOT be used as a component of your professional evaluation; only student evaluations of faculty and peer teaching observations are used for that purpose. General Procedure: Enclosed you will find, copies of a short article, assessment forms, scantron forms, an answer key, and a blank item analysis sheet(s). Please note that all of the materials, with the exception of the answer key, are separated according to section, if you are teaching more than one upper level course. During one class (Day1), we ask faculty to distribute the enclosed article to students who will be asked to read the article for the next class. Then, during the following class (Day 2), students will be given the enclosed multiple choice assessment. Faculty members are asked not to discuss or summarize the article at any point. Please give points for answering questions correctly (either as a stand-alone assignment, in conjunction with a test, or extra credit). ++++++++++++++++++++++++++++++++++++++++++ Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 Day 1. Distribute the enclosed two page article to students. Then, read statement below to students. The Department of Social Sciences is conducting a course assessment project for some Psychology classes this semester. Take the article home that is being distributed; read and study the article. In the next class, you will take a multiple choice assessment of the article. (Be sure to tell students the possible points that the assignment is worth.) ++++++++++++++++++++++++++++++++++++++++++ Day 2. Distribute the multiple choice assessment and scantrons to students. Then, read the statement below to students. As noted in the previous class, the Department of Social Sciences is conducting a course assessment. You were asked to take home an article in the last class and read it. Now, answer question based on the article. There are a total of 16 questions, please use a #2 pencil when completing the scantron. Be sure to put your name on the scantron. When you are finished, return the questions and the scantron to your instructor. Thank you for your participation. DO NOT ALLOW STUDENTS TO USE THE ARTICLE, OR ANY NOTES, WHEN TAKING THE ASSESSMENT. ++++++++++++++++++++++++++++++++++++++++++ Concluding Remarks. Please use the enclosed scantron answer key to score the assessment to record the grades. Be sure to also do an item analysis separately for each upper level class section. Then, after finishing the scoring, please return all materials (i.e. copies of the article, assessment forms, completed scantron forms, answer key, and completed item analysis sheet) to the mailbox in M104 of: J. Jankowski. If you have any questions, please contact: Jeff Jankowski, Ph.D. Phone: x5719 Email: jjankowski@qcc.cuny.edu or Anissa Moody, Ph.D. Phone: x6296 Email: amoody@qcc.cuny.edu Thank you for your participation in this important project. Psychology, PSYC (Upper Level Courses) Assessment, Spring Semester, 2015 Appendix 4: Critical Reading Scoring Rubric Rating Criteria Emerging 0-1 Weak Development 2 Strong Development 3 Mastery 4 Usually derives accurate meaning from texts by making sense of written words and analyzing reading with respect to prior knowledge, research, and experience Usually connects written language with contexts, such as prior experience, historical setting, physical setting, knowledge of the discipline, etc. Usually explores perspectives and assumptions contained within the text Consistently derives accurate meaning from texts by making sense of written words and analyzing reading with respect to prior knowledge, research, and experience Consistently connects written language with contexts, such as prior experience, historical setting, physical setting, knowledge of the discipline, etc. Consistently explores perspectives and assumptions contained within the text Draws conclusions that are logically tied to a range of information (because information is chosen to fit the desired conclusion); related outcomes (consequences and implications) are identified clearly Draws conclusions and related outcomes (consequences and implications) that are logical and reflect student’s informed evaluation and ability to place evidence and perspective discussed in priority order Constructing Meaning Derives meaning from texts in a confused or inaccurate way Derives meaning from texts in a limited fashion; makes sense of written words but no further analysis Contextualizing Reads written language in isolation or connects it to irrelevant or inaccurately understood contexts Using Other Perspectives and Positions Takes text at face value, showing minimal awareness of perspectives and assumptions contained within the text Conclusions are inconsistently tied to some of the information; related outcomes (consequences and implications) are oversimplified Makes connections in a limited fashion between written language and contexts, such as prior experience, historical setting, physical setting, knowledge of the discipline, etc. Attempts to explore perspectives and assumptions contained within the text Evaluating Evidence and Drawing Conclusions Draws conclusions that are logically tied to information (because information is chosen to fit the desired conclusion); some related outcomes (consequences and implications) are identified clearly