A Comparison of Pharmacy Students’ Hospital Pharmacy Experience and Performance on Errors and Omissions Testing Laura Morgan, PharmD, BCPS EDUS 660: Research Methods in Education Dr. Donna Jovanovich July 30, 2010 A Comparison of Pharmacy Students’ Hospital Pharmacy Experience and Performance on Errors and Omissions Testing In developing the skills needed for the practicing pharmacist, pharmacy students are expected to accurately and efficiently identify errors and omissions on prescription orders to verify them for appropriateness and completeness. In PHAR535: Foundations of Pharmacy Practice IV students are tested on identifying errors and omissions in prepared sterile products. These assessments are referred to as E&Os. The majority of pharmacy students who enter VCU School of Pharmacy have no hospital pharmacy practice experience and, therefore, have not been 1 2 Comparison of Pharmacy Students’ Experience and Test Performance exposed to preparing or reviewing sterile products. Students who lack experience working in a pharmacy frequently cite this as a reason for poor performance on skills in the classroom. Research has been conducted to determine if there is a relationship between student characteristics and success in Doctor of Pharmacy programs. These studies have aimed to determine which characteristics are associated with completion of the degree program and achievement of a pharmacist license. Several studies have examined various student characteristics potentially related to academic performance in schools of pharmacy including prior completion of a 4-year degree, pre-pharmacy grade point average (GPA), pre-pharmacy math/science grade point average, pharmacy college admission test scores (PCAT), and age. Evidence suggests the strongest predictors of first-year pharmacy GPA were pre-pharmacy math/science GPA and completion of a 4-year degree (Chisholm, Cobb, & Kotzan, 1995). PCAT scores have consistently demonstrated a relationship with academic performance in pharmacy school. Other research has suggested a strong relationship between overall pre-pharmacy GPA, GPA for required pre-pharmacy courses, and PCAT scores and first-year pharmacy GPA. Scores on PCAT and California Critical Thinking Skills Test (CCTST) were associated with successful performance in practice-based courses and APPEs (Allen & Bond, 2001). This study aims to determine whether there is a relationship between hospital pharmacy work experience and performance on errors and omissions testing, within a single course, PHAR535. The results of this study will be used to inform the course coordinator of the need to adjust teaching methods for those students without hospital pharmacy experience. Review of the Literature 2 3 Comparison of Pharmacy Students’ Experience and Test Performance Academic performance while in pharmacy school is important for completing the program but in order to practice as a pharmacist one must pass the North American Pharmacist Licensure Examination (NAPLEX) test. An evaluation of variables which predict graduate performance on the NAPLEX was conducted in over 400 students at Texas Tech University Health Sciences Center Doctor of Pharmacy program. Researchers found a positive correlation between GPA, PCAT, and CCTST scores. However, the reported coefficient of determination was only 0.211 which means only 21% of the variance in NAPLEX performance is explained by these three factors (McCall, McLaughlin, Fike, & Ruiz, 2007). Based on this research, other factors should continue to be considered in the review process of pharmacy student applicants. Pharmacy educator researchers have tried to identify the most important factors which may be related to academic performance of pharmacy students. Many of these studies have focused on success early in the pharmacy curriculum. However, competency as a practitioner is the ultimate goal of pharmacy training. Kidd & Latif (2003) conducted a blinded retrospective record review of three class years of Doctor of Pharmacy students evaluating both traditional and novel traits of students to determine if they were related to classroom and clerkship success. Their findings demonstrated a relationship between overall pharmacy GPA and PCAT score, essay, CCTST, and California Critical Thinking Disposition Inventory (CCTDI). The CCTST was the only variable which showed a significant relationship to students’ clinical performance on APPEs. Selection bias was evaluated by comparing the students group for which complete records were available to those with incomplete records. No statistical significance was found. Limitations to this study include the lack of consistency in the assessment tool used to evaluate clinical performance and generalizability to other pharmacy schools. 3 4 Comparison of Pharmacy Students’ Experience and Test Performance Predictors of academic success are not necessarily predictors of failure. Houglam, Aparasu, & Delfinis (2005) conducted a study to determine which admissions criteria were significant predictors of success or failure in a professional pharmacy program. Their findings showed that ACT composite score, average grade in organic chemistry courses and gender were associated with failure as defined by academic probation. Predictors of academic success represented by first-year GPA also included prior attainment of a bachelor’s degree and grades in math and science pre-pharmacy courses. Data included in the study represented 92.5% of the student population. However, the regression model that was used explained <50% of the variation in the dependent measure of academic success. The authors suggest including other variables such as communication skills, motivation, and critical thinking skills in future research of this area. There is one published study which relates students’ pharmacy-related work experience to performance in the professional pharmacy program. Mar et al. (2010) describe a study of students’ previous pharmacy-related work experience and pharmacy school performance. A survey of pharmacy-related work experience was conducted prior to pharmacy school enrollment. Student focus groups were conducted to review the survey instrument prior to dissemination. Measures of academic and clinical performance included pharmacy grade point average, scores on cumulative high-stakes examinations, and grades in advanced pharmacy practice experiences (APPEs). No significant difference in academic or clinical performance was found between those students with prior pharmacy experience and those without. A total of 206 surveys were complete and able to be used for analysis. This represents 60.1% of the total eligible population. Academic measures of the respondents were compared to those of nonrespondents and no statistically significant difference was found indicating no sampling bias. 4 5 Comparison of Pharmacy Students’ Experience and Test Performance There was a limited number of students (36.9%) who had completed APPEs which makes it difficult to assess the impact on clinical performance. Less than 10% of the total sample did not have work experience which may have made detecting differences between the groups more difficult. Survey response rate was good but did not meet the 70% which is generally considered adequate (McMillan, 2008). Generalizability is limited considering only one school of pharmacy was included. Specific research question Students who have worked in hospital pharmacy practice and been exposed to sterile preparation orders are more likely to perform better when tested on these types of formulations in the classroom. This study will compare errors and omissions test performance of students with hospital pharmacy experience to those without hospital pharmacy experience and examine the following hypothesis: Hypothesis: Students with hospital pharmacy experience will have higher scores on sterile preparation errors and omissions tests than students with no hospital pharmacy experience. Methodology Design Using a non-experimental quantitative research design, this study will describe students’ hospital pharmacy experience and compare errors and omissions test performance with students’ hospital pharmacy experience. A comparative design will be used to examine the independent variable, student hospital pharmacy work experience, with the dependent variable of grades on 5 6 Comparison of Pharmacy Students’ Experience and Test Performance errors and omissions testing. A convenience sample of all pharmacy students enrolled in PHAR535: Foundations of Pharmacy Practice IV in Spring 2010 will be included in the study. There are 120 students enrolled in the course. The population is predominately female (68%) and Caucasian (56%) with an average age of 26 years. Over 90% of students have completed a prior 4-year degree. Instrument The data collection instrument was developed by the course coordinator and consists of two questions each with five response options. Students used an audience response system to respond to items indicating the type and duration of any hospital pharmacy work experience. There was no incentive for students to complete the survey. The instrument has not been piloted which may affect its validity and reliability. The questions may have been ambiguous or poorly phrased. Also, the information is self-reported which may lead to lack of credibility of the responses. Students were not aware that the data was being collected for research purposes. The risk of embarrassment in front of classmates was reduced by the anonymous method of the survey. However, the individual responses were known to the course coordinator which could potentially influence the students’ responses. Procedures During the first class meeting of the Spring 2010 semester, data was collected on students’ prior hospital pharmacy work experience. Demographic data and frequencies of responses to questions will be reported. During the PHAR535 course, students were administered 6 7 Comparison of Pharmacy Students’ Experience and Test Performance 2 sterile preparation errors and omissions tests and grades were recorded. Test grades for students with and without hospital pharmacy experience will be compared. Data Analysis Plan Descriptive statistics on the characteristics of the study population and their responses will be reported. A t test will be used to determine if there is a difference in test performance between students with hospital pharmacy experience and those without hospital pharmacy experience. Limitations The major limitation of this study is the survey instrument. It is not a validated instrument and may have inherent bias in the wording of questions. Extraneous variables include gaps in students’ hospital pharmacy experience and unknown variability in the competency of their pharmacy technician performance. The generalizability of the results of this study are limited considering only one group of students in one course in one school of pharmacy is being evaluated. However, considering this is action research, the intention of the study is to use the results to determine whether a different level of instruction is needed for those students without hospital pharmacy experience. References Allen, D. D., Bond, C. A. (2001). Prepharmacy indicators of success in pharmacy school: grade point averages, pharmacy college admission test, communication abilities, and critical thinking skills. Pharmacotherapy, 21, 842-849. Chisholm, M. A., Cobb III, H. H., & Kotzan, J. A. (1995). Siginificant factors for predicting academic success of first-year pharmacy students. American Journal of Pharmaceutical Education, 59, 364-370. Houglum, J. E., Aparasu, R. R., & Delfinis, T. M. (2005). Predictors of academic success and failure in a pharmacy professional program. American Journal of Pharmaceutical Education, 69 (3), Article 43. 7 8 Comparison of Pharmacy Students’ Experience and Test Performance Kidd, R. S., Latif, D. A. (2003). Traditional and novel predictors of classroom and clerkship success of pharmacy students. American Journal of Pharmaceutical Education, 67 (4), Article 109. McCall, K. L., MacLaughlin, E. J., Fike, D. S., Ruiz, B. (2007). Preadmission predictors of PharmD graduates’ performance on the NAPLEX. American Journal of Pharmaceutical Education, 71 (1), Article 05. McMillan, J. H. (2008). Educational research: fundamentals for the consumer. (5th ed.). Boston, MA: Pearson. 8