In August 2015 and just before the start of the Advanced Pharmacy Practice Experience (APPE) rotation for the fourth professional year pharmacy students, the hospital in King Abdulaziz Medical City (KAMC) in Riyadh declared the spread of the “Middle East Respiratory Syndrome coronavirus” or MERS-CoV among its staff and patients. The virus was first isolated in September, 2012 from a patient in Saudi Arabia and has a high fatality rate and clinical features that resemble a severe acute respiratory syndrome [1].

This necessitated an elective shutdown of parts of the institution including the emergency department, the suspension of all patient admissions, and an early discharge for stable patients along with a number of preventive measures. All students and faculty were asked to refrain from visiting KAMC or any of its recreational facilities. Similarly, practicing clinical preceptors were prevented from coming in contact with faculty and students in the university campus. All these were temporary measures in an attempt to stop the spread of this disease in the tertiary care hospital and on campus that houses 6 health schools. When hospital faculty had to deliver lectures to students were done via video conferencing.

In lieu of cancelling the first APPE rotation and delaying students graduation, the college of pharmacy instead debated many alternatives and eventually decided to offer an “Evidence Based practice rotation” that will strengthen the students active learning and individualized problem solving techniques prior to subsequent clinical rotations. Moreover, by incorporating good understanding of research and statistical evidence into this rotation one would expect the gained students’ knowledge and clinical expertise to facilitate a full comprehension of the published therapeutic protocols which the preceptors of this rotation had intended to review systematically. This notion was supported in the pharmacy literature where using active learning techniques, journal club, and literature evaluation exercises improved the performance of students on APPE rotations [2].

The aim of this paper is to describe the development and evaluation of Evidence Based Practice (EBP) as an unconventional Advanced Pharmacy Practice Experience rotation during the MERS-CoV epidemic.

King Abdulaziz Medical City (KAMC) and King Saud Bin Abdulaziz University for Health Sciences (KSAU)

King Saud bin Abdulaziz University for Health Sciences (KSAUHS) operates under the Ministry of National Guard Health Affairs along with King Abdulaziz Medical City (KAMC). The latter houses the National Guard Health Affairs hospitals which are highly regarded medical complexes in the region. KSAU-HS is located in Riyadh, Saudi Arabia and houses 6 colleges which include: Medicine, Pharmacy, Dentistry, Nursing, Public Health and Health Informatics, and Applied Medical Sciences. The College of pharmacy at KSAU-HS has a collaborative agreement with the University of Tennessee College of pharmacy that permits the former to receive and teach its pharmacy curriculum leading to the Doctor of Pharmacy (PharmD) degree.

Students at KSAU-HS College of Pharmacy have 8 Advanced Pharmacy Practice Experience (APPE) rotations as a requirement prior to graduation. Each APPE rotation objectives identifies the skills to be acquired, students’ responsibilities, and outcomes to be met. There are 5 required rotations, each of 4 weeks, which include: internal medicine, critical care, drug information, advanced community/outpatient and advanced institutional/inpatient pharmacy. Other elective rotations depend on the individual students’ choices and preceptor availability.

Evidence-based practice (EBP) rotation is an alternative rotation created in response to a moratorium of clinical teaching rounds in the hospital that involved students, due to the MERS-CoV epidemic. This rotation aimed at providing comprehensive and evidence-based updates in the management of common disease states.

At the beginning of the epidemic, the experiential unit with the approval of the pharmacy practice department decided to replace APPE with simulated rounds rotation designed to introduce students to the hospital computerized physician order entry (CPOE) system and to assign each student patients they could review daily. However, despite students’ ability to access the hospital’s CPOE from the university, this plan did not succeed; missing patients’ information that could only be found in patients’ charts, as well as a lack of direct patient-student interactions to review current medications or counseling proved to be obstacles. Therefore, after consulting with the experiential unit, the primary preceptor of the rotation decided to conduct daily thorough topic discussions in which an evidence-based practice rotation was developed. At the time of this rotation’s development, no information was provided regarding the length of the hospital shut down. This led to an uncertainty as to how long will the rotation be other than that it will be used to fulfill the first, 4-week long cycle.

Organization of sessions

Evidence-based practice rotation discussions consisted mainly of current evidence-based practice guidelines review, journal clubs of recently published articles and new drug reviews. Literature review and practice guidelines centered on a variety of commonly encountered disease states (Table 1).

Table 1: Students’ weekly schedule.

Rotation requirements included attending daily topic discussions, presenting at least one journal club and/or topic discussion (30-60 minutes), summarize at least one newly approved medication, active participation during discussion (by answering and asking questions), and the submission of well written and referenced handouts prepared during the rotation.

Eight fourth professional year pharmacy students were enrolled in this rotation. Preceptors met with the students daily in 2 divided sessions: a morning session from 9:30 am-12:30 pm and an afternoon session from 1 pm-4 pm. Usually, 2-3 students presented in each session, with the amount of topics divided equally between students during the week.

At the completion of this rotation and associated activities, the student should be able to:

1. Demonstrate an understanding of basic statistical and analytical concepts.

2. Develop a logical, step-wise approach to search for clinical information using the available resources.

3. Select appropriate databases / resources of clinical / relevant information.

4. Identify the strengths and weaknesses of available clinical information resources.

5. Perform literature searches on different literature databases.

6. Critically evaluate primary, secondary, and tertiary medical literature.

7. Apply information in clinically relevant scenarios.

8. Develop an improved confidence and attitude related to retrieval, dissection, evaluation, peer discussion, and application of published literature.

9. Express clearly recommendation (s) in both written and oral forms with proper justification.

10. Apply up-to-date knowledge to clinically relevant patient scenarios through group and individual interactions using Web-based tools.

11. Develop ability to be part of clinical team during advanced pharmacy practice experiences.

The students’ performance was evaluated using an APPE evaluation form that was developed specifically for the EBP rotation. The form evaluated key areas of literature evaluation/critical thinking skills, clinical knowledge, communication skills, presentations, handouts preparation, and ethical and professional behaviors. The evaluation form used a standard 5-point Likert scale for responses: 5=excellent (student exceeds expectations), 4=very good (student meets expectations with minimal guidance), 3=satisfactory (student meets expectations with minimal to moderate guidance), 2=unsatisfactory (student does not meet expectations), 1=deficient (student displays serious deficiencies in knowledge, skills, and or attitudes)). In addition, strengths, and areas of improvement were discussed with each student individually.

A total score is calculated from the sum of the Likert scale ratings for the 11 EBP rotation learning outcomes (possible range: 5-55). The resultant score is then converted into a grading scale out of 100. In order to pass all APPE rotations, a student must achieve 75% or more on all rotation learning outcomes. Formal evaluation using APPE evaluation form and feedback is provided to students at midpoint and at the end of rotation.

Moreover, a comprehensive survey was administered to the students, before and after the EBP rotation. The survey consisted of 17 positively-phrased items to assess students’ perception about their skills and knowledge in EBP rotation activities as well as overall knowledge.

Each phrase was measured on a 5-point Likert scale: 5=excellent (I gained excellent knowledge and required minimal guidance from my preceptor), 4=very good (I gained very good knowledge and required minimal guidance from my preceptor), 3= satisfactory (I gained fair amount of knowledge and required minimal to moderate guidance from my preceptor, I need more improvement), 2=unsatisfactory (I gained poor amount of knowledge and required extensive guidance from my preceptor, I need extensive improvement) and 1=deficient (I was deficient in this area and required consistent guidance from my preceptor, I need significant and continuous improvement).

Higher ratings indicate better perception of skill/knowledge about pharmacy practice related topics and activities. In addition to the 17 items reflecting knowledge, students were also asked about their preference of taking an EBP rotation as well as whether they support mandating and implementing an EBP rotation in the APPE program. Evaluations and comparisons (before vs. after) were made using paired t tests with a p value of 0.05 being considered significant.

Internal consistency of the survey was checked using the Cronbach’s alpha coefficient, which measures how well items are related to each other or whether scale items measure the same concept or construct [3]. The closer Cronbach’s alpha coefficient is to 1, the greater the internal consistency of the items in the scale. Results showed that Cronbach’s alpha coefficient was 0.93 indicating a high level of internal consistency.

The results from the EBP rotation evaluation are displayed in Table 2. Mean ± SD midpoint evaluation score was 66.3 ± 8.2; median (range): 67 (54-78). Only one of the 8 students was able to meet the 75% requirement at mid-point evaluation. A formal midpoint evaluation and feedback was given to students in a one-on-one setting. The main issues students faced by midpoint evaluations included: underdeveloped presentation skills, incomplete literature evaluation and references, as well as difficulties with punctuality and professionalism. Students were informed of their performances, and had their weaknesses and strengths addressed in individual meetings with the preceptor, where they were encouraged to improve. At the end of the rotation, all 8 students, were able to achieve a 75% or above.

Table 2: Midpoint and Final Evaluation Scores.

Significantly improved EBP evaluations were observed at the end of rotation [Mean ± SD score was 81.9 ± 3.5; median (range): 81.7 (80.4- 84.6); p<0.001.

The results from the pre and post- rotation self-perception of EBP skills/knowledge ratings are summarized in Table 3. The table shows that there is a strong evidence that the EBP rotation substantially improved (P<0.05) students perception of their level of knowledge/ skills about several pharmacy practice related skills and activities including the ability of students to (1) review literature and discuss topics related to infectious diseases: respiratory infections, CNS, UTI, skin, soft tissue, join and bone infections, (2) make evidence based drug therapy selection and (3) discuss evidence supporting treatment recommendations.

Table 3: Comparison between Student Pre and Post EBP Rotation Ratings of the Level of Knowledge Gained from Pharmacy Practice Topics and Activities. N=8.

Results from comparison of a calculated total rating score obtained from summing the ratings for 16 items (items 1-16, see Table 3; possible total score range: 16-80) that reflect student perception of EBP skills/knowledge showed that the EBP rotation improved students’ perception of their skills/knowledge, on average, by approximately 10 points [61.8 ± 7.7 pre EBP rotation vs. 71.6 ± 7.3 post EBP rotation; p = 0.032]. Other results (not shown) showed that 75% of students were in favor of taking an EBP rotation before they actually took it. After having been exposed to the EBP rotation, 88% of students supported implementing it in the APPE program and 75% thought it should be made mandatory.

Evidence-based practice has enormous applications and importance in pharmacy practice and medicine. EBP ensures that interventions with the best evidence are used to improve patient care and optimize drug therapies [4]. It also trains student pharmacists to be able to answer questions during patient care rounds or support an answer with evidence. There are limited published studies about implementation of EBP rotation into advanced pharmacy practice rotations [5-8].

The current study is different than previous publications in that it evaluates a unique EBP rotation that was developed in response to spread of an epidemic. Other published studies evaluated evidence based rotation implementation either as a course during pharmacy school or as a rotation during pharmacy or medical residencies [2,4-8].

EBP rotation was a full 4-week block for all fourth professional year pharmacy students. During this rotation; the students were exposed to variety of practice-based guidelines and topic discussions which enabled them to apply what they learned during their classes and helped them success in their latter clinical rotations.

The results of the final evaluation indicated that students’ knowledge/ skills and performance has improved substantially form midevaluation indicating the effectiveness of the rotation. This finding was similar to previously published studies [5-8]. Hester et al. incorporated literature evaluation exercises in a primary care advanced pharmacy practice rotation and concluded that it had increased the students’ understanding and strengthened their therapeutic recommendations [5]. A 4-week evidence based medicine (EBM) APPE utilizing active learning techniques, on line modules and attendance of drug policy committee meetings and drug reviews increased the students’ evidence based skills and practices [7].

Bookstaver et al. conducted a study to evaluate the impact of elective evidence-based medicine (EBM) didactic course (rather than the practical rotation in our study) on student performance during APPE. They concluded that the skill set acquired from this EBM course improved students’ performance in APPE [2]. The researchers believe that, due to the practical nature of EBP, EBP rotation will improve students’ performance even more than didactic course; however this requires further investigation to be proved.

The EBP rotation improved the students’ perception of their skills/ knowledge (based on the average score of 16 questions), on average, by approximately 10 points (p=0.032). This confirms that the difference in students’ perception about their skills and knowledge levels was not just statistically significant but also relatively large and thus practically important.

The majority of students supported implementing the EBP rotation in the APPE program and thought it should be made mandatory. Similar to our findings, Akl et al. combined an EBM rotation into an internal medicine residency program for medical students. A survey was administered as a pre and post-test to assess the effectiveness of and attitude towards EBM rotation. There was a noticeable favorable attitude in the surveys of medical residents towards EBM rotation [8].

This study had several challenges and limitations. The small number of students enrolled in the course is the main limitation. Larger sample size on a more diverse students’ population would have demonstrated more reliably the values and limitations of this experience. Another challenge was the coordination of the schedule due to shortage of preceptors and unavailability of joint appointed clinical pharmacists due to the MERS-CoV crisis.

Although the development of an EBP rotation was under unforeseen circumstances, the results of this study proposes that implementation of an EBP rotation into APPE clerkships may result in favorable outcomes. Completion of an EBP rotation will prepare students with skills necessary to evaluate clinical literature and practice critical thinking when it comes to patient care decisions. Future plans include offering this rotation in the beginning of APPE to help students excel in other clinical rotations, however the decision of whether it will be an elective or required is still pending. In addition, a clear curriculum will promote expanding the implementation and help in providing high quality of patient care.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.