There has been a transition in pharmacy practice in the United States from a product-focused curriculum to a much more patient-centered emphasis. Graduating students need to possess certain key abilities that will enable them to function independently as entry-level pharmacy practitioners. The Accreditation Council for Pharmacy Education (ACPE) has established key outcomes that should be addressed within pharmacy curricula in order to prepare students for the practice of pharmacy. In 2007, ACPE expanded these outcomes, publishing new standards that emphasized the role of experiential education in the training of pharmacy students [1]. These standards called for a minimum of 300 introductory pharmacy practice experience (IPPE) hours, which, combined with 1440 advanced pharmacy practice experience (APPE) hours, comprises a full one-third of the entry-level doctorate of pharmacy curriculum. Within these experiential activities, students are to be given increasing levels of responsibility to prepare them for independent practice.

The University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences (SSPPS) have established a longitudinal experiential program for students that span the entire 4-year curriculum. This allows students to integrate skills and knowledge they are acquiring during didactic and skills courses and apply them in various settings. Students begin their experiential curriculum by gaining practice in community pharmacies and by providing service-learning nutrition lessons in elementary school classrooms, transitioning into health systems and hospital settings later in their 2nd year. With the rich diversity and patient exposure these IPPEs provide, it was determined that the experiential education portion of the curriculum would provide an excellent opportunity to assess two specific ability domains across a breadth of experiences: students’ communication and professionalism skills. A 48 item comprehensive rubric was developed, focusing on multiple aspects of communication and professionalism (Table 1). Preceptors at each of these experiential sites are tasked with evaluating their student(s) by utilizing this tool, ranking the student’s ability to perform across nine domains all focused on these core abilities. The utilization of this common rubric tool across all of the students’ IPPEs allows for a global assessment of students’ performance in the context of communication and professionalism. This manuscript reports on the performance of one class in the entry-level PharmD curriculum tracked across the first five semesters of the program, based upon this scored rubric.

Table 1: Comprehensive communication and professionalism rubric items.

The performance of students in the inaugural class of the revised experiential curriculum (Class of 2016) was analyzed by reviewing the standardized communication and professionalism IPPE assessment rubric. This rubric utilizes a 4-point scale with the definitions of a) has mastered the responsibility and is able to educate others through the use of evidence and interpersonal skills, b) without prompting or support from preceptors is able to demonstrate this ability, c) knows responsibility but is inconsistent in performing it, and d) does not seem to know this responsibility. Preceptors also receive a document that defines each of these fields with examples of performance that would match each category. These definitions were originally formed and tested based upon the educational literature describing increasing levels of student performance. The full rubric and definitions are available from the authors upon request. This rubric was completed by all IPPE preceptors to assess students’ performance across the scope of introductory practice experiences in the curriculum. For this manuscript, specific years of the program are designated as “P1, P2, or P3”, with clarifying fall or spring semesters listed. The dates of inclusion for this review followed the anticipated progression for the class of 2016, beginning with the P1 fall semester that commenced in August 2012, following this class’s performance through the end of the P3 fall semester, which ended in December 2014. All submitted rubrics for any student who was registered as part of this class at the time of rubric submission were included. Records belonging to students who may not have progressed with the class on the anticipated schedule were included only for those semesters in which the student was identified as a member of the 2016 cohort.

The communication and professionalism rubric consists of four and five sub-domains, respectively, and are provided in (Table 1). Student performance scores were calculated by summing the numeric scores assigned by the preceptor and dividing by the maximum points possible. Items which were marked as not applicable by the preceptor were not included, and were removed from the denominator of possible points. In contrast to the designed grading scheme of this rubric, by which weighting of scaled responses increased in rigor with each passing semester, all weighting was removed for the purpose of this analysis. Only evaluations submitted through the online portfolio system, E^*Value, (Advanced Informatics Systems, Minneapolis, MN) were included for review.

Statistics were calculated using IBM® SPSS® Statistics v22. Comparison of mean student performance scores across all IPPE evaluations were made by analysis of variance with a two-sided level of significance set at 0.05. Sub-group analyses were performed by looking at scores grouped by IPPE setting and semester. Bonferroni corrections were used for pairwise subgroup comparisons.

Between August 2012 and December 2014, 950 communication and professionalism IPPE assessments were completed for 156 individual students registered as part of the graduating class of 2016. Assessments were submitted by 29 service learning IPPE preceptors, 133 community pharmacy IPPE preceptors, and 147 health system IPPE preceptors. Not all assessment rubrics were submitted based on a 1:1 student to preceptor ratio; the service-learning preceptors evaluated students as a group, and a few subgroups of students received multiple evaluations during the same experience by nature of working with several preceptors at the same practice site. All students received multiple assessments from different IPPE programs during three of the five semesters analyzed, based on the overlapping design of the IPPE curriculum.

As a class, the rubric demonstrated that students performed well across the five IPPE semesters, with an average score of 92.95 % (95% CI 92.48-93.42). The mean score for communication across all five IPPE semesters was 98.42 % (95% CI 98.16 – 98.68), while the mean professionalism score was significantly lower with an average score of 89.89% (95% CI 89.21 – 90.57, p<0.001). Individual grade performance in the class progression was consistent with these findings with the class GPA ranging from 3.96 to 3.99 on a 4.0 scale.

Communication scores varied little during the progression of the IPPE curriculum with mean scores ranging from the lowest of 92.7% (P1 spring service learning rubric) to a high of 99.08% (P2 spring community IPPE rubric). The communication domains were consistently lower during the P2 spring semester. A sub-group analysis of the first communication domain for the spring P1 semester service-learning IPPEs revealed that these scores were significantly lower in comparison to all other IPPE environments across all subsequent semesters (p<0.001). Changes in student performance are illustrated in (Figures 1 and 2).

Figure 1: Changes in student performance.

Figure 2: Changes in student performance.

Figure 3: IPPE curriculum.

All five professional domains demonstrated significant differences when comparing scores by semester. A sub-group analysis by IPPE setting and semester identified that professionalism domains 1 and 2 were marked the lowest by P1 spring semester service-learning preceptors while domains 3, 4, and 5 were marked the lowest during the student’s initial health system IPPE evaluation in the spring of the P2 year. Students demonstrated significant improvement in their professionalism domain scores during subsequent IPPE semesters in several of the professionalism domains (Table 2).

Table 2: Professionalism domains improved with subsequent IPPE exposure.

There is strong evidence demonstrating that the skills of professionalism and communication are essential for entering practice as an entry-level pharmacy practitioner [2-5]. Thompson and colleagues surveyed practicing pharmacists in both community and health system settings, asking them to rank which characteristics were most essential when considering hiring a newly graduated pharmacist [6]. Of the 20 characteristics listed for prioritization, communication was ranked as the highest preferred characteristic a graduate could possess across all settings, with professionalism ranking second. While students may possess tremendous pharmacy knowledge, their ability to conduct themselves in a professional manner and convey that knowledge to patients and colleagues is essential. Bond and colleagues surveyed practicing IPPE and APPE preceptors across a diverse assortment of practice settings, to gauge what value these pharmacists place on curricular activities driven by ACPE. While not all required abilities were perceived as valuable by this practice community, communication was clearly identified as having paramount importance, with 86% of respondents classifying the skill as “very important” [7].

According to Brown and colleagues, a comprehensive IPPE program is the single most important experience a school can provide to their students to instill the right professional values for practice [8]. The American College of Clinical Pharmacy position statement on educational outcomes necessary to prepare students for residency programs state that schools should emphasize, assess, and provide adequate opportunities to practice communication and professionalism [9]. Both professionalism and communication are two domains that should be promoted and assessed early and longitudinally throughout PharmD curricula. Adequate and appropriate placement of IPPE activities in these domains, and those that can be easily assessed, is the key to ensuring that students are improving over time. A longitudinal IPPE program offers opportunities for students to practice communication skills in a variety of settings, and allows students to actively incorporate their professional values as it relates to patient care.

The University of Colorado SSPPS provides students with a diverse assortment of experiences to practice these critical skills. The P1 service-learning activities task students in groups of 4-6 to offer interactive presentations to 3rd, 4th, and 5th grade classrooms focused on health promotion and nutrition topics. The elementary school teachers evaluate these groups based upon their group interaction and performance with the school children in both professionalism (reliability, adaptability, working with others, honesty, respect, attire) and communication (verbal skills, classroom management, nonverbal cues, active listening, and establishing rapport with the children) at the end of the P1 fall and spring semesters. Simultaneously, students also complete IPPE hours in a community pharmacy setting for a total of 16 3-hour visits across four semesters beginning with P1 fall and ending in the P2 spring semester. This provides a very different forum for students to demonstrate their communication and professionalism abilities, which they are evaluated on at the end of each semester. While there are task-oriented assignments required in the community setting, there is a purposeful emphasis on students counseling patients on both self-care product selection as well as prescription drugs. Students transition into the health-system setting in P2 spring and complete 10 health-system visits across the next two semesters. This area of pharmacy practice involves a deeper level of interprofessional interactions and a very different patient base compared to community practice, challenging students to demonstrate these skills in a much different setting. Evaluations for the health-system experiences occur again at the end of each semester.

The utilization of one standard rubric across all of these experiences allows students to receive continuous feedback regarding their proficiency in demonstrating proper communication and professionalism abilities from many different professional preceptors, in practice settings where these abilities are highly coveted. Although the weighting of these categories differs slightly across semesters, the domains remain consistent throughout. The rubric relies on outside evaluators who are quite diverse in their experiences and training, each providing unique insight when utilizing the rubric tool to evaluate students. This challenges students to develop and demonstrate these skills uniformly across very different settings.

Looking at the rubric’s performance with the class of 2016, several trends become apparent. First and foremost, students tended to perform better in their communication skills than they did with professionalism. This was consistently observed across all five semesters in all three settings. This is not surprising; communication is a skill that students begin developing from early childhood, and the SSPPS program prioritizes good communication both with its pre-requisites and through the admissions process, where students engage in both individual interviews and group activities. It would be expected that a student entering the curriculum should have strong baseline communication abilities, which is reflected in the overall very high scores students received across all semesters. By contrast, professionalism is a concept that is somewhat nebulous and poorly understood by students as they enter pharmacy school; it is necessary to teach and model this skill in order to help students understand how performance translates to “professional”. The fact that students performed well after the first semester, and generally tended to improve as they progressed, suggests that the current curricula and preceptor support are cultivating this professionalism concept and inculcating it into students’ abilities.

With the exception of the P1 spring semester, overall communication scores tended to improve or stay consistent as the students progressed through the curriculum. This suggests some adaptability in the students, as they faced new pharmacy practice environments which require different aspects of communication, yet scoring suggested that students were able to meet these challenges and continue to develop or maintain these skills. Students receive a year-long course in professional communications in their first year, which may also contribute to the consistent high performance that occurs after this time. There is a slight drop in performance as students entered the P2 spring year, which is likely attributable to entering the health-system setting. As mentioned, this is a very different environment to what students had experienced up to this time, challenging students to apply these skills in a different manner. Health-system preceptors are also thought to be more stringent graders than their community pharmacy counterparts, so similar performance may be rated lower in this setting.

In regards to the P1 year, it is possible that the drop in scores relates to higher expectations by the preceptors. Both the service-learning and community settings are maintained for students across the entire 1st year, so preceptors in the spring would be seeing their student for a second time, and may be evaluating them more stringently. Mistakes or performance gaps in communication may have been overlooked by these preceptors in the first semester, since the student is so new to both their environment and to the pharmacy school program, but those same deficiencies a semester later may have resulted in lower scoring. It is interesting that the most pronounced drop in performance came in the service-learning setting; the fall component of this IPPE involves a larger emphasis on observing the school teacher and engaging with the children as mentors and role-models. In the spring, students are challenged with a much larger teaching component, delivering five live presentations to the classroom of students as a group, compared to only two presentations in fall. Although a public speaking course is a prerequisite for the SSPPS entry-level Doctor of Pharmacy program, speaking to elementary school children may not be a natural skill of these pharmacy students. By comparison, communication in the community pharmacy setting is much more individualized, with students engaging directly with patients. This may be an easier skill to develop in the new pharmacy student.

With professionalism, the decline in the P2 spring year is even more pronounced than with communication, likely for the same reasons of entering a health-system setting. What is encouraging here is that students maintain this same setting as they enter P3 fall, and scores across the professionalism domains improved between these semesters, again showing the successful adaptability and refinement of skills as the students progress. (Table 2) highlights this nicely, depicting how students who remained in the same settings across multiple semesters honed their skills and improved their performance as they progressed.

Looking at the individual domains within communication and professionalism, it is noted that students performed more highly in some areas than others. With communication, students tended to be ranked more highly in their ability to develop a trusting relationship than other domains, such as structuring the encounter or retrieval of information. This, again, is likely due to areas where students have had experience prior to pharmacy school, which may have helped hone their abilities to communicate. Professionally, students appeared to be better at upholding principles of integrity and respect and reliability, responsibility, and accountability, compared to relationship with others and lifelong learning. This may be due to some of these concepts being learned earlier in life, but may also relate to how their clinical work at the sites translated into the different domains; preceptors may have found more examples of how the student was reliable and respectful, while measuring lifelong learning may have been perceived as more difficult to assess. One domain, citizenship and professional engagement, was an outlier in professionalism as having the lowest relative scores across all semesters, again, likely due to challenges in displaying this ability in the various sites or in the preceptor’s difficulty to assess it. The fact that all domains showed very similar trends in both communication and professionalism is very positive, as it demonstrates that students can develop and consistently display these skills.

The utilization of one standardized rubric that evaluates students’ performance in very diverse practice environments is an extremely useful way to assess whether students are developing the abilities that they will need to successfully enter practice after graduation. The ability to compare “apples with apples” by having the preceptors rank students into specific domains of performance based on the students’ observed performance is critical to ensure that these abilities have been effectively modeled and taught. Viewed in totality, these data provide confidence that the cohort of students analyzed through this rubric’s performance have successfully developed the key characteristics of communication and professionalism through the first five semesters of their curriculum. These components will be continued to be evaluated as students complete their last IPPE semester in P3 spring and move into their APPE year, where they will have ample opportunity to utilize these skills as they practice full-time across multiple settings in the 4th and final year of the curriculum.

A standardized rubric focused on nine domains of professionalism and communication provides objective data that pharmacy students are successfully developing these key skills within the experiential IPPE curriculum.