Login Register Cart 0
Generic placeholder image

Current Drug Therapy

Editor-in-Chief

ISSN (Print): 1574-8855
ISSN (Online): 2212-3903

Back
Journal Home About Journal Editorial Board Journal Insight Current Issue Volumes/Issues
Subscribe
Research Article

Inter-rater Reliability of Preceptors on Clinical Pharmacy Competency Evaluation

Author(s): Dixon Thomas*, Sherief Khalifa, Jayadevan Sreedharan and Rucha Bond

Volume 16, Issue 2, 2021

Published on: 09 December, 2020

Page: [148 - 153] Pages: 6

DOI: 10.2174/1574885515999201209202624

Price: $65

Abstract

Background: Clinical competence of pharmacy students is better evaluated at their practice sites compared to the classroom. A clinical pharmacy competency evaluation rubric like that of the American College of Clinical Pharmacy (ACCP) is an effective assessment tool for clinical skills and can be used to show item reliability. The preceptors should be trained on how to use the rubrics as many inherent factors could influence inter-rater reliability.

Objective: To evaluate inter-rater reliability among preceptors on evaluating clinical competence of pharmacy students, before and after a group discussion intervention.

Materials and Methods: In this quasi-experimental study in a United Arab Emirates teaching hospital, Seven clinical pharmacy preceptors rated the clinical pharmacy competencies of ten recent PharmD graduates referring to their portfolios and preceptorship. Clinical pharmacy competencies were adopted from ACCP and mildly modified to be relevant for the local settings.

Results: Inter-rater reliability (Cronbach's Alpha) among preceptors was reasonable being practitioners at a single site for 2-4 years. At domain level, inter-rater reliability ranged from 0.79 - 0.93 before intervention and 0.94 - 0.99 after intervention. No inter-rater reliability was observed in relation to certain competency elements ranging from 0.31 - 0.61 before the intervention, but improved to 0.79 - 0.97 after the intervention. Intra-class correlation coefficient improved among all individual preceptors being reliable with each other after group discussion though some had no reliability with each other before group discussion.

Conclusion: Group discussion among preceptors at the training site was found to be effective in improving inter-rater reliability on all elements of the clinical pharmacy competency evaluation. Removing a preceptor from the analysis did not affect inter-rater reliability after group discussion.

Keywords: Inter-rater reliability, preceptorship, pharmacy, clinical competence, correlation, coefficient.

« Previous Next »
Graphical Abstract

[1]
Croft H, Gilligan C, Rasiah R, Levett-Jones T, Schneider J. Current trends and opportunities for competency assessment in pharmacy education-a literature review. Pharmacy (Basel) 2019; 7(2): 67.
[http://dx.doi.org/10.3390/pharmacy7020067] [PMID: 31216731]
[2]
Lee M, Badowski ME, Acquisto NM, et al. ACCP template for evaluating a clinical pharmacist. Pharmacotherapy 2017; 37(5): e21-9.
[http://dx.doi.org/10.1002/phar.1927] [PMID: 28417479]
[3]
Saseen JJ, Ripley TL, Bondi D, et al. ACCP clinical pharmacist competencies. Pharmacotherapy 2017; 37(5): 630-6.
[http://dx.doi.org/10.1002/phar.1923] [PMID: 28464300]
[4]
Liu C. An introduction to workplace-based assessments. Gastroenterol Hepatol Bed Bench 2012; 5(1): 24-8.
[PMID: 24834194]
[5]
Blommel ML, Abate MA. A rubric to assess critical literature evaluation skills. Am J Pharm Educ 2007; 71(4): 63.
[http://dx.doi.org/10.5688/aj710463] [PMID: 17786251]
[6]
Vos SS, Ray ME. Thomas D, eds What to Expect From a Pharmacist Preceptor Clinical Pharmacy Education, Practice and Research. Amsterdam: Elsevier 2019; 1: pp. 399-406.
[7]
Hayward MF, Curran V, Curtis B, Schulz H, Murphy S. Reliability of the interprofessional collaborator assessment rubric (ICAR) in multi source feedback (MSF) with post-graduate medical residents. BMC Med Educ 2014; 14: 1049.
[http://dx.doi.org/10.1186/s12909-014-0279-9] [PMID: 25551678]
[8]
Jonsson A, Svingby G. The use of scoring rubrics: Reliability, validity and educational consequences. Educ Res Rev 2007; 2(2): 130-44.
[http://dx.doi.org/10.1016/j.edurev.2007.05.002]
[9]
Wei CJ, Lu TH, Chien SC, et al. The development and use of a pharmacist-specific Mini-CEX for postgraduate year trainees in Taiwan. BMC Med Educ 2019; 19(1): 165.
[http://dx.doi.org/10.1186/s12909-019-1602-2] [PMID: 31118004]
[10]
Boursicot K, Etheridge L, Setna Z, et al. Performance in assessment: consensus statement and recommendations from the Ottawa conference. Med Teach 2011; 33(5): 370-83.
[http://dx.doi.org/10.3109/0142159X.2011.565831] [PMID: 21517685]
[11]
Gulf Medical University. Doctor of Pharmacy. Available from: https://gmu.ac.ae/college-pharmacy/doctor-of-pharmacy/#Overview (Accessed November 02, 2020).
[12]
Medina MS, Plaza CM, Stowe CD, et al. Center for the advancement of pharmacy education 2013 educational outcomes. Am J Pharm Educ 2013; 77(8): 162.
[http://dx.doi.org/10.5688/ajpe778162] [PMID: 24159203]
[13]
Gilliam EH, Brunner JM, Nuffer W, Patel TC, Thompson ME. Design and content validation of three setting-specific assessment tools for advanced pharmacy practice experiences. Am J Pharm Educ 2019; 83(9): 7067.
[http://dx.doi.org/10.5688/ajpe7067] [PMID: 31871343]
[14]
Haines ST, Pittenger A, Plaza C. Describing entrustable professional activities is merely the first step. Am J Pharm Educ 2017; 81(1): 18.
[http://dx.doi.org/10.5688/ajpe81118] [PMID: 28289308]
[15]
Zhou L, Almutairi AR, Alsaid NS, Warholak TL, Cooley J. Establishing the validity and reliability evidence of preceptor assessment of student tool. Am J Pharm Educ 2017; 81(8): 5908.
[http://dx.doi.org/10.5688/ajpe5908] [PMID: 29200445]
[16]
Kassam R, Collins JB. Validation of a survey instrument to evaluate students’ learning during community-based advanced pharmacy practice experiences. Am J Pharm Educ 2009; 73(6): 106.
[http://dx.doi.org/10.5688/aj7306106] [PMID: 19885075]

Rights & Permissions Print Cite
Article Metrics
6
1
© 2024 Bentham Science Publishers | Privacy Policy