Vikram Londhey 1, Charulata Londhe 2
1MD, Additional Professor, Unit Head of Medicine and Incharge Rheumatology, HinduhridaySamrat Balasaheb Thackeray Medical College and DR.R.N.Cooper Hospital, Mumbai, India
2MD, Associate Professor, Medicine Department LTMMC and GH Sion Mumbai, India
ABSTRACT
Internship is an appropriate time for enhancement and assessment of clinical skills. This study was undertaken with an objective to assess the interns at their workplace by DOPS for the procedural skill of blood collection. Mean scores of DOPS rating increased from 1st DOPS (5.45±0.43) to 2nd DOPS (7.56±0.51) and were sustained during the 3rd DOPS (7.84±0.34). Confidence levels and satisfaction of interns increased from 72% to 88% to 95% by third DOPS. DOPS tests the ability to perform a particular procedure in a real-life clinical setting. This study shows that DOPS is one of the useful tools in WPBA.
Keywords: DOPS, DOPS assessment, assessment clinical procedures, clinical procedures medical education, medical education internship, internship
Date submitted: 25-May-2024
Email: Vikram Londhey (vikramlondhey@yahoo.com)
This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.
Citation: Londhey V, Londhe C. Assessment of medical interns in India using Directly Observed Procedural Skills in the workplace. Educ Health 2024;37:281-283
Online access: www.educationforhealthjournal.org
DOI: 10.62694/efh.2024.102
Published by The Network: Towards Unity for Health
Internship is an appropriate time for enhancement of clinical skills particularly blood collection. DOPS (Directly Observed Procedural Skills) assesses the highest level of Miller’s Pyramid, the action level (does) that follows knowledge (knows), competence (knows how) and performance (shows how). So, it was decided to use DOPS as an instrument for WPBA (Workplace-Based Assessment) of medical interns for the procedural skill of blood collection. CBME (Competency-Based Medical Education) has been implemented by NMC (National Medical Commission) since 2020 to develop the Indian Medical Graduate Training Programme. For CBME interns, assessment of skills will be necessary before the graduate degree is conferred. Hence this educational project was undertaken as part of The Foundation for Advancement of International Medical Education and Research (FAIMER) Fellowship Program to assess the procedural skill of blood collection of the interns by using DOPS as the assessment tool at their workplace.
1) The objective was to assess the interns at their workplace by DOPS for the procedural skill of blood collection. 2) To give feedback to increase confidence levels and performance of the interns at their workplace. 3) To collect feedback from interns about their participation in the research project.
The project was conducted in medicine wards at a medical college in the year 2021–2022. Two Assistant Professors and one Associate Professor were trained to give constructive feedback, and who also performed the role of observing the interns for one DOPS each per intern. The procedure of blood collection was shown to the participating interns by means of a videoclip, followed by hands-on training on a mannequin, and then on actual patients.
Each intern was assessed by DOPS thrice at an interval of one week between each DOPS for blood collection by the patient’s bedside, by the trained faculty, using a validated checklist as a part of formative assessment. Feedback was given to each intern about that skill encounter after the assessment. Feedback was collected from the interns to assess their levels of confidence in performing the skill of blood collection, and to know about their overall experience of participating in this educational project.
The data was analyzed statistically using socscistatistics.com (free version) in percentages, mean, standard deviation of means, repeated measures ANOVA and Banfuroni test. Feedback given by the students had open ended questions which were analyzed by giving codes to different themes that emerged and then expressed as percentages.
Out of 132 interns, 100 interns consented to participate; 32 declined consent. The mean scores of DOPS rating increased from 1st DOPS (5.45±0.43) to 2nd DOPS (7.56±0.51) and were sustained during the 3rd DOPS (7.84±0.34). The difference between DOPS rating between first and second was statistically significant using Banfuroni test (p<0.001). The difference between DOPS rating between second and third was not statistically significant (p=0.631). The overall difference was also statistically significant (p=0.000) using repeated measure ANOVA. The confidence levels, performance and satisfaction of interns increased from 72% to 88% to 95% by the third DOPS. Five percent of the interns were given remedial measures, like retraining on the mannequin, after the second DOPS which improved their performance.
Interesting themes emerged from the feedback received from the interns, like fear, apprehension, understanding how to administer informed consent, receiving constructive feedback etc. Twelve percent said fear was aroused in their minds as they were going to be evaluated by faculty, they felt like they were appearing for an exam. This depended on the seniority of the observer. Twenty percent said they were otherwise comfortable in doing the procedure of the blood collection but had some apprehension as they were being observed and were going to receive feedback. (This was true for the first DOPS). Eighty-eight percent were happy to participate in a research project and were excited after reading the informed consent and the participant information sheet. An encouraging 28% were eager to help in the project further if any help was required: and 22% understood the overall process of administering consent in a research project. We found that keeping the faculty motivated throughout the project was a challenge.
In a review of literature by Naeem, DOPS was found to be a high-quality instrument with good reliability and acceptability.1 Erfani Khanghahi et al have mentioned that DOPS can be used as an effective and efficient evaluation method to assess medical students, as it has appropriate validity, reliability, positive impact on learning and students’ high level of satisfaction.2,3 The present study made use of three repeated DOPS for each intern. From 1st to 2nd DOPS, the mean overall rating increased significantly and remained sustained after 3rd DOPS. Repeated DOPS resulted in better performance, as seen in our study. Sethi S and Badyal DK found that repeated DOPS improved performance amongst interns for four core areas of ophthalmic procedures.4 Arezou Farajpour et al have recommended that WPBA should be considered in medical school for UG students with modified DOPS and not only in PGs.5 In this study it was shown that the final ward exam mean scores were significantly more than the mean scores of DOPS. Thus, students with high grades in the final exam may not have acquired adequate procedural skills.5 In a study by Hassan, practice of DOPS provided the means for supervisors, and incentives to trainees, to monitor their self-directed learning objectives and to improve quality of procedural skills acquired during their on-job (apprenticeship) learning in clinical education.6
The confidence levels of the interns improved from DOPS 1 to 2 to 3. Analysis of the feedback from the interns revealed that their performance of DOPS encounter was dependent upon how senior a supervisor or observer was; and what was the earlier experience of the trainee with that supervisor. So repeated DOPS is necessary with different supervisors to observe the performance of the trainee.
Supervisors’ feedback about their satisfaction and overall experience of participating in the project was not collected.
The three encounters of DOPS repeated for assessing the skill of blood collection helped to increase the confidence and performance of the interns. This study adds further evidence that DOPS is one of the useful tools in WPBA for formative assessment.
1. Naeem N. Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS). Journal of The College of Physicians and Surgeons Pakistan.2013;23(1):77–82. PMID: 23286629.
PubMed
2. Masoumeh Erfani Khanghahi, Farbod Ebadi Fard Azar. Direct observation of procedural skills (DOPS) evaluation method: Systematic review of evidence. Medical Journal of The Islamic Republic of Iran.2018;32:45. https://doi.org/10.14196/mjiri.32.45. eCollection 2018
3. Bagher M, Sadeghnezhad M, Sayyadee T, Hajiabadi F. The Effect of Direct Observation of Procedural Skills (DOPS) Evaluation Method on Learning Skills among Emergency Medicine Students. Iranian Journal of Medical Education.2014;13(12):1073–81. URL: http://ijme.mui.ac.ir/article-1-2854-en.html
4. Sethi S, Badyal DK. Clinical procedural skills assessment during internship in Ophthalmology. Journal of Advances in Medical Education and Professionalism. 2019 ;7(2):56–61. https://doi.org/10.30476/JAMP.2019.44709
PubMed PMC
5. Farajpour A, Amini M, Pishbin E, Mostafavian Z, Akbari Farmad S. Using Direct observation of Procedural Skills (DOPS) to assess undergraduate medical students. Journal of Advances in Medical Education and Professionalism. 2018;6(3):130–136. PMID: 30013997; PMCID: PMC6039823.
PubMed PMC
6. Hassan S. Faculty Development: DOPS as workplace-based Assessment. Education in Medicine Journal.2011;3(1):e32–43. https://doi.org/10.5959/eimj.3.1.2011.or4
Crossref
© Education for Health.
Education for Health | Volume 37, No. 3, July-September 2024