Sangeeta A

Effect of formative assessment on documentation of pediatric physical examination by undergraduate medical students

Arun Kumar T¹, Sangeeta A², Rashmi R³, Jyothi S⁴

¹Dr Arun Kumar T, Associate Professor of Pediatrics Unit Head, ²Dr Sangeeta A, Associate Professor of Pediatrics,³Dr Rashmi R, Associate Professor of Physiology, ⁴Dr Jyothi S,Associate Professor ofPhysiology. All authors are affiliated with Karpagam Faculty of Medical Sciences and Research, Coimbatore, India.

Address for Correspondence: Dr Sangeeta A, Assistant Professor of Pediatrics, Karpagam Faculty of Medical Sciences and Research, Pollachi Main Road, Coimbatore. Email: arunkfmsr@gmail.com

Abstract

Introduction: Medical educators have serious concern about the decline in physical examination skills of undergraduate medical students. Though formative assessments act as an educational tool to improve students’ performance, data to establish their educational benefits in recording pediatric physical examination skills are lacking. This study was conducted to analyse the competency gaps in documentation of the physical examination skills and to assess the effect of formative assessment on case sheet documentation. Methodology: This comparative study was conducted on fifty undergraduate students of sixth semester undergoing paediatric clinical posting. During the first session students’ examined a standardized patient and documented the findings in a case sheet. Marks were awarded based on checklist. Feedback was provided based on the case record. The same teaching learning program was conducted in the second session and results were compared. Results: Documentation regarding Central nervous system (4.22+0.932) examination was the area of concern noted in our study. The perception of the students’ about their proficiency in various components of physical examination assessed with the questionnaire was statistically significant (p<0.001). Formative assessment had a statistically significant effect on the performance and marks of students’ (p<0.001). Conclusion: Our findings indicate that formative assessment helps identify the competency gaps and increases the proficiency in documentation of pediatric physical examination skills. Adequate frequency of formative evaluation with immediate feedback has a beneficial effect on the students’ performance.

Key words: Assessment, Clinical skills, Feedback, Formative evaluation, Medical education

Manuscript received: 20th September 2017, Reviewed: 30th September 2017
Author Corrected: 9th October 2017, Accepted for Publication: 14th October 2017

Introduction

The physical examination is recognized as fundamental to the practice of medicine as critical decisions regarding management emerge from clinical findings [1]. The aim of medical education is to produce competent physicians and clinical skills from the core of physicians competence [2]. Inspite of current advances in technology and science,the physical examination skills are essential due to its diagnostic contribution, positive effect on patient care and cost optimization[3,4].Repeated careful examination of recently admitted patients has been shown to change diagnoses and treatment [5]. Expertise in the proficiency of physical examination should therefore be a priority in the training of medical undergraduates. There isoverall decline in the documentation of physical examination skills because of busy clinical workloads, lack of clinical teaching, increased availability and use of the specialized diagnostic equipment such as ultrasound, CT and MRI [6, 7].

Overuse of Technology has reduced the teaching and skill component in physical diagnosis and undermined the value of these skills resulting in shift away from bedside teaching and supervision of physical examination skills during undergraduate trainingand early years of practice [8]. At present there is a decline in the standards of physical examination skills of the undergraduate students. Likewise, other studies show deficiencies in physical examination skills among interns which raises serious concern on the quality of clinical care[9-11]. In the current scenario, significant modifications are needed in both medical school and hospital culture regarding physical examination skill acquisition, documentation and retention.

Medical records act as a communication tool between students and instructors during undergraduate years and facilitates continuity of care during medical practice. The Medical records is a legal document substantiating the appropriate diagnosis and treatment provided to the patient. Hence documentation skills must be provided appropriate time and attention to ensure it is “comprehensive and accurate”[12]. Various studies have utilized established methods to improve documentation with beneficial results [13-16].

Assessment drives learning. Medical educators emphasize more on physical examination skills of undergraduate students and document the findings comprehensively in the case records. They struggle to find methods to augment documentation and structure of students’ skills. Assessment of students’ competence provide insight into actual performance done habitually, as well as the capacity to adapt, change, find and generate new knowledge thereby improve performance [17]. The clinical skills and reasoning power of the students’ appear competent in the familiar areas but appear much less competent in the unfamiliar ones [18]. Greater emphasis should be placed on training in basic clinical skills and its precise documentation in real patient setting.

The method and timing of assessment is a challenge for medical instructors. Formativeassessment (FA) provide feedback to orient the learner and reinforce students’ intrinsic motivation to learn and inspire them to achieve set goals [19]. It guides future learning, provides reassurance, promotes reflection and shape values. It motivates learners to improve the clinical practice and thereby achieve the desired goals and skills [20]. FA helps students’ understand their own learning, develop appropriate strategies for “learning to learn”, acquisition of new concepts and develop skills valuable for learning throughout their medical practice.

The current assessment of clinical skills is through summative Direct Observation Clinical Encounter Exam (DOCEE) and summative Objective Structured Clinical Examination (OSCE). They are conducted at the end of the clinical posting with no feedback provided to the students’. On most occasions, the only feedback students’ receive is their grades or marks. Hence the current study was done to assess the effect of formative evaluation on the documentation of physical examination skills among undergraduate students’.

Materials and Methods

Aims and Objectives:

1)  To identify the competency gaps in documentation of pediatric physical examination.
2)  To assess the effect of formative assessments on documentation of pediatric physical examination skills among undergraduate medical students.

Methodology

This comparative study was done at the Department of Pediatrics, Karpagam Faculty of Medical Sciences and Research over a period of 6 months after ethics committee approval. Informed consent was taken from the study participants. 50MBBS students of sixth semester were posted in batches of 10 on rotation for one month duration. The posting was split into two session of 15 days duration each. At the start of session, students’ were informed about the learning and assessment method and provided with a copy of the entire check list, which specifies each of the maneuvers to be performed with a detailed description of how to perform it correctly. Small group teaching on history taking, physical examination including general examination, cardiovascular, respiratory, abdomen, central nervous system and documentation was conducted during the tenure of posting. At the end of first session (pre feedback),the students’ were instructed to examine a standardized patient for 30 minutes and document the findings. The case record was evaluated using the check list.

Students’ opinion about the session was collected through the structured questionnaire. Students’ received a feedback on all items including the omitted or incorrect ones with a written narrative to reinforce the feedback provided verbally. Contact classes were taken during the second session to narrate and reinforce the appropriate skills. At the end of postings, they were allowed to examine a standardized patient and document the findings within a duration of 30 minutes. Post feedback session opinion was again collected and compared. The case sheet records were scrutinized and evaluated based on the checklist. The marks provided to the students was verified by a senior facultymember. The feedback from the students’ and their documentation skills on a structured case record was analyzed using IBM SPSS 22.0 Software.

Inclusion criteria: 50 MBBS students of sixth semester.
Exclusion criteria: Students not having 100% attendance in all the sessions

Structured questionnaire
Name:
Session:                           Roll No:
Rate the following components of pediatric physical examination based on your understanding on a scale of 5 (strongly agree) to 1 (strongly disagree)

S. No.	Component	Strongly Agree(5)	Agree (4)	Neutral (3)	Disagree (2)	Strongly Disagree(1)
1.	General Examination
2.	Cardiovascular
3.	Respiratory
4.	Abdomen
5.	Central nervous system

6. The time given to document was adequate: Yes/No.
7. The aims and objectives of the session was communicated clearly by the instructor: Yes/No.
8. Feedback during the course of posting was helpful to my learning: Yes /No.
9. The feedback was relevant to my learning experience: Yes /No.
10. I was able to identify my competency gaps based on the feedback: Yes /No.
11. Do you wish to have similar feedback session in other clinical postings: Yes / No.
12. Any additional comments for the teacher:

Signature of student Signature of teacher

Results

50 undergraduate medical students of sixth semester were taken up for the study.

Table-1: Analysis of first and second session questionnaire:

S. No.	Component	n	Mean marks			SD		Correlation	Sig(p)	‘t-score
S. No.	Component	n	First	Second		First	Second	Correlation	Sig(p)	‘t-score
1	General examination	50	2.78		3.44	0.708	0.760	0.638	0.00*	7.452
2	Cardiovascular	50	2.66		3.24	0.658	0.716	0.740	0.00*	8.226
3	Respiratory	50	2.96		3.50	0.781	0.735	0.675	0.00*	6.228
4	Abdomen	50	2.92		3.54	0.778	0.813	0.618	0.00*	6.293
5	CentralNervous system	50	2.72		3.30	0.708	0.735	0.560	0.00*	5.429

*p<0.001 is statistically significant.

The perception of the students regarding their competency level in documentation of physical examination is significantly higher in the second session (p < 0.001).

Table-2: Analysis of first and second session marks

S. No.	Component	N	Mean marks			SD		Correlation	Sig(p)	‘t-score
S. No.	Component	N	First	Second		First	Second	Correlation	Sig(p)	‘t-score
1	General examination	50	4.54		5.66	1.249	0.872	0.547	0.00*	7.456
2	Cardiovascular	50	4.38		5.64	1.028	0.942	0.565	0.00*	9.667
3	Respiratory	50	4.80		5.72	1.010	1.144	0.675	0.00*	7.418
4	Abdomen	50	4.40		5.86	1.030	1.178	0.568	0.00*	9.981
5	Central Nervous system	50	4.22		5.18	0.932	0.962	0.410	0.003*	6.595

*p<0.001 is statistically significant

In the first session, the areas with competency gap in documentation of pediatric physical examination skills was related to the central nervous system (4.22+0.932) followed by cardiovascular system (4.38+1.028). It is worth noting that the areas of concern at the end of second session was central nervous system (5.18+ 0.962) followed by cardiovascular system (5.64+0.942). At the end of the first session proficiency related to documentation of respiratory system (4.80+1.010) and general examination (4.54+1.249) was performed better than the other components of pediatric physical examination. In the post feedback session, documentation of abdomen (5.86+1.178) and respiratory system (5.72+1.144) was performed better than the other components. Analysis of the first session results showed that the minimum and maximum marks scored was 15 out of 50 and 31 out of 50 marks respectively. The minimum and maximum marks scored at the end of second session was 20 and 38 respectively. There was significant difference between the test scores in the first and second session following feedback p (<0.001).

Discussion

This study shows that FA as an educational tool has improved the documentation skills of undergraduate medical students by providing an opportunity to identify their concern areas, receive feedback and improve their performance. It provides immediate feedback and enriches learning experience for students’ to identify their strengths and weaknesses [21]. To the best of our knowledge, this is the first study in Indian literature to diagnostically and formatively assess students’ documentation skills during the tenure of pediatric clinical postings. It was not linked to any particular patient scenario. Hence the complete repertoire of documentation of physical examination was assessed which is not compromised by the case-specificity. This research satisfied a dual purpose: one to assess the students’ competency and on the other, assess the effect of formative evaluation on writing skills.

During the clinical pediatric posting, students’ are required to perform a complete history, physical examination and document the findings so that they get to know the normal and abnormal findings. It has been shown that students’ perform a more extensive physical examination during assessment than they actually do during their internship or in medical practice [22]. Lacunae in the acquired skills was analyzed and rectified. The effect of remedial efforts was assessed in the next exam.

The deficiencies noted in the first session are inherently present at the start of the current posting. The knowledge, clinical skills acquired during the five previous semesters of medical curriculum along with the students’ capacity to recall of information, integration and application in the first half of the current posting were assessed from the pre feedback session marks [23]. Busy workloads and hectic class schedule may result in poor physical examination and documentation. Inadequate physical examination skill proficiency and their documentation achieved during earlier years without feedback gave little opportunity to improve these skills [24]. During the feedback sessions, specific feedback based on the students’ competency gaps was delivered by the facilitator to correct their deficiencies and enrich learning experience by providing them corrected records with comments on the mistakes done by them [25].

The participants involvedin this research felt they were not provided relevant feedback during the tenure of previous clinical postings as ward leaving exams are conducted at the end of posting and only grades were given to them. The written feedback with meaningful constructive data to the students’ provided immediate feedback while it is still fresh in the students’ mind. Immediate feedback had a pacifying effect on the teachers and students’ stress levels. All the students’ opined that formative evaluation was an acceptable method with a positive educational impact and appreciated the feedback provided by the faculty members. It is worthwhile to note that there was a significant increase in the students’ confidence level on the performed skills. Out of 50 students, 38(76%) were satisfied with the teaching learning experience in the first session whereas 50(100%) students had a positive opinion in the second session. In the first session 40(80%) students opined that the time allotted to document physical examination was adequate whereas 49(98%) felt the same in the second session. There was astatistically significant difference between the positive perception levels based on the questionnaire between the first and second session implying that feedback had a beneficial effect on the students’ (p< 0.001).

The formative assessment method is diagnostic as it is performed at the first half of study duration to identify the students’ level of knowledge, skills and attitudes. The information collected in the process is significant and useful to the facilitator. It allowed the teacher to make necessary adjustments to the content, teaching methodology and deliver relevant feedback. In our study the competency gap in the prefeedback and post feedback sessions was regarding the documentation of central nervous system [26]. The area of concern was the same in both the session. Anecdotal comments from undergraduate students suggest that senior students transmit a negative view regarding the complexities of these systems resulting in decreased motivation and poor preparation. It could also be due to thefact that concepts in central nervous system are voluminous, necessitates greater application in clinical setting and proper documentation requires extra time compared to other systems. The proficiency in respiratory system was demonstrated better in both the sessions.

In the post feedback session students achieve the objectives at the end of the posting so that they possess the examination skills, attitudes and interconnected ideas to develop clinical competencies based on relevant feedback. It provides a process for the students to integrate information, identify the concern areas, empower them to overcome the identified deficiencies and improve their skills and knowledge [27]. All the students’ scored more marks in the post feedback session in all the domains of physical examination implying that relevant feedback had a significant positive effect on their performance. There was a statistically significant difference between the test scores in the first and second session (P<0.001).

The duration, frequency and method of assessment is a challenge for facilitators to produce an Ideal Indian Medical Graduate (IMG). All assessment formats have both advantages and disadvantages and there is no individual standard assessment tool that results in a perfect assessment. The participants opined that relevant feedback during the tenure of posting enhanced their learning and performance [28-30]. The academic activities, feedback, evaluation of written case records prove that formative evaluation provide necessary avenues to develop appropriate vocabulary and skills to document pediatric physical examination [31]. Based on the results of this study, it is proposed to conduct formative type assessment during the course of clinical posting than the traditional way of conducting a single ward leaving exams at the last end of posting. The mastery of the physical exam repertoire assesses the students’ readiness to make the transition from preclinical years to clinical years of training [32]. The importance of FA may feedback medical teaching and strengthen primary training in medical education.

The main challenges during the research were busy schedule, heavy workload and the availability of clinical faculty. The study observation was not done with real patient encounters. For ethical concerns, privacy and logistical reasons we chose a setting with standardized patients. Direct observation of physical examination was not practiced and only the written records were verified. It is possible that some inspections performed by the participants were missed by the researchers. Alternatively, mastery of the documentation of physical examination skill may not predict a students’ ability toperform the same in real patient encounters. As they progress in their training, students will indeed select the items most relevant to a case. It is a single centre study done with a small group of students’ in the same semester. Further studies may be done with different batches of students’ and results compared. The challenging task would be to maintain formative assessment of documentation of physical examination and appropriate feedback as part of daily practice in clinical teaching-learning experience.

Conclusion

Our study shows that the introduction of formative assessment has a significant impact on the perception and scores among undergraduate medical students with a beneficial effect on students’ learning to document physical examination skills. It is a process of assessment with a powerful effect on language learning outcomes and skills improvement by enhancing the learner’s competence and confidence in a clinical setting. FA helps to assess the student’s strengths, weaknesses and provide them an opportunity towards targeted learning. Therefore, formative evaluation with appropriate feedback during the tenure of clinical posting provides a beneficial effect on the students’ education, proficiency and practical skills.

Contributions of Authors: The topic selection, core methodology and literature search were done by the first author. Contact classes, evaluation of case records, feedback sessions,manuscriptpreparationwas done by first and second authors. Designing of students’ feedback questionnaire, preparation of checklist,preparation of standardized patient, conduct of examination, dataanalysis and interpretation was done by the second and third and fourth authors. All authors read and approved the final manuscript.

Recommendations
1)  Formative assessment should be incorporated during the tenure of clinical postings after sensitization of the faculty members.
2)  Students’ and teachers should be made aware of the expected student’ performance during the posting.
3)  Students’ should be regularly monitored during the development of proficiency of examination skills through skill improvement programs.

Acknowledgements: This project was done as a part of Fellowship in Medical Education (FIME) conducted by Medical Council of India under Nodal center for faculty development - Sri Ramachandra Medical University, Chennai. The authors extend their gratitude to the Dean and Medical superintendent of KFMSR. The authors thank the students involved in the study for their cooperation and support.

Funding: Nil, Conflict of interest: None initiated.
Permission from IRB: Yes

References

1. Li JT. Assessment of basic physical examination skills of internal medicine residents. Acad Med.1994 Apr;69(4):296-9.

2. Fletcher RH, Fletcher SW. Has medicine outgrown physical diagnosis? Ann Intern Med. 1992 Nov 1;117(9):786-7. [PubMed]

3. Lembo NJ, Dell'Italia LJ, Crawford MH, O'Rourke RA. Bedside diagnosis of systolic murmurs. N Engl J Med. 1988 Jun 16;318(24):1572-8.

4. Shaver JA. Cardiac auscultation: a cost-effectivediagnosticskill. CurrProblCardiol. 1995 Jul;20(7):441-530. [PubMed]

5. Reilly BM. Physical examination in the care of medicalinpatients: an observational study. Lancet.2003Oct4;362(9390):1100-5. [PubMed]

6. Sheehan D, Wilkinson TJ, Billett S. Interns' participation and learning in clinical environments in a New Zealand hospital. Acad Med. 2005 Mar;80(3):302-8. [PubMed]

7. Williams KN, Ramani S, Fraser B, et al. Improving bedside teaching: findings from a focus group of study learners. Acad Med. 2008 Mar;83(3):257-64. doi: 10.1097/ACM.0b013e3181637f3e. [PubMed]

8. Crumlish CM, Yialamas MA, McMahon GT. Quantification of bedsideteaching by an academichospitalist group. J Hosp Med.2009May;4(5):304-7. doi: 10.1002/jhm.540. [PubMed]

9. Lippa LM, Boker J, Duke A, Amin A. A novel 3-year longitudinal pilot study of medical students’ acquisition and retention of screening eye examination skills. Ophthalmology. 2006 Jan;113(1):133-9.doi:10.1016/j.ophtha.2005.09.003.

10. Schoenfeld PS, Baker MD. Documentation in the pediatricemergency department: a review of resuscitationcases.Ann Emerg Med.1991 Jun;20(6):641-3. [PubMed]

11. Sharma S: A single-blinded, direct observational study of PGY-1 interns and PGY-2 residents in evaluating their history-taking and physical-examination skills. Permanente J. 2011 Fall;15(4): 23-29.

12. Cox JL, Zitner D, Courtney KD, MacDonald DL, Paterson G, Cochrane B, Mathers J, Merry H, Flowerdew G, Johnstone DE. Undocumented patient information: an impediment to quality of care. Am J Med. 2003 Feb 15;114(3):211-6.

13. Denny JC, Miller RA, Johnson KB, Spickard A 3rd. Development and evaluation of a clinicalnotesectionheaderterminology. AMIAAnnuSymp Proc.2008 Nov 6:156-60. [PubMed]

14. Tinsley JA. An educational intervention to improve residents’ inpatient charting. Acad Psychiatry. 2004;28(2):136-9. doi:10.1176/appi.ap.28.2.136.

15. Smith JJ, Bland SA, Mullett S. Temperature--the forgotten vital sign. AccidEmergNurs. 2005 Oct;13(4):247-50. Epub 2005 Sep 30. [PubMed]

16. Kanegaye JT, Cheng JC, McCaslin RI, et al. Improved documentation of wound care with a structured encounter form in the pediatric emergency department [Evaluation Studies]. AmbulPediatr. 2005 Jul-Aug;5(4):253-7. doi: 10.1367/A04-196R.1.

17. Dunlay SM, Alexander KP, Melloni C, et al. Medical records and quality of care in acute coronary syndromes: results from CRUSADE. Arch Intern Med. 2008 Aug 11;168(15):1692-8. doi: 10.1001/archinte.168.15.1692. [PubMed]

18. Chan-Yan C, Gillies JH, Ruedy J, Montaner JS, Marshall SA.Clinical skills of medical residents: a review of physical examination. CMAJ. 1988 Oct 1;139(7):629-32. [PubMed]

19. Ben-David MF. The role of assessment in expanding professional horizons. Med Teach. 2000;22(5):472-7. doi: 10.1080/01421590050110731. [PubMed]

20. Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet. 2001 Mar 24;357(9260):945-9. [PubMed]

21. Brazeau C, Boyd L, Crosson J. Changing an existing OSCE to a teaching tool: the making of a teaching OSCE. Acad Med. 2002 Sep;77(9):932. [PubMed]

22. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002 Jan 9;287(2):226-35. [PubMed]

23. Martínez-Gonzalez A, et al.:Diagnostic and formative assessment of competencies at the beginning of undergraduate medical internship. Gac Med Mex. 2017;153:4-12.

24. Sheehan D, Wilkinson TJ, Billett S. Interns' participation and learning in clinical environments in a New Zealand hospital. Acad Med. 2005 Mar;80(3):302-8. [PubMed]

25. Lane JL, Gottlieb RP: Structured clinical observations: a method to teach clinical skills with limited time and financial sources. Pediatrics. 2000 Apr;105(4 Pt 2):973-7. [PubMed]

26. Oliver CM, Hunter SA, Ikeda T, Galletly DC. Juniordoctorskill in the art of physical examination: a retrospective study of the medicaladmissionnoteoverfourdecades.BMJ Open. 2013 Apr 3;3(4). pii: e002257. doi: 10.1136/bmjopen-2012-002257. Print 2013.

27. Pusic MV, Kessler D, Szyld D, Kalet A, Pecaric M, Boutis K. Experiencecurves as an organizingframework for deliberatepractice in emergency medicine learning. AcadEmerg Med.2012Dec;19(12):1476-80. doi: 10.1111/acem.12043. Epub2012Dec11. [PubMed]

28. Jain V, Agrawal V, Biswas S. Use of formativeassessment as an educationaltool.J Ayub Med Coll Abbottabad. 2012 Jul-Dec;24(3-4):68-70. [PubMed]

29. Sangeeta A, Arun Kumar T, Someshwaran R, Rajeswari. Formative assessment on documentation of pediatric history taking skills by undergraduate medical students. J Pediatr Res. 2017 Aug;4(08):504-510.doi:10. 17511/ijpr.2017.08.02.

30. Naik TB, Amruthkishan KU, Swaroop Rani NB, Wadekar MD, Mane V, Biradar A. Students perception on different formats of written assessment in Medical Education, J Educational Res & Med Teach. 2015 Apr;3(1):8-12.

31. Liu Y. Preliminary study on application of formative assessment in college English writing class. Theory and Practice in Language Studies. 2013 Dec 1; 3 (12):2186-2195.doi:10.4304/tpls.3.12.2186-2195.

32. Yudkowsky R, Downing S, Klamen D, Valaski M, Eulenberg B, Popa M. Assessing the head-to-toe physical examination skills of medical students. Med Teach. 2004 Aug;26(5):415-9.doi:10.1080/01421590410001696452. [PubMed]

How to cite this article?

Arun Kumar T, Sangeeta A, Rashmi R, Jyothi S. Effect of formative assessment on documentation of pediatric physical examination by undergraduate medical students. Int J Pediatr Res. 2017;4(10):603-610.doi:10. 17511/ijpr.2017.10.04.