Research Article| Volume 220, ISSUE 2, P328-333, August 2020

Download started.


Oral examinations in undergraduate medical education – What is the ‘value added’ to evaluation?

Published:January 02, 2020DOI:


      • Comments provided on oral examination score sheets reflect qualities examiners considered when evaluating performance.
      • Oral examinations test qualities not readily captured by other assessment tools.
      • Assessment of cognitive processing distinguishes strong from weak performance.



      Given the long tradition of oral examinations in surgical training, surgical clerkships continue to use oral examinations to evaluate medical students even though the value of oral examination at the post-graduate level has been questioned. The key issue in the context of undergraduate surgical training then is to understand value of the oral examination in assessment. The goal of this study is to clarify what oral examinations do, or appear to, test and how this complements other methods of assessment.


      The study is a retrospective, qualitative study of comments provided by examiners on the oral examination score sheets evaluating performance of students completing their core surgery clerkship at an academic medical center. Through immersion in and initial familiarization with the data we develop a scheme of codes for labeling the data for subsequent synthesis. Using these inductive codes, all comments were reviewed and analyzed to determine what qualities examiners detect, or naturally comment on, when administering and scoring the oral examinations.


      Thirteen substantive codes (Communication, Critical Thinking, Decisiveness, Demeanor, Differential Diagnosis, Focus, Knowledge, Management, Organization, Pace, Prompting, Thoroughness, and Work Up) and three valence codes (Negative, Neutral, and Positive) were developed and used to code the data. The most universal code was ‘Knowledge’, used by 43 (100%) of examiners; the most frequently used code was ‘Work Up’, applied to the comments 437 (21.1%) times. Overall, positive valence was attached to 1146 (55.2%) of codes and negative valence to 879 (42.3%) codes. The most discriminating codes in grading were ‘Demeanor’, ‘Focus’, and ‘Organization’.


      Oral examinations provide rich opportunity for testing qualities readily tested on other examinations but also many intangible qualities that are otherwise less well or not well tested. As such, the ‘value-added’ by oral examinations likely justifies their continued use in the evaluation of surgical trainees. The identification of testable qualities should aid in the development of a standardized scoring rubric, the use of which may aid in minimizing subjectivity and bias in what otherwise is a rich assessment tool.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to The American Journal of Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Griffin W.O.
        The American Board of Surgery in the 20th Century – Then and Now.
        ABS Office of the Secretary, Philadelphia, PA2004
        • Schwartz R.W.
        • Sloan D.A.
        • Griffin W.O.
        • Bland K.I.
        • Rhodes R.S.
        • Strodel W.E.
        The necessity, practicality, and feasibility of modern educational and evaluative methods for residency training: economic and governing body perspectives. The 1994 Association for Academic Surgery Panel on Education.
        Curr Surg. 1997; 54: 261-269
        • General Surgery Certifying Exam
        (Last accessed 12/19/2019)
        • Measurement Research Associates
        Standardized oral examinations.
        (Last accessed 9/7/2011)
        • Haq I.
        • Higham J.
        • Morris R.
        • Dacre J.
        Effect of ethnicity and gender on performance in undergraduate medical examinations.
        Med Educ. 2005; 39: 1126-1128
        • Fernandez A.
        • Wang F.
        • Braveman M.
        • Finkas L.K.
        • Hauer K.E.
        Impact of student ethnicity and primary childhood language of communication skill assessment in a clinical performance examination.
        J Gen Intern Med. 2007; 22: 1150-1160
        • Ullman Y.
        • Fodor L.
        • Meilick B.
        • Eshach H.
        • Ramon Y.
        • Meilick A.
        The oral board examination for plastic surgery: seeking a better way.
        Med Teach. 2006; 28: 360-364
        • Burchard K.W.
        • Rowland-Morin P.A.
        • Coe N.P.
        • Garb J.L.
        A surgery oral examination: interrater agreement and the influence of rater characteristics.
        Acad Med. 1995; 70: 1044-1046
        • Stahlfeld K.R.
        Is the American board of surgery certifying examination archaic?.
        Curr Surg. 2004; 61: 405
        • Ritchie J.
        • Spencer L.
        • O’Connor W.
        Carrying out qualitative analysis.
        in: Ritchie J. Lewis J. Qualitative Research Practice. SAGE Publications, London, UK2009: 219-262
        • Pope C.
        • Ziebland S.
        • Mays N.
        Qualitative research in Health Care: analysing qualitative data.
        BMJ. 2000; 320: 114-116
        • Silber C.G.
        • Nasca T.J.
        • Paskin D.L.
        • Eiger G.
        • Robeson M.
        • Veloski J.J.
        Do global rating forms enable program directors to assess the ACGME competencies?.
        Acad Med. 2004; 79: 549-556
        • Pernar L.I.
        • Peyre S.E.
        • Warren L.E.
        • et al.
        Mini-clinical evaluation exercise as a student assessment tool in a surgery clerkship: lessons learned from a 5-year experience.
        Surgery. 2011; 150: 272-277