Clinical Science| Volume 207, ISSUE 6, P964-973, June 2014

Standardized methodological assessment of research presentations (SHARP): development of a new instrument

Published:January 06, 2014DOI:



      There are currently no validated guidelines to assess the quality of the content and the delivery style of scientific podium surgical presentations. We have developed a simple, short, and reliable instrument to objectively assess the overall quality of scientific podium presentations.


      A simple and efficient rating instrument was developed to assess the scientific content and presentation style/skills of the surgical residents' presentations from 1996 to 2013. Absolute and consistency agreement for the different sections of the instrument was determined and assessed overtime, by stage of the project and study design. Intraclass correlation coefficients with 95% confidence intervals were calculated and reported using a mixed-effects model.


      Inter-rater reliability for both absolute and consistency agreement was substantial for total score and for each of the 3 sections of the instrument. The absolute agreement for the overall rating of the presentations was .87 (.63 to .98) and .78 (.50 to .95), and the consistency agreement was .90 (.70 to .99) and .87 (.67 to .97) for the 2012 and 2013 institutional research presentations, respectively. Rater agreement for evaluating project stage and different study designs varied from .70 to .81 and was consistent over the years. The consistency agreement in rating of the presentation was .77 for both faculty and resident raters.


      Standardized methodological assessment of research presentations (SHARP) instrument rates the scientific quality of the research and style of the delivered presentation. It is highly reliable in scoring the quality of the all study designs regardless of their stage. We recommend that researchers focus on presenting the key concepts and significant elements of their evidence using visually simple slides in a professionally engaging manner for effective delivery of their research and better communication with the audience.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to The American Journal of Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Nicholson L.J.
        • Shieh L.Y.
        Teaching evidence-based medicine on a busy hospitalist service: residents rate a pilot curriculum.
        Acad Med. 2005; 80: 607-609
        • Waljee J.F.
        • Larson B.P.
        • Chang K.W.
        • et al.
        Developing the art of scientific presentation.
        J Hand Surg Am. 2012; 37: 2580-2588.e1–2
        • Goodhand J.R.
        • Giles C.L.
        • Wahed M.
        • et al.
        Poster presentations at medical conferences: an effective way of disseminating research?.
        Clin Med. 2011; 11: 138-141
        • Smith P.E.
        • Fuller G.
        • Dunstan F.
        Scoring posters at scientific meetings: first impressions count.
        J R Soc Med. 2004; 97: 340-341
        • Church J.
        • Balliet J.
        The quality of podium presentations at the American Society of Colon and Rectal Surgeons: does a decade make a difference?.
        Dis Colon Rectum. 2005; 48: 1569-1572
        • Knobloch K.
        • Yoon U.
        • Vogt P.M.
        Quality of reporting in poster versus oral presentations at the American Society of Plastic Surgeons 2008 conference in Chicago.
        Plast Reconstr Surg. 2010; 125: 219e-221e
        • Yoon U.
        • Knobloch K.
        Assessment of reporting quality of conference abstracts in sports injury prevention according to CONSORT and STROBE criteria and their subsequent publication rate as full papers.
        BMC Med Res Methodol. 2012; 12: 47
        • Knobloch K.
        • Yoon U.
        • Vogt P.M.
        Observational studies and STROBE reporting quality in plastic surgical conference abstracts.
        Plast Reconstr Surg. 2011; 128: 108e-109e
        • Knobloch K.
        • Yoon U.
        • Vogt P.M.
        Clinical trial quality and reporting quality in American Society of Plastic Surgeons and German conference abstracts: a transatlantic perspective.
        Plast Reconstr Surg. 2010; 126: 281e-282e
        • Mann E.
        • Meyer G.
        Reporting quality of conference abstracts on randomised controlled trials in gerontology and geriatrics: a cross-sectional investigation.
        Z Evid Fortbild Qual Gesundhwes. 2011; 105: 459-462
        • Poolman R.W.
        • Keijser L.C.
        • de Waal Malefijt M.C.
        • et al.
        Reviewer agreement in scoring 419 abstracts for scientific orthopedics meetings.
        Acta Orthop. 2007; 78: 278-284
        • Zerwic J.
        • Grandfield K.
        • Kavanaugh K.
        • et al.
        Tips for better visual elements in posters and podium presentations.
        Educ Health (Abingdon). 2010; 23: 267
        • Solagberu B.A.
        How to present a paper at a scientific meeting.
        West Afr J Med. 2004; 23: 260-263
        • Hardicre J.
        • Coad J.
        • Devitt P.
        Ten steps to successful conference presentations.
        Br J Nurs. 2007; 16: 402-404
        • Bushy A.
        A rating scale to evaluate research posters.
        Nurse Educ. 1991; 16: 11-15
        • Viera A.J.
        • Garrett J.M.
        Understanding interobserver agreement: the kappa statistic.
        Fam Med. 2005; 37: 360-363
        • Begg C.
        • Cho M.
        • Eastwood S.
        • et al.
        Improving the quality of reporting of randomized controlled trials. The CONSORT statement.
        JAMA. 1996; 276: 637-639
        • Detsky A.S.
        • Naylor C.D.
        • O'Rourke K.
        • et al.
        Incorporating variations in the quality of individual randomized trials into meta-analysis.
        J Clin Epidemiol. 1992; 45: 255-265
        • Slim K.
        • Nini E.
        • Forestier D.
        • et al.
        Methodological index for non-randomized studies (minors): development and validation of a new instrument.
        ANZ J Surg. 2003; 73: 712-716
        • Streiner D.L.
        • Norman G.R.
        Health Measurement Scales.
        Oxford University Press, Oxford2006: 126-152
        • Cook D.A.
        • Beckman T.J.
        Current concepts in validity and reliability for psychometric instruments: theory and application.
        Am J Med. 2006; 119: 166.e7-166.e16
        • Farrokhyar F.
        • Karanicolas P.J.
        • Thoma A.
        • et al.
        Randomized controlled trials of surgical interventions.
        Ann Surg. 2010; 251: 409-416
        • Poolman R.W.
        • Struijs P.A.A.
        • Krips R.
        • et al.
        Does a “Level I Evidence” rating imply high quality of reporting in orthopaedic randomised controlled trials?.
        BMC Med Res Methodol. 2006; 6: 44
        • Bhandari M.
        • Richards R.R.
        • Sprague S.
        • et al.
        The quality of reporting of randomized trials in the Journal of Bone and Joint Surgery from 1988 through 2000.
        J Bone Joint Surg Am. 2002; 84-A: 388-396
        • Farrokhyar F.
        • Chu R.
        • Whitlock R.
        • et al.
        A systematic review of the quality of publications reporting coronary artery bypass grafting trials.
        Can J Surg. 2007; 50: 266-277
        • Soares H.P.
        • Daniels S.
        • Kumar A.
        • et al.
        Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group.
        BMJ. 2004; 328: 22-24