Platinum Priority – Review – Education Editorial by Alberto Breda and Angelo Territo on pp. 1081–1082 of this issue| Volume 69, ISSUE 6, P1065-1080, June 01, 2016

A Systematic Review of Virtual Reality Simulators for Robot-assisted Surgery



      No single large published randomized controlled trial (RCT) has confirmed the efficacy of virtual simulators in the acquisition of skills to the standard required for safe clinical robotic surgery. This remains the main obstacle for the adoption of these virtual simulators in surgical residency curricula.


      To evaluate the level of evidence in published studies on the efficacy of training on virtual simulators for robotic surgery.

      Evidence acquisition

      In April 2015 a literature search was conducted on PubMed, Web of Science, Scopus, Cochrane Library, the Clinical Trials Database (US) and the Meta Register of Controlled Trials. All publications were scrutinized for relevance to the review and for assessment of the levels of evidence provided using the classification developed by the Oxford Centre for Evidence-Based Medicine.

      Evidence synthesis

      The publications included in the review consisted of one RCT and 28 cohort studies on validity, and seven RCTs and two cohort studies on skills transfer from virtual simulators to robot-assisted surgery. Simulators were rated good for realism (face validity) and for usefulness as a training tool (content validity). However, the studies included used various simulation training methodologies, limiting the assessment of construct validity. The review confirms the absence of any consensus on which tasks and metrics are the most effective for the da Vinci Skills Simulator and dV-Trainer, the most widely investigated systems. Although there is consensus for the RoSS simulator, this is based on only two studies on construct validity involving four exercises. One study on initial evaluation of an augmented reality module for partial nephrectomy using the dV-Trainer reported high correlation (r = 0.8) between in vivo porcine nephrectomy and a virtual renorrhaphy task according to the overall Global Evaluation Assessment of Robotic Surgery (GEARS) score. In one RCT on skills transfer, the experimental group outperformed the control group, with a significant difference in overall GEARS score (p = 0.012) during performance of urethrovesical anastomosis on an inanimate model. Only one study included assessment of a surgical procedure on real patients: subjects trained on a virtual simulator outperformed the control group following traditional training. However, besides the small numbers, this study was not randomized.


      There is an urgent need for a large, well-designed, preferably multicenter RCT to study the efficacy of virtual simulation for acquisition competence in and safe execution of clinical robotic-assisted surgery.

      Patient summary

      We reviewed the literature on virtual simulators for robot-assisted surgery. Validity studies used various simulation training methodologies. It is not clear which exercises and metrics are the most effective in distinguishing different levels of experience on the da Vinci robot. There is no reported evidence of skills transfer from simulation to clinical surgery on real patients.


      To read this article in full you will need to make a payment


      Subscribe to European Urology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Intuitive Surgical. Investor relations.

      2. Bernstein Leibhard LLP. da Vinci robot lawsuit information center.

      3. Commonwealth of Massachusetts Board of Registration in Medicine. Advisory on robot-assisted surgery.

        • Dawe S.R.
        • Windsor J.A.
        • Broeders J.A.
        • et al.
        A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy.
        Ann Surg. 2014; 259: 236-248
        • Goh A.C.
        • Goldfarb D.W.
        • Sander J.C.
        • et al.
        Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills.
        Urology. 2012; 187: 247-252
      4. Transparent Reporting of Systematic Reviews and Meta-analyses. The PRISMA statement.

      5. Centre for Reviews and Dissemination. Guidance for undertaking reviews in health care.

      6. Oxford Centre for Evidence-based Medicine. Levels of evidence 2011.

      7. The Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions.

        • Lendvay T.S.
        • Casale P.
        • Sweet R.
        Initial validation of a virtual-reality robotic simulator.
        J Robotic Surg. 2008; 2: 145-149
        • Kenney P.A.
        • Wszolek M.F.
        • Gould J.J.
        • et al.
        Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery.
        Urology. 2009; 73: 1288-1292
        • Sethi A.S.
        • Peine W.J.
        • Mohammadi Y.
        • et al.
        Validation of a novel virtual reality robotic simulator.
        J Endourol. 2009; 23: 503-508
        • Korets R.
        • Mues A.C.
        • Graversen J.A.
        • et al.
        Validating the use of the Mimic dV-trainer for robotic surgery skill acquisition among urology residents.
        Urology. 2011; 78: 1326-1330
        • Hung A.J.
        • Zehnder P.
        • Patil M.B.
        • et al.
        Face, content and construct validity of a novel robotic surgery simulator.
        Urology. 2011; 186: 1019-1024
        • Lee J.Y.
        • Mucksavage P.
        • Kerbl D.C.
        • et al.
        Validation study of a virtual reality robotic simulator—role as an assessment tool?.
        J Urol. 2012; 187: 998-1002
        • Liss M.A.
        • Abdelshehid C.
        • Quach S.
        • et al.
        Validation, correlation, and comparison of the da Vinci trainer™ and the da Vinci surgical skills simulator™ using the Mimic™ software for urologic robotic surgical education.
        J Endourol. 2012; 26: 1629-1634
        • Perrenot C.
        • Perez M.
        • Tran N.
        • et al.
        The virtual reality simulator dV-Trainer® is a valid assessment tool for robotic surgical skills.
        Surg Endosc. 2012; 26: 2587-2593
        • Egi H.
        • Hattori M.
        • Tokunaga M.
        • et al.
        Face, content and concurrent validity of the Mimic® dV-Trainer for robot-assisted endoscopic surgery: a prospective study.
        Eur Surg Res. 2013; 50: 292-300
        • Schreuder H.W.
        • Persson J.E.
        • Wolswijk R.G.
        • et al.
        Validation of a novel virtual reality simulator for robotic surgery.
        Sci World J. 2014; 2014: 507076
        • Kang S.G.
        • Cho S.
        • Kang S.H.
        • et al.
        The Tube 3 module designed for practicing vesicourethral anastomosis in a virtual reality robotic simulator: determination of face, content, and construct validity.
        Urology. 2014; 84: 345-350
        • Hung A.J.
        • Shah S.H.
        • Dalag L.
        • et al.
        Development and validation of a novel robotic procedure-specific simulation platform: partial nephrectomy.
        J Urol. 2015; 194: 520-526
        • Alzahrani T.
        • Haddad R.
        • Alkhayal A.
        • et al.
        Validation of the da Vinci surgical skill simulator across three surgical disciplines: a pilot study.
        Can Urol Assoc J. 2013; 7: E520-E529
        • Kelly D.C.
        • Margules A.C.
        • Kundavaram C.R.
        • et al.
        Face, content, and construct validation of the da Vinci skills simulator.
        Urology. 2012; 79: 1068-1072
        • Lyons C.
        • Goldfarb D.
        • Jones S.L.
        • et al.
        Which skills really matter? Proving face, content, and construct validity for a commercial robotic simulator.
        Surg Endosc. 2013; 27: 2020-2030
        • Ramos P.
        • Montez J.
        • Tripp A.
        • et al.
        Face, content, construct and concurrent validity of dry laboratory exercises for robotic training using a global assessment tool.
        BJU Int. 2014; 113: 836-842
        • Seixas-Mikelus S.A.
        • Kesavadas T.
        • Srimathveeravalli G.
        • et al.
        Face validation of a novel robotic surgical simulator.
        Urology. 2010; 76: 357-360
        • Chowriappa A.
        • Raza S.J.
        • Fazili A.
        • et al.
        Augmented-reality-based skills training for robot-assisted urethrovesical anastomosis: a multi-institutional randomised controlled trial.
        BJU Int. 2015; 115: 336-345
        • van der Meijden O.A.
        • Broeders I.A.
        • Schijven M.P.
        The SEP “robot”: a valid virtual reality robotic simulator for the da Vinci surgical system?.
        Surg Technol Int. 2010; 19: 51-58
        • Gavazzi A.
        • Bahsoun A.N.
        • Van Haute W.
        • et al.
        Face, content and construct validity of a virtual reality simulator for robotic surgery (SEP robot).
        Ann R Coll Surg Engl. 2011; 93: 152-156
        • Seixas-Mikelus S.A.
        • Stegemann A.P.
        • Kesavadas T.
        • et al.
        Content validation of a novel robotic surgical simulator.
        BJU Int. 2011; 107: 1130-1135
        • Finnegan K.T.
        • Meraney A.M.
        • Staff I.
        • et al.
        da Vinci skills simulator construct validation study: correlation of prior robotic experience with overall score and time score simulator performance.
        Urology. 2012; 80: 330-335
        • Hung A.J.
        • Jayaratna I.S.
        • Teruya K.
        • et al.
        Comparative assessment of three standardized robotic surgery training methods.
        BJU Int. 2013; 112: 864-871
        • Connolly M.
        • Seligman J.
        • Kastenmeier A.
        • et al.
        Validation of a virtual reality-based robotic surgical skills curriculum.
        Surg Endosc. 2014; 28: 1691-1694
        • Chowriappa A.J.
        • Shi Y.
        • Raza S.J.
        • et al.
        Development and validation of a composite scoring system for robot-assisted surgical training—the Robotic Skills Assessment Score.
        J Surg Res. 2013; 185: 561-569
        • Raza S.J.
        • Froghi S.
        • Chowriappa A.
        • et al.
        Construct validation of the key components of Fundamental Skills of Robotic Surgery (FSRS) curriculum—a multi-institution prospective study.
        J Surg Educ. 2014; 71: 316-324
        • Balasundaram I.
        • Aggarwal R.
        • Darzi A.
        Short-phase training on a virtual reality simulator improves technical performance in tele-robotic surgery.
        Int J Med Robot. 2008; 4: 139-145
        • Hung A.J.
        • Patil M.B.
        • Zehnder P.
        • et al.
        Concurrent and predictive validation of a novel robotic surgery simulator: a prospective, randomized study.
        J Urol. 2012; 187: 630-637
        • Moglia A.
        • Ferrari V.
        • Morelli L.
        • et al.
        Distribution of innate ability for surgery amongst medical students assessed by an advanced virtual reality surgical simulator.
        Surg Endosc. 2014; 28: 1830-1837
        • Lerner M.A.
        • Ayalew M.
        • Peine W.J.
        • et al.
        Does training on a virtual reality robotic simulator improve performance on the da Vinci surgical system?.
        J Endourol. 2010; 24: 467-472
        • Cho J.S.
        • Hahn K.Y.
        • Kwak J.M.
        • et al.
        Virtual reality training improves da Vinci performance: a prospective trial.
        J Laparoendosc Adv Surg Tech A. 2013; 23: 992-998
        • Whitehurst S.V.
        • Lockrow E.G.
        • Lendvay T.S.
        • et al.
        Comparison of two simulation systems to support robotic-assisted surgical training: a pilot study (swine model).
        J Minim Invasive Gynecol. 2015; 22: 483-488
        • Vaccaro C.M.
        • Crisp C.C.
        • Fellner A.N.
        • et al.
        Robotic virtual reality simulation plus standard robotic orientation versus standard robotic orientation alone: a randomized controlled trial.
        Female Pelvic Med Reconstr Surg. 2013; 19: 266-270
        • Culligan P.
        • Gurshumov E.
        • Lewis C.
        • et al.
        Predictive validity of a training protocol using a robotic surgery simulator.
        Female Pelvic Med Reconstr Surg. 2014; 20: 48-51
        • Kiely D.J.
        • Gotlieb W.H.
        • Lau S.
        • et al.
        A randomized controlled trial of a proficiency-based, virtual-reality robotic simulation curriculum to teach robotic suturing.
        Gynecol Oncol. 2014; 133: 193
        • Stegemann A.P.
        • Ahmed K.
        • Syed J.R.
        • et al.
        Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum.
        Urology. 2013; 81: 767-774
        • Gallagher A.G.
        • O'Sullivan G.C.
        Fundamentals of surgical simulation.
        Springer Verlag, London, UK2011
        • Maan Z.N.
        • Maan I.N.
        • Darzi A.W.
        • et al.
        Systematic review of predictors of surgical performance.
        Br J Surg. 2012; 99: 1610-1621
        • Gupta V.
        • Lantz A.G.
        • Alzharani T.
        • et al.
        Baseline urologic surgical skills among medical students: Differentiating trainees.
        Can Urol Assoc J. 2014; 8: 242-246
        • Stone R.J.
        • McCloy R.F.
        Virtual environment training systems for laparoscopic surgery; activities at the UK's Wolfson Centre for Minimally Invasive Therapy.
        J Med Virtual Reality. 1996; 1: 42-51
        • Seymour N.E.
        • Gallagher A.G.
        • Roman S.A.
        • et al.
        Virtual reality training improves operating room performance: results of a randomized, double-blinded study.
        Ann Surg. 2002; 236: 458-463
        • Hyltander A.
        • Liljegren E.
        • Rhodin P.H.
        • et al.
        The transfer of basic skills learned in a laparoscopic simulator to the operating room.
        Surg Endosc. 2002; 16: 1324-1328
        • Ahlberg G.
        • Enochsson L.
        • Gallagher A.G.
        • et al.
        Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies.
        Am J Surg. 2007; 193: 797-804
        • Baheti A.
        • Seshadri S.
        • Kumar A.
        • et al.
        RoSS: virtual reality robotic surgical simulator for the da Vinci surgical system. In: Proceedings of the 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.
        IEEE;. 2008; : 479-480
        • Lendvay T.S.
        • Casale P.
        • Sweet R.
        • et al.
        VR robotic surgery: randomized blinded study of the dV-Trainer robotic simulator.
        Stud Health Technol Inf. 2008; 132: 242-244
      8. Lawson Health Research Institute. Randomized controlled evaluation of robotic cardiac surgery training modalities.

        • Sperry S.M.
        • O’Malley Jr., B.W.
        • Weinstein G.S.
        The University of Pennsylvania curriculum for training otorhinolaryngology residents in transoral robotic surgery.
        J Otorhinolaryngol Relat Spec. 2014; 76: 342-352
        • Foell K.
        • Finelli A.
        • Yasufuku K.
        • et al.
        Robotic surgery basic skills training: evaluation of a pilot multidisciplinary simulation-based curriculum.
        Can Urol Assoc J. 2013; 7: 430-434
        • Fisher R.A.
        • Dasgupta P.
        • Mottrie A.
        • et al.
        An over-view of robot assisted surgery curricula and the status of their validation.
        Int J Surg. 2015; 13: 115-123
        • Ahmed K.
        • Khan R.
        • Mottrie A.
        • et al.
        Development of a standardised training curriculum for robotic surgery: a consensus statement from an international multidisciplinary group of experts.
        BJU Int. 2015; 116: 93-101
        • Volpe A.
        • Ahmed K.
        • Dasgupta P.
        • et al.
        Pilot validation study of the European Association of Urology robotic training curriculum.
        Eur Urol. 2015; 68: 292-299
        • Zhang N.
        • Sumer B.D.
        Transoral robotic surgery: simulation-based standardized training.
        JAMA Otolaryngol Head Neck Surg. 2013; 139: 1111-1117
        • Teishima J.
        • Hattori M.
        • Inoue S.
        • et al.
        Retention of robot-assisted surgical skills in urological surgeons acquired using Mimic dV-Trainer.
        Can UrolAssoc J. 2014; 8: E493-E497
        • Rehman S.
        • Raza S.J.
        • Stegemann A.P.
        • et al.
        Simulation-based robot-assisted surgical training: a health economic evaluation.
        Int J Surg. 2013; 11: 841-846
        • Roscoe S.N.
        Incremental transfer effectiveness.
        Hum Factors. 1971; 13: 561-567
        • Fletcher J.D.
        • Orlansky J.
        Recent studies on the cost-effectiveness of military training in TTCP countries. IDA Paper P-1896.
        Institute for Defense Analyses, Alexandria, VA1986