Robotic surgical education: a systematic review of strategies trainees and attendings can utilize to optimize skill development

Introduction

Robotic-assisted surgery has rapidly expanded across surgical specialties and procedures over the past two decades. Early adopters were primarily urology and gynecology (1-4), which is reflected in the research of how best to teach robotic surgery. Now, however, robotic surgery has spread to every other specialty in the abdomen and thorax including even pancreaticoduodenectomies within surgical oncology (5-8) and mitral valve replacement and internal mammary artery harvesting within cardiac surgery (9,10).

Over time, approaches to teaching robotic surgery have evolved. While the initial robotic surgeons needed to develop proficiency in robotic operations at the same time as educating their trainees, there has since been a growth in research on how to train residents and fellows in robotic surgery efficiently and safely. While some of this information has been synthesized into specialty specific reviews (11-17) or reviews focused on an individual modality (e.g., virtual reality simulation) (18,19), an interdisciplinary review that covers all aspects of training is lacking. This is needed as surgeons are continuing to develop strategies for optimal surgical education given the completing modalities that modern trainees must learn. Given this, we sought to review how best to train surgical residents and fellows how to perform robotic surgery in the thorax, abdomen and pelvis, focusing on curriculum development, types of simulation, and assessments. This systematic review on robotic surgery education was done in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) reporting checklist (available at https://asj.amegroups.com/article/view/10.21037/asj-24-14/rc) (20).

Methods

Search strategy

A search was performed on PubMed and Embase from 2015 to August 2024 on all relevant studies using the following keywords: (robot, robotic or Da Vinci) and (surgery or surgical) and (education or training) and (resident or residents or fellows or trainees). The search results were imported into Covidence for initial screening. Two reviewers screened all titles and abstracts based on inclusion and exclusion criteria described below. Whenever disagreement occurred, reviewers (W.M.O., L.D.J., V.P., C.H.) met and came to consensus. Any articles that were not excluded after this review had their full text assessed. The references of included articles were then reviewed to identify additional articles that met inclusion criteria (Figure 1).

Figure 1 Study selection flow diagram.

Study inclusion

All study designs were included from any setting if published during 2015 to August 2024 and if they met our inclusion criteria developed using the Problem, Intervention, Comparison, Outcome and Study design (PICOS) framework.

The study population of interest was surgical trainees being taught robotic surgery in the thorax, abdomen and pelvis. While the focus was on residents and fellows, studies comparing novices to experts or including students were also included. However, studies focusing on training current attendings or other operating room staff such as scrub techs were excluded. Additionally, if the study focused on teaching robotic surgery outside of the thorax, abdomen, or pelvis (e.g., orthopedics), these were excluded as were studies focused exclusively on other modalities such as endoscopy.

The interventions of interest were strategies for robotic skill development and assessment. Specifically, methods of teaching robotic surgery either through curriculum structure, simulation, or events were included. For simulation this includes the creation of virtual reality, wet lab and dry lab simulated exercises. Additionally, articles that evaluated a method of assessing robotic skill were included.

The outcomes of interest were measures of trainee performance of robotic surgery such as Global Evaluative Assessment of Robotic Skills (GEARS) score, time to complete a simulation, number of errors. For curriculum focused papers, the individual components of the curriculum were the outcomes of interest. Studies that focused only on changes in case volumes, costs, attending learning curves, or clinical outcomes, were excluded.

Study designs could be randomized control trials, retrospective or prospective studies. Perspectives without data, conference abstracts and studies for which full text was unavailable or not available in English were excluded. Review articles were excluded after citations were reviewed to identify additional articles that met inclusion criteria.

Study quality assessment

The Cochrane’s Risk of Bias 2 (RoB2) (21) and the Risk of Bias in Non-Randomized Studies of Interventions (ROBINS-I) (22) tools were used to assess the quality of included randomized controlled trials (RCTs) and nonrandomized studies, respectively. Additionally, the Critical Appraisal Skills Programme (CASP) for Qualitative Studies was employed for the assessment of qualitative studies (23), and the National Institute of Health Studies quality assessment tool was used for evaluating cross-sectional studies (24). The quality assessment of individual studies was assessed by two authors (V.P. and C.H.).

Data extraction and analysis

Data was compiled into a structured form that included the following basic parameters for each study: citation, first author last name, year published, geographic location of study, study period, specialty, trainee category and sample size, and educational focus. Further, details relevant for curriculum, simulation and assessment were also aggregated and shared in supplemental files. For studies that described or evaluated a curriculum, the components incorporated into the curriculum were noted, specifically the use of didactics, virtual reality simulation, dry or wet labs, bedside assisting and case volume requirement were recorded. For virtual reality simulations, dry labs and wet labs, the focus of the simulation (e.g., skill or procedure) and materials used (e.g., simulator platform, type of tissue) were recorded. Additionally, a qualitative summary of findings was recorded for each study. For assessments, the name of the assessment tool and level of automation were compiled then a qualitative summary of findings were recorded.

Descriptive statistics were used to describe the frequency of surgical specialty, curriculum components, simulation types, simulation focus, wet lab material type, and the types of assessment using Excel v16.891 (Redmond, WA, USA).

Results

A total of 2,493 potential studies were initially identified via the search strategy outlined above (Figure 1). After removing duplicates and screening titles and abstracts, full texts were sought for the remaining 512 articles. After review, an additional 335 studies were excluded and a total of 188 articles were included in the review. Table 1 shows the baseline characteristics of the included studies. The vast majority of studies came from the US (n=136), and the most common specialty was urology (n=56) followed by studies with multiple specialties (n=36) (Figure 2). The majority of articles discussed types of simulation (n=110) with 59 focused on virtual reality simulation, 50 on dry labs, 30 on wet labs, and 9 on training events such as so-called “boot camps”. Skills assessment was discussed in 81 of the articles and curriculums to teach robotic surgery was covered in 33 articles. Overall, most studies were classified as low quality with a high risk of bias. Observational studies, which made up the majority of the included papers, exhibited significant issues, particularly related to inadequate adjustment for confounders and the use of unvalidated measurement tools, such as novel surveys. In contrast, most RCTs, qualitative studies, and cross-sectional studies demonstrated reasonable study quality overall.

Figure 2 The number of articles from each specialty on training residents & fellows in robotic surgery. MIS, minimally invasive surgery.

Curriculums

Surveys of general surgery programs directors, such as that conducted by Tom et al., found that while 92% of the programs (105/114) reported that their residents participate in robotic surgeries, only 67% had implemented a formal robotic surgery training curriculum as of 2019 (125,181). In total, we identified 33 papers that focused on robotic curricula for training surgical residents and fellows (Table S1). While some older recommendations focused on learning in the operating room: first through observation, then bedside assistance, then working on the console; contemporary recommendations are more nuanced and utilize multimodality teaching by combining didactics, simulations, and operating room experiences. Some researchers shared in detail the analysis that went into their curriculum development (34,81). Baimas-George et al. explained their hepato-pancreato-biliary (HPB) fellowship curriculum development matrix and fully outlined their learning activities and assessment tools (34). However, Green et al. was the only study to explicitly use Kern’s 6-step model of curriculum development. They used a systematic process of design, implementation, and assessment of the general surgery robotic curriculum at University of California, San Francisco (UCSF) (81). One of Kern’s key steps missing from many curriculums was evaluation and feedback, with only seven additional studies having this curricular component (13,64,70,83,136,151,191).

Regardless of the curriculum specific to each institution, Winder et al. concluded that the majority of successful curricula had the following four key components: (I) didactic education; (II) simulation training; (III) bedside assistant experience; and (IV) operative training in cases (202). This is further supported by Stewart et al., who looked at 7 general surgery programs across the US and concluded that orientation with online didactic modules, required robotic simulation, and clinical practice were all common components to formal robotic surgery curricula (175). In our review, we found 24 studies with proposed curricula that included at least 3 of of these 4 components (Figure 3) (13,33,34,77,81,108,130,131,146,148,153,154,157,158,161,170,175,178,191,202). Multiple articles also called for a need to standardize the robotic surgery training curriculum, mostly in general surgery residency training programs (70,186), obstetrics and gynecology (190), and fellowship training programs such as surgical oncology (8). However, some of these specialties already have specific proposed curricula, specifically obstetrics and gynecology, surgical oncology (7), and transplant surgery (64).

Figure 3 Articles with different robotic surgery curricular components.

Virtual reality simulation

In this review, 110 articles studied the use of simulation as part of robotic surgery training (Table S2). This included 59 articles on virtual reality simulations, 50 on dry labs, 30 on wet labs, and 9 on robotic training events (Figure 4). Virtual reality simulators allow trainees to practice independently and were the most commonly studied simulation method. A significant number of articles focused on the face, content, and construct validity the da Vinci Skills Simulator (6,28,29,46,50,70,73,75,83,86,103,106,137,144,146,179,187,200) and the Mimic Technologies’ da Vinci trainer (29,89, 160). However, there were a number of additional virtual reality platforms studied, namely the da Vinci Research Kit (127,142), SimNow modules (25,26,62,67,154), Medtronic Hugo RAS System (49), RobotiX (141,167,197,198), the Versius trainer (44), and the Robotic-assisted Surgical Simulator (RoSS) (57). The majority of studies focused on multiple skill-based exercises (e.g., energy pedals) (29,41,44,46,49,50,67,68,70,73,75,106,113,126,127,133,135,137,144,146,154,166,183,200). Stegemann et al. curated 16 tasks to be part of a Fundamental Skills of Robotic Surgery (FSRS) virtual reality curriculum (192,207), which was then used in subsequent studies (57). However, a study by Gurung et al. found that simulation exercise order may affect skill acquisition (83). They found that having novices start with the hardest version of an exercise first (e.g., Suture Sponge 3), learners were able to more rapidly obtain proficiency. In contrast to skill-based simulations, other studies focused on virtual reality simulations of specific procedures such as hysterectomy (39,61,185) or prostatectomy (28,30,42,57,62,104,141,143,160). Proficiency in a simulation was defined as an overall score of at least 80% (52,179) or 90% (9,29,73,83,93,106) on an exercise.

Figure 4 Types of simulations used to teach residents and fellows robotic surgery.

Across surgical specialties as varied as urology and cardiac surgery (9), there is advocacy for the inclusion of virtual reality simulation in robotic surgery education. This is in part driven by the large number of studies which found that incorporation of simulation modules into training curriculums was associated with an improvement in practice assessment scores (5,6,25,75,106,127,154,161). Nonetheless, other researchers studied how simulation impacted performance in other modalities (9). In a randomized trial by Valdis et al., the cardiac surgery trainees randomized to complete a 9-exercise virtual reality series were faster than the control group for internal thoracic artery harvest and mitral annuloplasty with higher intraoperative scores (9). Others have looked at the role of simulation as a warm up before operating, though neither study by Chen et al., nor by Berges et al., showed a benefit (39,54).

Researchers have studied multiple approaches to optimize virtual reality simulator training, such as by incorporating didactics and feedback. Chowriappa et al. created the Hands-on Surgical Training (HoST), an augmented reality platform that combines real surgical procedures and didactic education (e.g., anatomy illustrations and audio explanations) into a virtual reality platform finding this improved trainee performance on a dry lab (57). Similarly, studies have also evaluated the role of having a teacher while using the simulator. In a study of urology residents, Lee and Lee found that residents who received feedback from an expert surgeon rather than simply reviewing their own metrics had larger improvements in performance (42,50,113,185). Other researchers have also found that incorporating a teacher to simulation use results in higher performance gains and satisfaction amongst trainees (42,50,110,113,185). Additionally, Kun et al. found that sharing recordings of simulation performance with trainees led to improved performance, thus suggesting that feedback even when not in real time increased the utility of simulation (109).

While trainees and attendings have both highlighted the importance of virtual reality simulator experience before active control time in the operating room, there is also concern that trainees are not utilizing simulation enough with one study finding that 45% of general surgery residents reported never using the simulator (145,159). Given this, gamification has been explored as a way to increase simulator use, finding increased use and performance when competing individually (135) or as a team (133). Other researchers have looked at training frequency (30,46,75,102,104,200). Wiener et al. showed that approximately 10 hours dedicated to simulation tasks should be enough to achieve proficiency in a given robotic training curriculum (200). Liu et al. studied learning decay after initially reaching proficiency, finding that those with a 3-month break nearly maintained skills while a 6-month break resulted significant worsening of performance (120).

Dry and wet lab exercises

In addition to virtual reality simulations, residents and fellows are also taught with dry and wet labs. For dry labs (Table S3), some still focused on skill acquisition through exercises like suturing on foam (48,74). However, most dry lab exercises focused on procedural models (57,76,79,85,89,99,102,111,115,116,123,124,128,132,164,167,169,171,176,180,182,194,203). The materials used for these models varied from three-dimensional (3D)-printed mitral valves (10) and renal models (132,167), to silicone models for transabdominal preperitoneal inguinal hernia repair (76) and pancreaticojejunostomy (8,116), to more accessible models like a pelvic lymphadenectomy model made from rubber tubing, wire, cotton balls, plastic wrap, and gelatin solution (102).

Wet labs were the least common given the resource intensive nature of the exercises and were often implemented as part of an event like a specialty association sponsored bootcamp (45,91,112,138,162,163,165). Models used tissues like ex vivo porcine organs (56,72,112,138,162,163), anesthetized pig models (37,96,134,165), and human cadavers (40,45,91) (Table S4). Nonetheless, a wide variety of procedures were taught via wet labs, including partial nephrectomies (56,63), hiatal hernia repair (84,163), hemicolectomy (59,91,162), lobectomy (112,138), cholecystectomy, and Heller myotomy (162). Some wet labs focused on key steps of procedures such as vesicourethral anastomosis (66,150), intestinal anastomosis (193), or vaginal cuff closure (72). Trainees reported preferring wet labs when asked to compare them to virtual reality simulation practice (40,72); however, a study that captured biometric data of cognitive mental workload found a similar increase for trainees for both a virtual reality and dry lab simulation of a vesicourethral anastomosis (62). This similarity in workload suggests that trainees are equally challenged by both simulations.

Assessment of skills

Skill assessment was studied in 81 of the included robotic surgery education articles, with approaches varying between subjective manual measures (e.g., questionnaires) or automatic objective performance indicators (e.g., console time) (Table S5). Figure 5 summarizes the types and frequency of assessment methods. For objective performance indicators, some studies focused on data that is readily available to surgeons and trainees, e.g., through Da Vinci’s Intuitive application. Specifically, active control time, the time trainees are in control of the robot, was used as a way to assess performance for several studies (58,64,71,152,195,196), with fewer using number of handoffs (64,152). Quinn et al. validated these reported measures by comparing them to those recorded by research personnel during inguinal hernia repairs (152). Others use kinematic data recorded by robotic systems to assess performance. Lazar et al. measured 20 different objective performance indicators such as idle time and wrist-angle distances, during a perfused lobectomy simulation (112). They found that trainees who experienced bleeding during the first procedural step—dividing the superior pulmonary vein—differed in metrics such as idle time, total instrument distance, and wrist articulation (112). There were some other studies that used kinematic data to evaluate performance during procedure focused dry labs (55,118), such as hepaticojejunostomy (123). Nonetheless, the majority of studies using these metrics were virtual reality simulations (25,141,173,185,194). Other objective measured included studies by Soangra et al., which linked time to complete procedure to electromyography (EMG) data to predict experience (174), Cowan et al. compared performance and biometric data of cognitive mental workload (62), and Shafiei et al. [2024] used electroencephalogram (EEG) and eye tracking data to predict performance (169).

Figure 5 Types of assessments used to assess trainee robotic performance. GEARS, Global Evaluative Assessment of Robotic Skills; R-OSATS, Robotic-Objective Structured Assessment of Technical Skills; RACE, Robotic Anastomosis Competence Evaluation; RO-SCORE, Robotic Ottawa Surgical Competency Evaluation.

Nonetheless, the majority of articles (64 of 81) included assessments that required input from expert surgeons, either through video review or with questionnaires at the end of cases. Goh et al. developed the GEARS assessment—a Likert scale questionnaire completed by an expert surgeon while reviewing an operative video that assesses six domains: autonomy, bimanual dexterity, depth perception, efficiency, force sensitivity, and robotic control. This assessment was used by multiple other studies (61,66,69,82,88-90,96,103,106,110,111,132,149,164,168,176,178,180,187,191,193,199,203), most of which implemented expert surgeons reviewing blinded videos. However, Brown & Kuchenbecker utilized a smart task board data collection system to automate the GEARS score for a dry lab exercise (Peg transfer) (48). They found that sharing the GEARS score did not accelerate skill acquisition but did improve self-awareness of performance (48). Multiple studies compared GEARS scores from experts to crowdsourced scores [Crowd Sourced Assessment of Technical Skill (C-SATS)], with most finding them to be consistent in their assessment of dry lab and operative cases with faster turnaround (38,88,149,199). GEARS was also used as the standard to validate the scores generated by virtual reality simulations and to compare machine learning models using EEG and eye tracking data (168).

Other subjective scoring systems have also been developed to assess robotic score: binary metric checklist (66), Robotic modification of Ottawa Surgical Competency Evaluation (RO-SCORE) (64,70,84), Robotic-Objective Structured Assessment of Technical Skills (R-OSATS) (29,54,106,114,124,136,137,147,157,172), Assessment of Robotic Console Skills (ARCS) (208), Robotic Anastomosis Competence Evaluation (RACE) (89,101,169,203), and Robotic Skills Assessment (RSA) Score; though others applied existing measures that were not robotic specific (84,85). Some studies also incorporated procedural steps into their assessment of performance (13,35,53,64,71,92,112,121,123,132,138,185), while others used procedural steps as a standalone approach to the skill assessment (31,150). Baldea et al. implemented a system for logging common robotic urology procedures in which trainees specify which steps they completed and then receive feedback from attendings in four domains: bimanual dexterity, instrument handling, time and motion, and respect for tissue, as well as qualitative comments (35). Trainees then receive monthly summaries showing which portions of cases they are performing. Two other interesting studies on feedback studied open-ended feedback. Wong et al. conducted a qualitative analysis of verbal feedback given to trainees while in active control during robotic cases and identified six types of feedback (204). Carter et al. had residents in the intervention arm of an RCT upload videos of their virtual reality simulation, and receive open-ended feedback from their peers before performing the simulation two more consecutive times (51). The peer feedback resulted in faster improvement than those without feedback.

Other training considerations

In addition to the studies of curriculums, simulation and assessment, there were a number of articles on other topics to facilitate trainee robotic skill acquisition. In multiple surveys, both trainees and attendings have reported the importance of dual consoles (78,159,170,184,206), with attendings noting it helps address the anxiety associated with fully giving trainee control of robot as well as facilitating coaching (184). The impact of dual consoles was quantified by Leon et al., who compared steps performed and active control time for cases using dual vs. single consoles finding that fellows completed more steps and operated for longer when using dual consoles (117). Others studies looked at the ability of a trainee to utilize a third arm for collaborative surgery with an attending (87). However, a study comparing dual to single console cases found that dual console cases did not result in differences in trainee autonomy or operative time nor faculty stress or coaching quality (32).

A number of studies evaluated attending teaching approaches and coaching styles (52,60,65,80,98,122). Researchers like Clanahan, Awad and Dimou found that after distributing case guides with pictures and narrated operative videos, trainee active control time increased for each trainee level (60,105). Lovegrove et al. recommend that trainees learn procedural steps in order of difficulty rather than chronological order (122). In the PROVESA multicenter RCT, De Groote et al. found that a proficiency-based progression approach led to increased trainee proficiency and fewer errors (65,66). This approach required passing e-learning modules that reviewed operative metrics, defined steps, and pointed out potential errors. Then during skills practice, two learners were responsible for recording metric-based feedback for co-learner. One study described the challenge of not being able to demonstrate steps visually because only one user can control the robot at a time, thereby making the teacher completely reliant on verbal communication to describe the desired conduct of an operation (80).

Additional factors found to improve operative experience and performance were having a dedicated bedside assistant in the operating room and educating trainees on ergonomics. Jones et al. found that a dedicated physician assistant at bedside increased the proportion of thoracic trainee active control time from 28.0% of a case to 77.1% of a case (100). Regarding ergonomics, Van’t Hullenaar et al. found that giving trainees ergonomics instructions improved ergonomics and improved efficiency of motion on clutch-oriented exercises (189).

Discussion

This systematic review has highlighted the components important to a robotic surgery curriculum, the role and impact of simulation, and the ways in which robotic surgery performance can be assessed. Robotic curriculums are a combination of didactics, simulation, bed side assistance, observation of surgery, and active time on the console in the operating room. There is extensive data showing the validity of simulation, though there is still opportunity in determining how best to ensure trainees are getting this practice and the needed frequency. Further, while many use the simulation scores as a marker of proficiency or readiness for the operating room, evaluating performance in the operating room itself has diverging approaches with some researchers focusing on video review and manual assessments while others are working to develop automated assessments through data recorded by robotic systems.

As seen in our review of the literature, there is a strong push for formalization of a standardized robotic-assisted surgery curriculum (8,70,131,148,170,175,181,186,190) with support noting that it is not only important to incorporate didactics, but also including simulation training, bedside experience, as well as operating in cases (175,202). While most articles did not dive deeply into the didactic component, the proficiency-based progression highlighted by De Groote et al. defined procedural steps and highlighted potential errors with trainees being required to show proficiency before progression (65,66). These findings point to the need to educate trainees on not just the technical logistics of using the robotic platform but also procedure-specific fundamentals. However, even with technical skill development, there are ways this development can be accelerated by educators. Several studies found that trainees receiving feedback during virtual reality simulation were found to lead to faster proficiency (42,50,110,113,185). However, even if an educator is not available, if recordings are shared with trainees to review (109) or trainees are given a platform to provide each other feedback (51), these interventions have also been found to improve performance.

The need to incorporate virtual reality simulation exercises into a formal curriculum was emphasized and validated in multiple studies (5,6,25,75,106,127,154,161,179,183). While many use a score cut off of 80–90% to be considered proficient (9,29,52,73,93,106,179), it is unclear what role, if any, simulation plays after this has been achieved and learners are operating in the operating room. While studies do not show utility of warm up (39), there have been documented skill degradation if a trainee does not use their robotic skills, which may occur as early as a hiatus of 4 weeks to 3 months (120). Given this, researchers have studied ways to further emphasize this independent practice, with some finding benefits when gamification is used (133,135). In addition to educators gamifying simulator use, robotic platforms could further build gamification into their platforms by adding features like high score and ranking information on conclusion pages.

Dry labs and wet labs always have facilitators, but studies have also shown that trainees have more performance gains and increased satisfaction when virtual reality simulations are mentored with active feedback (42,50,110,113,185). One potential way to bridge this gap asynchronously with less time required from expert surgeons is to implement video review for simulations. One study found that trainees sharing simulation recordings with one another anonymously and then giving and receiving feedback improved performance (109). This may be because providing feedback to others required trainees to critically reflect to identify ways another can be more efficient or precise and thus likely thinking about their own movements more. Most studies of dry and wet labs are focused on procedural simulation, sometimes focusing on individual steps [e.g., vesicourethral (37,66,150,171) or pancreatojejunostomy (8) anastomosis]. However, these simulations can be more resource intensive and thus many of the studies on the topic highlighted the incorporation of dry and wet lab simulations into events such as bootcamps (45,91,112,138,162,163,165) sponsored by specialty associations such as Southeast Section of the American Urologic Association (134,165) or Thoracic Surgery Directors Association (112,138).

Assessment of trainee skill is a key part of robotic education that was not highlighted in curriculum development with diverging approaches highlighted by researchers. With the volume of data generated by robotic surgery, there has been a growth in objective performance indicators. Virtual reality simulators provide scores to help learners understand their performance (173), which have been validated by GEARS assessments. Then during operative cases with Da Vinci, surgeons are provided information on active control time and handoffs that can be reviewed by trainees and teachers, with the benefit of clarifying operative autonomy received and given (58,64,152,195,196). Now there is also growing study to evaluate the other kinematic data recorded by robotic systems (e.g., idle time, wrist articulation) to estimate expertise (112,118) and outcomes (138). Nonetheless, while researchers have shown that metrics generated by robotic platforms are consistent with expert assessments (e.g., GEARS & R-OSATS), when compared side by side, it was manual review of skills that predicted independence (64). Assessments like GEARS may thus fill a gap that is not yet replaced by automated data. GEARS was certainly the most common assessment tool, used in 35 of 81 studies on assessment. Nonetheless, questionnaires require attendings to complete forms at the end of operations or require programs to have expert surgeons review videos and provide feedback. Thus, some researchers have shown potential efficiencies with crowdsourcing. Multiple researchers found that using the Amazon Turk service could result in GEARS scores consistent with expert assessments and were completed in 1–5 days (88,149,199). Similarly, peer feedback was also shown to lead to accelerated skill acquisition (51,65). One study showed a hybrid model in which trainees reflect on which steps they completed and then may also receive categorical or open ended feedback from attendings (35).

Conclusions

In conclusion, curricula to teach robotic surgery to residents and fellows should include a combination of didactics, simulations, and bedside assisting. While dry and wet labs are recommended by many, these are less common given the need to have more materials available and facilitators, which may be why they were more commonly highlighted as part of society bootcamps. For virtual reality simulations, there is general agreement that they play a critical role in introducing learners to robotic surgery though the utility after proficiency scores have been met is less well understood. Lastly, there continues to be differing research on the best way to assess robotic skill. While the plethora of kinematic data is being used to delineate skill, this is not yet widely available, and some studies still point to the superiority of manual assessments.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://asj.amegroups.com/article/view/10.21037/asj-24-14/rc

Peer Review File: Available at https://asj.amegroups.com/article/view/10.21037/asj-24-14/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://asj.amegroups.com/article/view/10.21037/asj-24-14/coif). B.W. serves as an unpaid editorial board member of AME Surgical Journal from April 2023 to March 2025. W.M.O. is supported by the AAS/AASF Clinical Outcomes/Health Services Trainee Research Award (Program Award Number 000541250) and the Department of Veterans Affairs, Veterans Health Administration, Office of Academic Affiliations VA Quality Scholars Advanced Fellowship Program (Award Number 3Q022019C). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Lerner MA, Ayalew M, Peine WJ, et al. Does training on a virtual reality robotic simulator improve performance on the da Vinci surgical system? J Endourol 2010;24:467-72. [Crossref] [PubMed]
2. Rashid HH, Leung YY, Rashid MJ, et al. Robotic surgical education: a systematic approach to training urology residents to perform robotic-assisted laparoscopic radical prostatectomy. Urology 2006;68:75-9. [Crossref] [PubMed]
3. Sethi AS, Peine WJ, Mohammadi Y, et al. Validation of a novel virtual reality robotic simulator. J Endourol 2009;23:503-8. [Crossref] [PubMed]
4. Geller EJ, Schuler KM, Boggess JF. Robotic surgical training program in gynecology: how to train residents and fellows. J Minim Invasive Gynecol 2011;18:224-9. [Crossref] [PubMed]
5. Ahmad SB, Rice M, Chang C, et al. Will It Play in Peoria? A Pilot Study of a Robotic Skills Curriculum for Surgical Oncology Fellows. Ann Surg Oncol 2021;28:6273-82. [Crossref] [PubMed]
6. Hogg ME, Tam V, Zenati M, et al. Mastery-Based Virtual Reality Robotic Simulation Curriculum: The First Step Toward Operative Robotic Proficiency. J Surg Educ 2017;74:477-85. [Crossref] [PubMed]
7. Mark Knab L, Zenati MS, Khodakov A, et al. Evolution of a Novel Robotic Training Curriculum in a Complex General Surgical Oncology Fellowship. Ann Surg Oncol 2018;25:3445-52. [Crossref] [PubMed]
8. Tam V, Zenati M, Novak S, et al. Robotic Pancreatoduodenectomy Biotissue Curriculum has Validity and Improves Technical Performance for Surgical Oncology Fellows. J Surg Educ 2017;74:1057-65. [Crossref] [PubMed]
9. Valdis M, Chu MW, Schlachta CM, et al. Validation of a Novel Virtual Reality Training Curriculum for Robotic Cardiac Surgery: A Randomized Trial. Innovations (Phila) 2015;10:383-8. [Crossref] [PubMed]
10. Premyodhin N, Mandair D, Ferng AS, et al. 3D printed mitral valve models: affordable simulation for robotic mitral valve repair. Interact Cardiovasc Thorac Surg 2018;26:71-6. [Crossref] [PubMed]
11. Hompe ED, Furlow PW, Schumacher LY. Starting and Developing a Robotic Thoracic Surgery Program. Thorac Surg Clin 2023;33:11-7. [Crossref] [PubMed]
12. Childs BS, Manganiello MD, Korets R. Novel Education and Simulation Tools in Urologic Training. Curr Urol Rep 2019;20:81. [Crossref] [PubMed]
13. Harji D, Aldajani N, Cauvin T, et al. Parallel, component training in robotic total mesorectal excision. J Robot Surg 2023;17:1049-55. [Crossref] [PubMed]
14. Linsky PL, Wei B. Training in robotic thoracic surgery. J Vis Surg 2018;4:1. [Crossref] [PubMed]
15. Mitzman B, Smith BK, Varghese TK Jr. Resident Training in Robotic Thoracic Surgery. Thorac Surg Clin 2023;33:25-32. [Crossref] [PubMed]
16. Alicuben ET, Wightman SC, Shemanski KA, et al. Training residents in robotic thoracic surgery. J Thorac Dis 2021;13:6169-78. [Crossref] [PubMed]
17. Abdallat M, Wee JO. Assessment of the educational approaches for robotic minimally invasive esophagectomy. Video-assist Thorac Surg 2023;8:14. [Crossref]
18. Badash I, Burtt K, Solorzano CA, et al. Innovations in surgery simulation: a review of past, current and future techniques. Ann Transl Med 2016;4:453. [Crossref] [PubMed]
19. Howard KK, Makki H, Novotny NM, et al. Value of robotic surgery simulation for training surgical residents and attendings: a systematic review protocol. BMJ Open 2022;12:e059439. [Crossref] [PubMed]
20. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Int J Surg 2021;88:105906. [Crossref] [PubMed]
21. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898. [Crossref] [PubMed]
22. Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919. [Crossref] [PubMed]
23. CAS P. CASP Checklists.: CASP UK-OAP Ltd. 2023. Available online: https://casp-uk.net/casp-tools-checklists/
24. NIH. Study Quality Assessment Tools 2021. Available online: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools
25. Abreu AA, Rail B, Farah E, et al. Baseline performance in a robotic virtual reality platform predicts rate of skill acquisition in a proficiency-based curriculum: a cohort study of surgical trainees. Surg Endosc 2023;37:8804-9. [Crossref] [PubMed]
26. Abreu AA, Farah E, Kannan A, et al. From the simulation lab to the operating room: simulation performance predicts intraoperative performance in robotic gastrojejunostomy. Surg Endosc 2024;38:5967-73. [Crossref] [PubMed]
27. Aghazadeh MA, Jayaratna IS, Hung AJ, et al. External validation of Global Evaluative Assessment of Robotic Skills (GEARS). Surg Endosc 2015;29:3261-6. [Crossref] [PubMed]
28. Aghazadeh MA, Mercado MA, Pan MM, et al. Performance of robotic simulated skills tasks is positively associated with clinical robotic surgical performance. BJU Int 2016;118:475-81. [Crossref] [PubMed]
29. Ahmad SB, Rice M, Chang C, et al. dV-Trainer vs. da Vinci Simulator: Comparison of Virtual Reality Platforms for Robotic Surgery. J Surg Res 2021;267:695-704. [Crossref] [PubMed]
30. Almarzouq A, Hu J, Noureldin YA, et al. Are basic robotic surgical skills transferable from the simulator to the operating room? A randomized, prospective, educational study. Can Urol Assoc J 2020;14:416-22. [Crossref] [PubMed]
31. Altok M, Achim MF, Matin SF, et al. A decade of robot-assisted radical prostatectomy training: Time-based metrics and qualitative grading for fellows and residents. Urol Oncol 2018;36:13.e19-25. [Crossref] [PubMed]
32. Anand A, Gan C, Jensen R, et al. Differences in coaching in single- versus dual-console robotic cases: a mixed-methods study. Surg Endosc 2024;38:6008-16. [Crossref] [PubMed]
33. Araujo PHXN, Pêgo-Fernandes PM. Robotic surgery training. Sao Paulo Med J 2023;141:e20231415. [Crossref] [PubMed]
34. Baimas-George M, Watson M, Martinie J, et al. Curriculum matrix development for a hepato-pancreato-biliary robotic surgery fellowship. Can J Surg 2021;64:E657-62. [Crossref] [PubMed]
35. Baldea KG, Thorwarth R, Bajic P, et al. Design and Implementation of a Robotic Surgery Training Experience Logging System. J Surg Educ 2017;74:1047-51. [Crossref] [PubMed]
36. Ballas DA, Cesta M, Gothard D, et al. Emergency Undocking Curriculum in Robotic Surgery. Cureus 2019;11:e4321. [PubMed]
37. Ballesta Martinez B, Kallidonis P, Tsaturyan A, et al. Transfer of acquired practical skills from dry lab into live surgery using the avatera robotic system: An experimental study. Actas Urol Esp (Engl Ed) 2023;47:611-7. [Crossref] [PubMed]
38. Bendre HH, Rajender A, Barbosa PV, et al. Robotic dismembered pyeloplasty surgical simulation using a 3D-printed silicone-based model: development, face validation and crowdsourced learning outcomes assessment. J Robot Surg 2020;14:897-902. [Crossref] [PubMed]
39. Berges AJ, Vedula SS, Malpani A, et al. Virtual Reality Simulation Has Weak Correlation with Overall Trainee Robot-Assisted Laparoscopic Hysterectomy Performance. J Minim Invasive Gynecol 2022;29:507-18. [Crossref] [PubMed]
40. Bertolo R, Garisto J, Dagenais J, et al. Single Session of Robotic Human Cadaver Training: The Immediate Impact on Urology Residents in a Teaching Hospital. J Laparoendosc Adv Surg Tech A 2018;28:1157-62. [Crossref] [PubMed]
41. Beulens AJW, Brinkman WM, Porte PJ, et al. The value of a 1-day multidisciplinary robot surgery training for novice robot surgeons. J Robot Surg 2019;13:435-47. [Crossref] [PubMed]
42. Beulens AJW, Hashish YAF, Brinkman WM, et al. Training novice robot surgeons: Proctoring provides same results as simulator-generated guidance. J Robot Surg 2021;15:397-428. [Crossref] [PubMed]
43. Beulens AJW, Vaartjes L, Tilli S, et al. Structured robot-assisted surgery training curriculum for residents in Urology and impact on future surgical activity. J Robot Surg 2021;15:497-510. [Crossref] [PubMed]
44. Bjerrum F, Collins JW, Butterworth J, et al. Competency assessment for the Versius surgical robot: a validity investigation study of a virtual reality simulator-based test. Surg Endosc 2023;37:7464-71. [Crossref] [PubMed]
45. Boitano TKL, Smith HJ, Cohen JG, et al. Implementation and evaluation of a novel subspecialty society fellows robotic surgical course: the SGO minimally invasive academy surgical curriculum. J Gynecol Oncol 2021;32:e26. [Crossref] [PubMed]
46. Brown K, Mosley N, Tierney J. Battle of the bots: a comparison of the standard da Vinci and the da Vinci Surgical Skills Simulator in surgical skills acquisition. J Robot Surg 2017;11:159-62. [Crossref] [PubMed]
47. Brown KC, Bhattacharyya KD, Kulason S, et al. How to Bring Surgery to the Next Level: Interpretable Skills Assessment in Robotic-Assisted Surgery. Visc Med 2020;36:463-70. [Crossref] [PubMed]
48. Brown JD, Kuchenbecker KJ. Effects of automated skill assessment on robotic surgery training. Int J Med Robot 2023;19:e2492. [Crossref] [PubMed]
49. Cacciatore L, Costantini M, Tedesco F, et al. Robotic Medtronic Hugo™ RAS System Is Now Reality: Introduction to a New Simulation Platform for Training Residents. Sensors (Basel) 2023;23:7348. [Crossref] [PubMed]
50. Carneiro A, Claros OR, Cha JD, et al. Can remote assistance for robotic surgery improve surgical performance in simulation training? A prospective clinical trial of urology residents using a simulator in south america. Int Braz J Urol 2022;48:952-60. [Crossref] [PubMed]
51. Carter SC, Chiang A, Shah G, et al. Video-based peer feedback through social networking for robotic surgery simulation: a multicenter randomized controlled trial. Ann Surg 2015;261:870-5. [Crossref] [PubMed]
52. Cerfolio RJ, Ferrari-Light D. How to get the most out of your trainees in robotic thoracic surgery-"the coachability languages". Ann Cardiothorac Surg 2019;8:269-73. [Crossref] [PubMed]
53. Chen AB, Liang S, Nguyen JH, et al. Machine learning analyses of automated performance metrics during granular sub-stitch phases predict surgeon experience. Surgery 2021;169:1245-9. [Crossref] [PubMed]
54. Chen CCG, Malpani A, Waldram MM, et al. Effect of pre-operative warm-up on trainee intraoperative performance during robot-assisted hysterectomy: a randomized controlled trial. Int Urogynecol J 2023;34:2751-8. [Crossref] [PubMed]
55. Chen G, Li L, Hubert J, et al. Effectiveness of a vision-based handle trajectory monitoring system in studying robotic suture operation. J Robot Surg 2023;17:2791-8. [Crossref] [PubMed]
56. Chow AK, Wong R, Monda S, et al. Ex Vivo Porcine Model for Robot-Assisted Partial Nephrectomy Simulation at a High-Volume Tertiary Center: Resident Perception and Validation Assessment Using the Global Evaluative Assessment of Robotic Skills Tool. J Endourol 2021;35:878-84. [Crossref] [PubMed]
57. Chowriappa A, Raza SJ, Fazili A, et al. Augmented-reality-based skills training for robot-assisted urethrovesical anastomosis: a multi-institutional randomised controlled trial. BJU Int 2015;115:336-45. [Crossref] [PubMed]
58. Clanahan JM, Yee A, Awad MM. Active control time: an objective performance metric for trainee participation in robotic surgery. J Robot Surg 2023;17:2117-23. [Crossref] [PubMed]
59. Clanahan JM, Han BJ, Klos CL, et al. Use of Simulation For Training Advanced Colorectal Procedures. J Surg Educ 2024;81:758-67. [Crossref] [PubMed]
60. Clanahan JM, Awad MM, Dimou FM. Use of targeted educational resources to improve robotic bariatric surgery training. Surg Endosc 2024;38:894-901. [Crossref] [PubMed]
61. Cope AG, Lazaro-Weiss JJ, Willborg BE, et al. Surgical Science-Simbionix Robotic Hysterectomy Simulator: Validating a New Tool. J Minim Invasive Gynecol 2022;29:759-66. [Crossref] [PubMed]
62. Cowan A, Chen J, Mingo S, et al. Virtual Reality vs Dry Laboratory Models: Comparing Automated Performance Metrics and Cognitive Workload During Robotic Simulation Training. J Endourol 2021;35:1571-6. [Crossref] [PubMed]
63. Croghan SM, Voborsky M, Roche AF, et al. Design and utilisation of a novel, high-fidelity, low-cost, hybrid-tissue simulation model to facilitate training in robot-assisted partial nephrectomy. J Robot Surg 2024;18:103. [Crossref] [PubMed]
64. Davidson JT 4th, Clanahan JM, Vachharajani N, et al. A novel assessment model for teaching robot-assisted living donor nephrectomy in abdominal transplant surgery fellowship. Am J Surg 2023;225:420-4. [Crossref] [PubMed]
65. De Groote R, Puliatti S, Amato M, et al. Proficiency-based progression training for robotic surgery skills training: a randomized clinical trial. BJU Int 2022;130:528-35. [Crossref] [PubMed]
66. De Groote R, Puliatti S, Amato M, et al. Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial. Ann Surg Open 2023;4:e307. [Crossref] [PubMed]
67. Dickinson KJ, Racher ML, Jackman K, et al. Surgical Residents' Perception of Multi-Specialty Learning. J Surg Educ 2022;79:1363-78. [Crossref] [PubMed]
68. Dioun SM, Fleming ND, Munsell MF, et al. Setting Benchmarks for the New User: Training on the Robotic Simulator. JSLS 2017;21:e2017.00059.
69. Dubin AK, Julian D, Tanaka A, et al. A model for predicting the GEARS score from virtual reality surgical simulator metrics. Surg Endosc 2018;32:3576-81. [Crossref] [PubMed]
70. Gerull W, Zihni A, Awad M. Operative performance outcomes of a simulator-based robotic surgical skills curriculum. Surg Endosc 2020;34:4543-8. [Crossref] [PubMed]
71. Gerull WD, Liebendorfer A, Awad MM. Automated task-level autonomy assessment in robotic surgery. Surg Endosc 2024;38:6033-6. [Crossref] [PubMed]
72. Gheza F, Pinkard L, Grand A, et al. Development of an affordable, immersive model for robotic vaginal cuff closure: a randomized trial. J Robot Surg 2023;17:109-16. [PubMed]
73. Gleason A, Servais E, Quadri S, et al. Developing basic robotic skills using virtual reality simulation and automated assessment tools: a multidisciplinary robotic virtual reality-based curriculum using the Da Vinci Skills Simulator and tracking progress with the Intuitive Learning platform. J Robot Surg 2022;16:1313-9. [Crossref] [PubMed]
74. Goh AC, Aghazadeh MA, Mercado MA, et al. Multi-Institutional Validation of Fundamental Inanimate Robotic Skills Tasks. J Urol 2015;194:1751-6. [Crossref] [PubMed]
75. Gomez PP, Willis RE, Van Sickle KR. Development of a virtual reality robotic surgical curriculum using the da Vinci Si surgical system. Surg Endosc 2015;29:2171-9. [Crossref] [PubMed]
76. Gonçalves MR, Morales-Conde S, Gaspar Reis S, et al. RAWS4all project: validation of a new silicone model for robotic TAPP inguinal hernia repair. Surg Endosc 2024;38:1329-41. [Crossref] [PubMed]
77. Grannan HR, Hetzel E, Goldblatt MI, et al. Robotic General Surgery Resident Training Curriculum: A Pilot Experience. Surg Laparosc Endosc Percutan Tech 2021;31:588-93. [Crossref] [PubMed]
78. Green CA, Mahuron KM, Harris HW, et al. Integrating Robotic Technology Into Resident Training: Challenges and Recommendations From the Front Lines. Acad Med 2019;94:1532-8. [Crossref] [PubMed]
79. Green CA, O'Sullivan PS, Chern H. A robotic teaching session: separating tool from technique to emphasize a cognitive focused teaching environment. J Robot Surg 2019;13:735-9. [Crossref] [PubMed]
80. Green CA, Chu SN, Huang E, et al. Teaching in the robotic environment: Use of alternative approaches to guide operative instruction. Am J Surg 2020;219:191-6. [Crossref] [PubMed]
81. Green CA, Chern H, Rogers SJ, et al. Transforming Surgical Education through a Resident Robotic Curriculum. Ann Surg Open 2021;2:e076. [Crossref] [PubMed]
82. Guni A, Raison N, Challacombe B, et al. Development of a technical checklist for the assessment of suturing in robotic surgery. Surg Endosc 2018;32:4402-7. [Crossref] [PubMed]
83. Gurung PMS, Campbell T, Wang B, et al. Accelerated Skills Acquisition Protocol (ASAP) in optimizing robotic surgical simulation training: a prospective randomized study. World J Urol 2020;38:1623-30. [Crossref] [PubMed]
84. Han BJ, Sherrill W 3rd, Awad MM. The use of advanced robotic simulation labs to advance and assess senior resident robotic skills and operating room leadership competency: a pilot study. Surg Endosc 2023;37:3053-60. [Crossref] [PubMed]
85. Haque TF, Knudsen JE, You J, et al. Competency in Robotic Surgery: Standard Setting for Robotic Suturing Using Objective Assessment and Expert Evaluation. J Surg Educ 2024;81:422-30. [Crossref] [PubMed]
86. Hertz AM, George EI, Vaccaro CM, et al. Head-to-Head Comparison of Three Virtual-Reality Robotic Surgery Simulators. JSLS 2018;22:e2017.00081.
87. Hoffman MS, Zgheib NB, Young C, et al. Simulated management of inferior vena cava injury during robotic paraaortic lymphadenectomy utilizing the porcine model. J Robot Surg 2020;14:649-53. [Crossref] [PubMed]
88. Holst D, Kowalewski TM, White LW, et al. Crowd-sourced assessment of technical skills: an adjunct to urology resident surgical simulation training. J Endourol 2015;29:604-9. [Crossref] [PubMed]
89. Hoogenes J, Wong N, Al-Harbi B, et al. A Randomized Comparison of 2 Robotic Virtual Reality Simulators and Evaluation of Trainees' Skills Transfer to a Simulated Robotic Urethrovesical Anastomosis Task. Urology 2018;111:110-5. [Crossref] [PubMed]
90. Hung AJ, Bottyan T, Clifford TG, et al. Structured learning for robotic surgery utilizing a proficiency score: a pilot study. World J Urol 2017;35:27-34. [Crossref] [PubMed]
91. Iqbal A, Sakharuk I, Tan S, et al. Improvement in Resident Robotic Surgical Perception with a Cadaver Training Lab: A Pilot Study. Am Surg 2017;83:e414-20. [Crossref] [PubMed]
92. Iqbal U, Jing Z, Ahmed Y, et al. Development and Validation of an Objective Scoring Tool for Robot-Assisted Partial Nephrectomy: Scoring for Partial Nephrectomy. J Endourol 2022;36:647-53. [Crossref] [PubMed]
93. Jackson T, Cho EE, Nagatomo K, et al. Teacher and Trainee Learning Together-Dual Console and the 3 Arms. J Surg Educ 2020;77:720-2. [Crossref] [PubMed]
94. Jacob MO, Karatassas A, Hewett P, et al. The use of a porcine model to teach advanced abdominal wall dissection techniques. Surg Endosc 2023;37:9684-9. [Crossref] [PubMed]
95. Jarc AM, Shah SH, Adebar T, et al. Beyond 2D telestration: an evaluation of novel proctoring tools for robot-assisted minimally invasive surgery. J Robot Surg 2016;10:103-9. [Crossref] [PubMed]
96. Jarc AM, Stanley AA, Clifford T, et al. Proctors exploit three-dimensional ghost tools during clinical-like training scenarios: a preliminary study. World J Urol 2017;35:957-65. [Crossref] [PubMed]
97. Jiang J, Xing Y, Wang S, et al. Evaluation of robotic surgery skills using dynamic time warping. Comput Methods Programs Biomed 2017;152:71-83. [Crossref] [PubMed]
98. Jogerst KM, Coe TM, Petrusa E, et al. Multidisciplinary perceptions on robotic surgical training: the robot is a stimulus for surgical education change. Surg Endosc 2023;37:2688-97. [Crossref] [PubMed]
99. Johnson BA, Timberlake M, Steinberg RL, et al. Design and Validation of a Low-Cost, High-Fidelity Model for Urethrovesical Anastomosis in Radical Prostatectomy. J Endourol 2019;33:331-6. [Crossref] [PubMed]
100. Jones BT, Ha JS, Lawrence C, et al. A dedicated robotic bedside physician assistant significantly enhances trainee console operating time in general thoracic surgery. JTCVS Open 2023;16:1070-3. [Crossref] [PubMed]
101. Khan H, Kozlowski JD, Hussein AA, et al. Use of Robotic Anastomosis Competency Evaluation (RACE) for assessment of surgical competency during urethrovesical anastomosis. Can Urol Assoc J 2019;13:E10-6. [PubMed]
102. Kiely DJ, Gotlieb WH, Jardon K, et al. Advancing surgical simulation in gynecologic oncology: robotic dissection of a novel pelvic lymphadenectomy model. Simul Healthc 2015;10:38-42. [Crossref] [PubMed]
103. Kiely DJ, Gotlieb WH, Lau S, et al. Virtual reality robotic surgery simulation curriculum to teach robotic suturing: a randomized controlled trial. J Robot Surg 2015;9:179-86. [Crossref] [PubMed]
104. Kim JY, Kim SB, Pyun JH, et al. Concurrent and predictive validation of robotic simulator Tube 3 module. Korean J Urol 2015;56:756-61. [Crossref] [PubMed]
105. Kim MP, Del Calvo H, Chihara R, et al. Video-based curriculum improves resident participation during robot-assisted surgery. J Thorac Dis 2022;14:4641-9. [Crossref] [PubMed]
106. Kim JS, Jonas N, Rizvi TZ, et al. Validation of a multidisciplinary virtual reality (VR) robotic surgical curriculum. J Robot Surg 2023;17:2495-502. [Crossref] [PubMed]
107. Ko YH, Choi JY, Song PH. Concurrent validation of a robotic simulator curriculum focused on "core" exercises: Does it help overcome baseline skill levels? Int J Urol 2018;25:760-1. [Crossref] [PubMed]
108. Krause W, Bird J. Training robotic community surgeons: our experience implementing a robotics curriculum at a rural community general surgery training program. J Robot Surg 2019;13:385-9. [Crossref] [PubMed]
109. Kun Y, Hubert J, Bin L, et al. Self-debriefing Model Based on an Integrated Video-Capture System: An Efficient Solution to Skill Degradation. J Surg Educ 2019;76:362-9. [Crossref] [PubMed]
110. Laca JA, Kocielnik R, Nguyen JH, et al. Using Real-time Feedback To Improve Surgical Performance on a Robotic Tissue Dissection Task. Eur Urol Open Sci 2022;46:15-21. [Crossref] [PubMed]
111. Laverty RB, Khan MT, Patnaik R, et al. Intentional enterotomies: validation of a novel robotic surgery training exercise. J Robot Surg 2023;17:2109-15. [Crossref] [PubMed]
112. Lazar JF, Brown K, Yousaf S, et al. Objective performance indicators of cardiothoracic residents are associated with vascular injury during robotic-assisted lobectomy on porcine models. J Robot Surg 2023;17:669-76. [Crossref] [PubMed]
113. Lee GI, Lee MR. Can a virtual reality surgical simulation training provide a self-driven and mentor-free skills learning? Investigation of the practical influence of the performance metrics from the virtual reality robotic surgery simulator on the skill learning and associated cognitive workloads. Surg Endosc 2018;32:62-72. [Crossref] [PubMed]
114. Lee CR, Rho SY, Han SH, et al. Comparison of Training Efficacy Between Custom-Made Skills Simulator (CMSS) and da Vinci Skills Simulators: A Randomized Control Study. World J Surg 2019;43:2699-709. [Crossref] [PubMed]
115. Lee CS, Khan MT, Patnaik R, et al. Model Development of a Novel Robotic Surgery Training Exercise With Electrocautery. Cureus 2022;14:e24531. [Crossref] [PubMed]
116. Lee M, Han Y, Kang JS, et al. Training efficacy of robotic duct-to-mucosa pancreaticojejunostomy simulation using silicone models for surgical fellows. Ann Surg Treat Res 2024;106:45-50. [Crossref] [PubMed]
117. Leon MG, Carrubba AR, DeStephano CC, et al. Impact of robotic single and dual console systems in the training of minimally invasive gynecology surgery (MIGS) fellows. J Robot Surg 2022;16:1273-80. [Crossref] [PubMed]
118. Liang K, Xing Y, Li J, et al. Motion control skill assessment based on kinematic analysis of robotic end-effector movements. Int J Med Robot 2018; [Crossref] [PubMed]
119. Liu JS, Dickmeyer LJ, Nettey O, et al. Disparities in female urologic case distribution with new subspecialty certification and surgeon gender. Neurourol Urodyn 2017;36:399-403. [Crossref] [PubMed]
120. Liu S, Watkins K, Hall CE, et al. Utilizing Simulation to Evaluate Robotic Skill Acquisition and Learning Decay. Surg Laparosc Endosc Percutan Tech 2023;33:317-23. [Crossref] [PubMed]
121. Lovegrove C, Novara G, Mottrie A, et al. Structured and Modular Training Pathway for Robot-assisted Radical Prostatectomy (RARP): Validation of the RARP Assessment Score and Learning Curve Assessment. Eur Urol 2016;69:526-35. [Crossref] [PubMed]
122. Lovegrove C, Ahmed K, Novara G, et al. Modular Training for Robot-Assisted Radical Prostatectomy: Where to Begin? J Surg Educ 2017;74:486-94. [Crossref] [PubMed]
123. Lyman WB, Passeri MJ, Murphy K, et al. An objective approach to evaluate novice robotic surgeons using a combination of kinematics and stepwise cumulative sum (CUSUM) analyses. Surg Endosc 2021;35:2765-72. [Crossref] [PubMed]
124. Ma R, Kiyasseh D, Laca JA, et al. Artificial Intelligence-Based Video Feedback to Improve Novice Performance on Robotic Suturing Skills: A Pilot Study. J Endourol 2024;38:884-91. [Crossref] [PubMed]
125. Madion MP, Kastenmeier A, Goldblatt MI, et al. Robotic surgery training curricula: prevalence, perceptions, and educational experiences in general surgery residency programs. Surg Endosc 2022;36:6638-46. [Crossref] [PubMed]
126. Margueritte F, Sallée C, Legros M, et al. Description of an initiation program to robotic in vivo gynecological surgery for junior surgeons. J Gynecol Obstet Hum Reprod 2020;49:101627. [Crossref] [PubMed]
127. Mariani A, Pellegrini E, De Momi E. Skill-Oriented and Performance-Driven Adaptive Curricula for Training in Robot-Assisted Surgery Using Simulators: A Feasibility Study. IEEE Trans Biomed Eng 2021;68:685-94. [Crossref] [PubMed]
128. Melich G, Pai A, Shoela R, et al. Rectal Dissection Simulator for da Vinci Surgery: Details of Simulator Manufacturing With Evidence of Construct, Face, and Content Validity. Dis Colon Rectum 2018;61:514-9. [Crossref] [PubMed]
129. Melnyk R, Campbell T, Holler T, et al. See Like an Expert: Gaze-Augmented Training Enhances Skill Acquisition in a Virtual Reality Robotic Suturing Task. J Endourol 2021;35:376-82. [Crossref] [PubMed]
130. Merriman AL, Tarr ME, Kasten KR, et al. A resident robotic curriculum utilizing self-selection and a web-based feedback tool. J Robot Surg 2023;17:383-92. [Crossref] [PubMed]
131. Moit H, Dwyer A, De Sutter M, et al. A Standardized Robotic Training Curriculum in a General Surgery Program. JSLS 2019;23:e2019.00045.
132. Monda SM, Weese JR, Anderson BG, et al. Development and Validity of a Silicone Renal Tumor Model for Robotic Partial Nephrectomy Training. Urology 2018;114:114-20. [Crossref] [PubMed]
133. Moran GW, Margolin EJ, Wang CN, et al. Using gamification to increase resident engagement in surgical training: Our experience with a robotic surgery simulation league. Am J Surg 2022;224:321-2. [Crossref] [PubMed]
134. Mouraviev V, Klein M, Schommer E, et al. Urology residents experience comparable workload profiles when performing live porcine nephrectomies and robotic surgery virtual reality training modules. J Robot Surg 2016;10:49-56. [Crossref] [PubMed]
135. Nakamoto K, Jones DB, Adra SW. Gamification of robotic simulation to train general surgery residents. Surg Endosc 2023;37:3136-44. [Crossref] [PubMed]
136. Nathan A, Patel S, Georgi M, et al. Virtual classroom proficiency-based progression for robotic surgery training (VROBOT): a randomised, prospective, cross-over, effectiveness study. J Robot Surg 2023;17:629-35. [Crossref] [PubMed]
137. Newcomb LK, Bradley MS, Truong T, et al. Correlation of Virtual Reality Simulation and Dry Lab Robotic Technical Skills. J Minim Invasive Gynecol 2018;25:689-96. [Crossref] [PubMed]
138. Oh D, Brown K, Yousaf S, et al. Differences Between Attending and Trainee Surgeon Performance Using Objective Performance Indicators During Robot-Assisted Lobectomy. Innovations (Phila) 2023;18:479-88. [Crossref] [PubMed]
139. Olsen RG, Konge L, Hayatzaki K, et al. Laypersons versus experienced surgeons in assessing simulated robot-assisted radical prostatectomy. World J Urol 2023;41:3745-51. [Crossref] [PubMed]
140. Olsen RG, Svendsen MBS, Tolsgaard MG, et al. Automated performance metrics and surgical gestures: two methods for assessment of technical skills in robotic surgery. J Robot Surg 2024;18:297. [Crossref] [PubMed]
141. Olsen RG, Svendsen MBS, Tolsgaard MG, et al. Surgical gestures can be used to assess surgical competence in robot-assisted surgery: A validity investigating study of simulated RARP. J Robot Surg 2024;18:47. [Crossref] [PubMed]
142. Oquendo YA, Coad MM, Wren SM, et al. Haptic Guidance and Haptic Error Amplification in a Virtual Surgical Robotic Training Environment. IEEE Trans Haptics 2024;17:417-28. [Crossref] [PubMed]
143. Papalois ZA, Aydın A, Khan A, et al. HoloMentor: A Novel Mixed Reality Surgical Anatomy Curriculum for Robot-Assisted Radical Prostatectomy. Eur Surg Res 2022;63:40-5. [Crossref] [PubMed]
144. Patel E, Mascarenhas A, Ahmed S, et al. Evaluating the ability of students to learn and utilize a novel telepresence platform, Proximie. J Robot Surg 2022;16:973-9. [Crossref] [PubMed]
145. Perry B, Howard K, Novotny NM, et al. Identifying barriers to resident robotic console time in a general surgery residency through a targeted needs assessment. J Robot Surg 2023;17:2783-9. [Crossref] [PubMed]
146. Phé V, Cattarino S, Parra J, et al. Outcomes of a virtual-reality simulator-training programme on basic surgical skills in robot-assisted laparoscopic surgery. Int J Med Robot 2017; [Crossref] [PubMed]
147. Polin MR, Siddiqui NY, Comstock BA, et al. Crowdsourcing: a valid alternative to expert evaluation of robotic surgery skills. Am J Obstet Gynecol 2016;215:644.e1-7. [Crossref] [PubMed]
148. Porterfield JR Jr, Podolsky D, Ballecer C, et al. Structured Resident Training in Robotic Surgery: Recommendations of the Robotic Surgery Education Working Group. J Surg Educ 2024;81:9-16. [Crossref] [PubMed]
149. Powers MK, Boonjindasup A, Pinsky M, et al. Crowdsourcing Assessment of Surgeon Dissection of Renal Artery and Vein During Robotic Partial Nephrectomy: A Novel Approach for Quantitative Assessment of Surgical Performance. J Endourol 2016;30:447-52. [Crossref] [PubMed]
150. Puliatti S, Mazzone E, Amato M, et al. Development and validation of the objective assessment of robotic suturing and knot tying skills for chicken anastomotic model. Surg Endosc 2021;35:4285-94. [Crossref] [PubMed]
151. Puliatti S, Amato M, Farinha R, et al. Does quality assured eLearning provide adequate preparation for robotic surgical skills; a prospective, randomized and multi-center study. Int J Comput Assist Radiol Surg 2022;17:457-65. [Crossref] [PubMed]
152. Quinn KM, Chen X, Runge LT, et al. The robot doesn't lie: real-life validation of robotic performance metrics. Surg Endosc 2023;37:5547-52. [Crossref] [PubMed]
153. Raad WN, Ayub A, Huang CY, et al. Robotic Thoracic Surgery Training for Residency Programs: A Position Paper for an Educational Curriculum. Innovations (Phila) 2018;13:417-22. [Crossref] [PubMed]
154. Radi I, Tellez JC, Alterio RE, et al. Feasibility, effectiveness and transferability of a novel mastery-based virtual reality robotic training platform for general surgery residents. Surg Endosc 2022;36:7279-87. [Crossref] [PubMed]
155. Rahimi AM, Hardon SF, Willuth E, et al. Force-based assessment of tissue handling skills in simulation training for robot-assisted surgery. Surg Endosc 2023;37:4414-20. [Crossref] [PubMed]
156. Raison N, Poulsen J, Abe T, et al. An evaluation of live porcine simulation training for robotic surgery. J Robot Surg 2021;15:429-34. [Crossref] [PubMed]
157. Ramirez Barriga M, Rojas A, Roggin KK, et al. Development of a Two-Week Dedicated Robotic Surgery Curriculum for General Surgery Residents. J Surg Educ 2022;79:861-6. [Crossref] [PubMed]
158. Rusch P, Kimmig R, Lecuru F, et al. The Society of European Robotic Gynaecological Surgery (SERGS) Pilot Curriculum for robot assisted gynecological surgery. Arch Gynecol Obstet 2018;297:415-20. [Crossref] [PubMed]
159. S Schmiederer I. Developing a Robotic General Surgery Training Curriculum: Identifying Key Elements Through a Delphi Process. J Surg Educ 2021;78:e129-36. [Crossref] [PubMed]
160. Sanford DI, Ma R, Ghoreifi A, et al. Association of Suturing Technical Skill Assessment Scores Between Virtual Reality Simulation and Live Surgery. J Endourol 2022;36:1388-94. [Crossref] [PubMed]
161. Satava RM, Stefanidis D, Levy JS, et al. Proving the Effectiveness of the Fundamentals of Robotic Surgery (FRS) Skills Curriculum: A Single-blinded, Multispecialty, Multi-institutional Randomized Control Trial. Ann Surg 2020;272:384-92. [Crossref] [PubMed]
162. Schlottmann F, Patti MG. Novel simulator for robotic surgery. J Robot Surg 2017;11:463-5. [Crossref] [PubMed]
163. Schlottmann F, Long JM, Brown S, et al. Low confidence levels with the robotic platform among senior surgical residents: simulation training is needed. J Robot Surg 2019;13:155-8. [Crossref] [PubMed]
164. Schneyer RJ, Scheib SA, Green IC, et al. Validation of a Simulation Model for Robotic Myomectomy. J Minim Invasive Gynecol 2024;31:330-340.e1. [Crossref] [PubMed]
165. Schommer E, Patel VR, Mouraviev V, et al. Diffusion of Robotic Technology Into Urologic Practice has Led to Improved Resident Physician Robotic Skills. J Surg Educ 2017;74:55-60. [Crossref] [PubMed]
166. Scott SI, Dalsgaard T, Jepsen JV, et al. Design and validation of a cross-specialty simulation-based training course in basic robotic surgical skills. Int J Med Robot 2020;16:1-10. [Crossref] [PubMed]
167. Scott ER, Singh A, Quinn AM, et al. The use of individualized 3D-printed models on trainee and patient education, and surgical planning for robotic partial nephrectomies. J Robot Surg 2023;17:465-72. [Crossref] [PubMed]
168. Shafiei SB, Shadpour S, Mohler JL, et al. Surgical skill level classification model development using EEG and eye-gaze data and machine learning algorithms. J Robot Surg 2023;17:2963-71. [Crossref] [PubMed]
169. Shafiei SB, Shadpour S, Mohler JL, et al. Prediction of Robotic Anastomosis Competency Evaluation (RACE) metrics during vesico-urethral anastomosis using electroencephalography, eye-tracking, and machine learning. Sci Rep 2024;14:14611. [Crossref] [PubMed]
170. Shaw RD, Eid MA, Bleicher J, et al. Current Barriers in Robotic Surgery Training for General Surgery Residents. J Surg Educ 2022;79:606-13. [Crossref] [PubMed]
171. Shee K, Koo K, Wu X, et al. A novel ex vivo trainer for robotic vesicourethral anastomosis. J Robot Surg 2020;14:21-7. [Crossref] [PubMed]
172. Siddiqui NY, Tarr ME, Geller EJ, et al. Establishing Benchmarks For Minimum Competence With Dry Lab Robotic Surgery Drills. J Minim Invasive Gynecol 2016;23:633-8. [Crossref] [PubMed]
173. Simmonds C, Brentnall M, Lenihan J. Evaluation of a novel universal robotic surgery virtual reality simulation proficiency index that will allow comparisons of users across any virtual reality simulation curriculum. Surg Endosc 2021;35:5867-75. [Crossref] [PubMed]
174. Soangra R, Jiang P, Haik D, et al. Beyond Efficiency: Surface Electromyography Enables Further Insights into the Surgical Movements of Urologists. J Endourol 2022;36:1355-61. [Crossref] [PubMed]
175. Stewart CL, Green C, Meara MP, et al. Common Components of General Surgery Robotic Educational Programs. J Surg Educ 2023;80:1717-22. [Crossref] [PubMed]
176. Tarr ME, Anderson-Montoya BL, Vilasagar S, et al. Validation of a Simulation Model for Robotic Sacrocolpopexy. Female Pelvic Med Reconstr Surg 2022;28:14-9. [Crossref] [PubMed]
177. Tellez JC, Radi I, Alterio RE, et al. Proficiency Levels and Validity Evidence for Scoring Metrics for a Virtual Reality and Inanimate Robotic Surgery Simulation Curriculum. J Surg Educ 2024;81:589-96. [Crossref] [PubMed]
178. Thomaschewski M, Kist M, Zimmermann M, et al. Conception and prospective multicentric validation of a Robotic Surgery Training Curriculum (RoSTraC) for surgical residents: from simulation via laboratory training to integration into the operation room. J Robot Surg 2024;18:53. [Crossref] [PubMed]
179. Tillou X, Collon S, Martin-Francois S, et al. Robotic Surgery Simulator: Elements to Build a Training Program. J Surg Educ 2016;73:870-8. [Crossref] [PubMed]
180. Timberlake MD, Garbens A, Schlomer BJ, et al. Design and validation of a low-cost, high-fidelity model for robotic pyeloplasty simulation training. J Pediatr Urol 2020;16:332-9. [Crossref] [PubMed]
181. Tom CM, Maciel JD, Korn A, et al. A survey of robotic surgery training curricula in general surgery residency programs: How close are we to a standardized curriculum? Am J Surg 2019;217:256-60. [Crossref] [PubMed]
182. Towner MN, Lozada-Capriles Y, LaLonde A, et al. Creation and Piloting of a Model for Simulating a Minimally Invasive Myomectomy. Cureus 2019;11:e4223. [Crossref] [PubMed]
183. Turbati MS, Goldblatt MI, Gould JC, et al. Robotic simulation: validation and qualitative assessment of a general surgery resident training curriculum. Surg Endosc 2023;37:2304-15. [Crossref] [PubMed]
184. Turner SR, Mormando J, Park BJ, et al. Attitudes of robotic surgery educators and learners: challenges, advantages, tips and tricks of teaching and learning robotic surgery. J Robot Surg 2020;14:455-61. [Crossref] [PubMed]
185. Turner TB, Kim KH. Mapping the robotic hysterectomy learning curve and re-establishing surgical training metrics. J Gynecol Oncol 2021;32:e58. [Crossref] [PubMed]
186. Unruh K, Stovall S, Chang L, et al. Implementation of a structured robotic colorectal curriculum for general surgery residents. J Robot Surg 2023;17:2331-8. [Crossref] [PubMed]
187. Valdis M, Chu MW, Schlachta C, et al. Evaluation of robotic cardiac surgery simulation training: A randomized controlled trial. J Thorac Cardiovasc Surg 2016;151:1498-1505.e2. [Crossref] [PubMed]
188. Vanstrum EB, Ma R, Maya-Silva J, et al. Development and validation of an objective scoring tool to evaluate surgical dissection: Dissection Assessment for Robotic Technique (DART). Urol Pract 2021;8:596-604. [Crossref] [PubMed]
189. Van't Hullenaar CDP, Mertens AC, Ruurda JP, et al. Validation of ergonomic instructions in robot-assisted surgery simulator training. Surg Endosc 2018;32:2533-40. [Crossref] [PubMed]
190. Vetter MH, Palettas M, Hade E, et al. Time to consider integration of a formal robotic-assisted surgical training program into obstetrics/gynecology residency curricula. J Robot Surg 2018;12:517-21. [Crossref] [PubMed]
191. Volpe A, Ahmed K, Dasgupta P, et al. Pilot Validation Study of the European Association of Urology Robotic Training Curriculum. Eur Urol 2015;68:292-9. [Crossref] [PubMed]
192. von Bechtolsheim F, Franz A, Schmidt S, et al. The development of tissue handling skills is sufficient and comparable after training in virtual reality or on a surgical robotic system: a prospective randomized trial. Surg Endosc 2024;38:2900-10. [Crossref] [PubMed]
193. von Rundstedt FC, Aghazadeh MA, Scovell J, et al. Validation of a Simulation-training Model for Robotic Intracorporeal Bowel Anastomosis Using a Step-by-step Technique. Urology 2018;120:125-30. [Crossref] [PubMed]
194. Walker JL, Nathwani JN, Mohamadipanah H, et al. Residents' response to bleeding during a simulated robotic surgery task. J Surg Res 2017;220:385-90. [Crossref] [PubMed]
195. Wang VL, Pieper H, Gupta A, et al. Expectations versus reality: trainee participation on the robotic console in academic surgery. Surg Endosc 2021;35:4805-10. [Crossref] [PubMed]
196. Wang TN, Woelfel IA, Pieper H, et al. Is Robotic Console Time a Surrogate for Resident Operative Autonomy? J Surg Educ 2023;80:1711-6. [Crossref] [PubMed]
197. Whittaker G, Aydin A, Raison N, et al. Validation of the RobotiX Mentor Robotic Surgery Simulator. J Endourol 2016;30:338-46. [Crossref] [PubMed]
198. Whittaker G, Aydin A, Raveendran S, et al. Validity assessment of a simulation module for robot-assisted thoracic lobectomy. Asian Cardiovasc Thorac Ann 2019;27:23-9. [Crossref] [PubMed]
199. White LW, Kowalewski TM, Dockter RL, et al. Crowd-Sourced Assessment of Technical Skill: A Valid Method for Discriminating Basic Robotic Surgery Skills. J Endourol 2015;29:1295-301. [Crossref] [PubMed]
200. Wiener S, Haddock P, Shichman S, et al. Construction of a Urologic Robotic Surgery Training Curriculum: How Many Simulator Sessions Are Required for Residents to Achieve Proficiency? J Endourol 2015;29:1289-93. [Crossref] [PubMed]
201. Wile RK, Brian R, Rodriguez N, et al. Home practice for robotic surgery: a randomized controlled trial of a low-cost simulation model. J Robot Surg 2023;17:2527-36. [Crossref] [PubMed]
202. Winder JS, Juza RM, Sasaki J, et al. Implementing a robotics curriculum at an academic general surgery training program: our initial experience. J Robot Surg 2016;10:209-13. [Crossref] [PubMed]
203. Witthaus MW, Farooq S, Melnyk R, et al. Incorporation and validation of clinically relevant performance metrics of simulation (CRPMS) into a novel full-immersion simulation platform for nerve-sparing robot-assisted radical prostatectomy (NS-RARP) utilizing three-dimensional printing and hydrogel casting technology. BJU Int 2020;125:322-32. [Crossref] [PubMed]
204. Wong EY, Chu TN, Ma R, et al. Development of a Classification System for Live Surgical Feedback. JAMA Netw Open 2023;6:e2320702. [Crossref] [PubMed]
205. Zia A, Essa I. Automated surgical skill assessment in RMIS training. Int J Comput Assist Radiol Surg 2018;13:731-9. [Crossref] [PubMed]
206. Zhao B, Lam J, Hollandsworth HM, et al. General surgery training in the era of robotic surgery: a qualitative analysis of perceptions from resident and attending surgeons. Surg Endosc 2020;34:1712-21. [Crossref] [PubMed]
207. Stegemann AP, Ahmed K, Syed JR, et al. Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. Urology 2013;81:767-74. [Crossref] [PubMed]
208. Liu M, Purohit S, Mazanetz J, et al. Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery. Surg Endosc 2018;32:526-35. [Crossref] [PubMed]