(UroToday.com) In a Hot Topic session during the Société Internationale D’Urologie (SIU) 2021 annual meeting focused on the role of artificial intelligence (AI) in urology, Dr. Andrew Hung assessed the role of AI to predict patient outcomes after robotic surgery and for automating skills assessment.
He began by emphasizing work from the University of Southern California Center of Robotic Simulation and Education to collect surgical video of robotic radical prostatectomy and systems data (including instrument kinematic data and systems event data) to derive automated performance metrics. The simplest, this may amount to the cumulative moving time of a given instrument while more advanced examples include wrist articulation. He emphasized that work on these automated performance metrics has been recognized on the cover of the Journal of Urology four times since the beginning of 2018.
Dr. Hung highlighted an example from their first pilot study published in 2018 comparing the during bladder mobilization in a robotic-assisted radical prostatectomy. As highlighted in the figure below, there is significant conservation of motion for the expert, compared to novice, surgeon.
While they initially looked at these metrics averaged over the whole surgical case, the granularity of these data has allowed for more nuanced understanding using a machine learning algorithm to extract more nuanced data. In their initial assessment of this approach, Dr. Hung and colleagues used hospital length of stay (dichotomized at 2 days) as the outcome of interest. The machine learning algorithm was then used to predict the length of stay with an 85% accuracy.
Thus, these were then able to assess these metrics not just averaged over the whole case but broken down according to each step of the case individually. This approach, based on a deep-learning model, was then trained to predict urinary continence recovery following robotic assisted radical prostatectomy. Dr. Hung emphasized that patient, disease, and surgeon factors may contribute to postoperative outcomes including continence. In building these models, the authors considered 16 clinicopathologic characteristics at the patient level as well as 41 automated performance metrics for each of the 12 steps of the operation, yielding a total of 508 features per case.
Among 100 patients with complete outcome data, the authors examined three prediction models: one based on a Cox proportional hazards model, one based on a random survival forest, and one based on a survival analysis-based deep-learning approach. Using both patient and surgeon factors, the authors identified that the top 10 features most strongly associated with continence recovery were related to technical APMs. Further, five of these related to characteristics during the vesico-urethral anastomosis, and three of these involved right instrument wrist articulation.
Surgeons were then grouped on the basis of their experience and of their performance based on the top 5 ranked features for urinary continence. Dr. Hung emphasized that there is a re-ordering of the surgeons with many surgeons with less experience performing better on these metrics than their more experienced colleagues. In fact, when grouped according to their performance on these metrics, patients treated by higher performing surgeons were more likely to be continence at 3 and 6 months postoperatively than those treated by the lower performing surgeons.
Moving forward, Dr. Hung emphasized a detailed breakdown of each port of a step. This most recent work focused again on urinary continence. Here, in addition to the 16 clinicopathologic characteristics at the patient level and 41 automated performance metrics for each of the 12 steps of the operation, the authors added an assessment of granular suture performance metrics. These suturing metrics included needle positioning, needle driving, and suture cinching. Additionally, the authors considered suturing technical skills including needle positioning, needle entry, needle driving, and tissue trauma, suture placement, tissue approximation, and knot tying.
Whether using summary or detailed automated performance metrics, the AUC for urinary continence recovery was approximately 0.6. However, the technical skills increased the AUC into 0.7. However, combining all details including patient factors and surgeon technical skills yielded an AUC of approximately 0.8 in a deep-learning approach. Again, technical skills are among the most important characteristics for predicting time to continence recovery. This suggests an ability to identify surgeon-level strengths and weaknesses which may be amenable to feedback and quality improvement efforts.
These metrics may be used to measure surgeon performance. Again focusing on the vesico-urethral anastomosis. During needle driving, individual gestures may differ on the basis of the hand used, the direction of wrist rotation, and the manner of needle grasp. Capturing these characteristics, the authors were able to characterize patterns of suturing gestures for each suture position on the anastomosis. These data have allowed the derivation of a step-by-step tutorial to teach vesicourethral anastomosis. While these gestures may be characterised by trained humans, Dr. Hung’s group has also examined the use of deep learning-based computer vision to recognize and classify these gestures. This approach has an accuracy of approximately 70% among five of the most commonly utilized gestures.
This approach may be utilized to assess the renal hilum dissection during robotic assisted partial nephrectomy, in addition to not suturing during robotic assisted radical prostatectomy.
During hilar dissection, Dr. Hung highlighted that the majority of surgical movements (57%) related directly to dissection while the remaining comprised retraction, camera movement, assistant motion, coagulation, ineffective motions, and idle time. Interestingly, this work identified that the behaviour of surgeons during hilar dissection differed on the basis of their experience: more expert surgeons were more likely to use the peel/push (blunt dissection) around the vein than novice surgeons who were more likely to employ energy. In contrast, around the artery, experts were more likely to pedicalize tissue while novice surgeons used blunt dissection.
Moving back to robotic prostatectomy, Dr. Hung described a heat map of various surgical techniques employed during a left neurovascular bundle dissection.
Using these gestures, Dr. Hung emphasized that patterns may differ between those who recover erectile function and those patients who do not (with approximately 80% accuracy).
In summary, he emphasized that automated performance metrics and technical skills are predictive of patient outcomes. Further computer vision can recognize surgical gestures which are identifiable by machine learning and may be associated with patient outcomes.
Presented by: Andrew Hung, MD, Assistant Professor of Urology, USC, Los Angeles, CA