Purpose: The Japanese Urological Association and Japanese Society of Endourology established a urological laparoscopic skills qualification system called the Endoscopic Surgical Skill Qualification (ESSQ) System in Urological Laparoscopy in 2004.
The reliability of video assessments by referees was evaluated.
Materials and Methods: Videos of nephrectomies or adrenalectomies performed by the applicants were assessed by 2 referees selected among a pool of 42 referees. From 2004 to 2011, 1308 urologists applied and 60.2% were qualified after video assessments. The results of skills assessments on 1220 videos which had fixed points by 2 referees were analyzed statistically.
Results: The average number of videos which each referee assessed was 58.1, with a range of 16 to 87. The accordance rate of the results of the video assessment, pass or fail, by the two referees was 68.9%. The scores of the video assessment by each referee averaged 62.7+/-2.4(SD) (full score was set at 75 points and >=60 points was required to pass). There was statistically significant difference in the average video assessment score among the referees (p< 0.001), and 5 referees showed significantly higher or lower average scores than the other referees. The percentage qualification of the final decision made by the referee committee on the videos originally assessed by each referee showed no significant differences among the 42 referees. The accordance rate of the results from the video assessment by each referee with the final decision by the committee showed a statistically significant positive correlation with the number of videos assessed by each referee (r=0.404, p=0.0080).
Conclusions: The ESSQ system showed moderate reliability for the video assessments by the referees. It was concluded that the video assessments by the referees were fair for all applicants, because the final qualification rates showed no significant differences among the referees.
Written by:
Matsuda T, Kanayama H, Ono Y, Kawauchi A, Mizoguchi H, Nakagawa K, Iwamura M, Shigeta M, Habuchi T, Terachi T. Are you the author?
Kansai Medical University, Department of Urology and Andrology, Shinmachi 2-5-1, Hirakata, Osaka, Japan.
Reference: J Endourol. 2014 May 12. Epub ahead of print.
doi: 10.1089/end.2014.0092
PubMed Abstract
PMID: 24819163
UroToday.com Endourology Section
