Assessment of the Non-Technical Skills for Surgeons (NOTSS) framework in the USA

Background Surgeons' non-technical skills are important for patient safety. The Non-Technical Skills for Surgeons assessment tool was developed in the UK and recently adapted to the US surgical context (NOTSS-US). The aim of this study was to evaluate the reliability and distribution of non-technical skill ratings given by attending (consultant) surgeons who underwent brief online training. Methods Attending surgeons across six specialties at a large US academic medical centre underwent a 10-min online training, then rated 60-s standardized videos of simulated operations. Intraclass correlation coefficient (ICC), and mean(s.d.) values for NOTSS-US ratings were determined for each non-technical skill category (score range 1-5, where 1 indicates poor, 3 average and 5 excellent) and for total NOTSS-US score (range 4-20; sum of 4 category scores). Outcomes were adjusted for rater characteristics including sex, specialty and clinical experience. Results A total of 8889 ratings were submitted by 81 surgeon raters on 30 simulated intraoperative videos. The mean(s.d.) total NOTSS-US score for all videos was 9 center dot 5(4 center dot 8) of 20. The within-video ICC for total NOTSS-US score was 0 center dot 64 (95 per cent c.i. 0 center dot 57 to 0 center dot 70). For individual non-technical skill categories, the ICC was highest for social skills (communication/teamwork: 0 center dot 63, 95 per cent c.i. 0 center dot 56 to 0 center dot 71; leadership: 0 center dot 64, 0 center dot 55 to 0 center dot 72) and lowest for cognitive skills (situation awareness: 0 center dot 54, 0 center dot 45 to 0 center dot 62; decision-making: 0 center dot 50, 0 center dot 41 to 0 center dot 59). Women gave higher total NOTSS-US scores than men (adjusted mean difference 0 center dot 93, 95 per cent c.i. 0 center dot 44 to 1 center dot 43; P = 0 center dot 001). Conclusion After brief online training, the inter-rater reliability of the NOTSS-US assessment tool achieved moderate strength among trained surgeons rating simulated intraoperative videos.