Clinical validation of an automatic atlas-based segmentation tool for male pelvis CT images

Purpose: This retrospective work aims to evaluate the possible impact on intra- and inter-observer variability, contouring time, and contour accuracy of introducing a pelvis computed tomography (CT) auto-segmentation tool in radiotherapy planning workflow. Methods: Tests were carried out on five structures (bladder, rectum, pelvic lymph-nodes, and femoral heads) of six previously treated subjects, enrolling five radiation oncologists (ROs) to manually re-contour and edit auto-contours generated with a male pelvis CT atlas created with the commercial software MIM MAESTRO. The ROs first delineated manual contours (M). Then they modified the auto-contours, producing automatic-modified (AM) contours. The procedure was repeated to evaluate intra-observer variability, producing M1, M2, AM1, and AM2 contour sets (each comprising 5 structures × 6 test patients × 5 ROs = 150 contours), for a total of 600 contours. Potential time savings was evaluated by comparing contouring and editing times. Structure contours were compared to a reference standard by means of Dice similarity coefficient (DSC) and mean distance to agreement (MDA), to assess intra- and inter-observer variability. To exclude any automation bias, ROs evaluated both M and AM sets as “clinically acceptable” or “to be corrected” in a blind test. Results: Comparing AM to M sets, a significant reduction of both inter-observer variability (p < 0.001) and contouring time (-45% whole pelvis, p < 0.001) was obtained. Intra-observer variability reduction was significant only for bladder and femoral heads (p < 0.001). The statistical test showed no significant bias. Conclusion: Our atlas-based workflow proved to be effective for clinical practice as it can improve contour reproducibility and generate time savings. Based on these findings, institutions are encouraged to implement their auto-segmentation method.