Plan Quality Analysis of Automated Treatment Planning Workflow With Commercial Auto-Segmentation Tools and Clinical Knowledge-Based Planning Models for Prostate Cancer

This study evaluated the feasibility of using artificial intelligence (AI) segmentation software for volume-modulated arc therapy (VMAT) prostate planning in conjunction with knowledge-based planning to facilitate a fully automated workflow. Two commercially available AI software programs, Radformation AutoContour (Radformation, New York, NY) and Siemens AI-Rad Companion (Siemens Healthineers, Malvern, PA) were used to auto-segment the rectum, bladder, femoral heads, and bowel bag on 30 retrospective clinical cases (10 intact prostate, 10 prostate bed, and 10 prostate and lymph node). Physician-segmented target volumes were transferred to AI structure sets. In-house RapidPlan models were used to generate plans using the original, physician-segmented structure sets as well as Radformation and Siemens AI-generated structure sets. Thus, there were three plans for each of the 30 cases, totaling 90 plans. Following RapidPlan optimization, planning target volume (PTV) coverage was set to 95%. Then, the plans optimized using AI structures were recalculated on the physician structure set with fixed monitor units. In this way, physician contours were used as the gold standard for identifying any clinically relevant differences in dose distributions. One-way analysis of variation (ANOVA) was used for statistical analysis. No statistically significant differences were observed across the three sets of plans for intact prostate, prostate bed, or prostate and lymph nodes. The results indicate that an automated volumetric modulated arc therapy (VMAT) prostate planning workflow can consistently achieve high plan quality. However, our results also show that small but consistent differences in contouring preferences may lead to subtle differences in planning results. Therefore, the clinical implementation of auto-contouring should be carefully validated.


Introduction
Prostate cancer (PCa) patients represent a significant percentage of the total patient population treated in radiotherapy clinics worldwide. PCa is the most frequently diagnosed cancer in men, with 1.4 million new cases diagnosed per year, leading to 375,000 deaths worldwide [1]. With the increasing incidence rate of PCa due to an aging male population combined with an increase in the overall therapeutic utilization of radiation therapy for all treatment sites, the cumulative number of cancer patients has been projected to significantly increase [1]. This cumulative increase has been projected to create an industry-wide surge in the demand for treatment planning staff and resources: a previous study has predicted a shortage of medical dosimetrists beginning in 2021 and extending into 2030 [2]. In response to these shortages, clinical workloads are estimated to increase, which poses challenges to clinical planning efficiency and quality. In general, it is important to provide timely access to radiotherapy, as extended delays correlate with an increased risk of local recurrence [3] and poor survival rates [4]. At the same time, ensuring high plan quality is also critical, as suboptimal quality has been linked to poor clinical outcomes [5]. The demand to provide a greater number of high-quality treatment plans is a real challenge due to a shortage of qualified medical dosimetrists.
In the current treatment planning workflow, image segmentation and plan optimization are two bottlenecks that consume tremendous time and energy from a medical dosimetrist, yet at the same time present opportunities for improved efficiency through innovation. Segmenting organs at risk (OARs), during the treatment planning process, serves as a crucial and necessary step to accurately delineate radiosensitive healthy tissues. Traditionally, segmentation has been performed manually, which requires significant time and effort by the planner to draw OAR boundaries slice by slice on a CT scan. Though a number of segmentation tools have been clinically available for over a decade, dosimetrists have been tasked with the time-consuming manual edits. Thanks to recent breakthroughs, deep learning-based auto-segmentation algorithms have achieved substantial improvements over the previous generation of atlas-based algorithms [6]. It has been shown that commercial software programs can generate contours comparable to manual contours, with variations reduced to those seen at inter-human-observer levels [7]. Another time-consuming yet crucial part of the planning workflow is setting plan optimization, which requires strategic plan constraints. The dosimetrist is typically tasked with setting these constraints through a manual trial-anderror process. The resulting plan quality depends strongly on planner skills [8]. Knowledge-based planning (KBP) methods have been developed to guide this inverse planning process based on empirical data from prior high-quality plans [9]. It has been shown that KBP can effectively reduce intra-and inter-planner plan quality variations and increase planning efficiency [10].
In light of the significant staffing challenges and the increased demands for high-quality treatment plans, the goal of this study is to evaluate the feasibility and quality of an automated workflow for VMAT prostate treatment planning facilitated by artificial intelligence (AI) auto-segmentation and KBP.

Materials And Methods
Male pelvic anatomy was segmented on CT image sets acquired from 30 clinical prostate patients. Among these 30 cases, there were 10 intact prostate cases, 10 prostate and lymph node cases, and 10 prostate bed cases. Three sets of contours were generated for each case: manual physician contours, Radformation AutoContours (Radformation, New York, NY), and contours auto-segmented by Siemens AI-Rad Companion (Siemens Healthineers, Malvern, PA), which is built into Syngo.via (Siemens Healthineers). We shall refer to these three datasets as MD, Radformation, and Syngo.via in the following sections of this paper. All manually drawn contours were generated and reviewed by staff physicians at a university hospital. The Radformation and Syngo.via artificial intelligence (AI) software contoured the bowel bag, femoral heads, bladder, and rectum. MD target volumes were transferred to these structure sets. VMAT treatment plans were created for each contour set using an in-house RapidPlan model, which was created specifically for intact prostate, prostate bed, or prostate and lymph nodes. Varian Eclipse photon optimizer (PO; Varian Medical Systems, Palo Alto, California) V16.1 was used for plan optimization, and the anisotropic analytical algorithm (AAA) V16.1 was used for the final dose calculations. Following the dose calculation, plans were normalized so that the prescription dose covered 95% of a PTV. After normalization, plans optimized using AI contours were reassigned to the MD structure set. These plans were then recalculated using prefixed monitor units. By taking this approach, AI contours were used for plan optimization, but the dose-volume histogram (DVH) analysis was based on the gold standard MD contours.
Clinically relevant plan quality metrics, including bladder V40Gy (%), bladder V70Gy (cc), rectum V45Gy (%), rectum V70Gy (cc), and femoral head Dmean (Gy), were evaluated for all plans. A one-way analysis of variation (ANOVA) test was used to evaluate the statistical significance of the differences in plan metrics between plan groups (MD, Radformation, Syngo.via). In this work, we have selected alpha=0.05/6=0.0083 as the significance threshold, with Bonferroni correction applied to six tests. Any result smaller than 0.0083 indicates statistical significance within the group.

One-way ANOVA
The null hypothesis for the one-way ANOVA tests is that all matching samples come from the same distribution. Based on the one-way ANOVA results shown in Table 1, no significant difference was observed between the three sets of plans for intact prostate, prostate bed, or prostate and lymph node.  Intact prostate Figure 1 shows the dose distribution of all three plans for a randomly selected intact prostate patient. It is evident that the contours and final dose distributions are consistent between all three plans. This can be confirmed by Figure 2, which shows the dosimetric results of all intact prostate plans included in this study.       Figure 4 shows an example case that could help explain the discrepancy between these plans. While MD and Radformation contours match very well. It is evident that the Syngo.via algorithm contoured additional OAR volumes inside the PTV for both the bladder and the rectum and thereby resulted in greater penalties on these OARs during plan optimization.

Radformation auto-segmentation, (C1-C3) Syngo.via auto-segmentation
Prostate and lymph node Figure 5 shows a representative prostate and lymph node case. The contours created by Radformation and Syngo.via are visually comparable with the clinical contours. The final dose distributions are also highly similar between the three plans. Table 4 and Figure 6 show the dosimetric values for the three sets of prostate and lymph node plans.

Discussion
Accurate contouring of the OARs is an essential requirement for crafting treatment plans that achieve sufficient target coverage while adequately sparing the healthy tissues to reduce the risk of normal tissue toxicities. In this study, no substantial contour errors were observed, and no contour edits were required for any contoured structures. The ability to generate accurate contours without needing edits reduces the human time required for treatment planning and potentially shortens the time between simulation and treatment so that a patient has faster access to radiotherapy.
Note that both AI auto-contouring software programs allow the planner to review and edit auto-segmented contours prior to export to the treatment planning system. This function was not utilized in the testing, as the study aimed to determine clinical acceptability based on an automatic workflow. This provided a significant decrease in planner-dependent cognitive input and time investment, allowing for every patient included in the study to be contoured and planned within a day. Even with these encouraging results, careful contour and plan review is strongly recommended in clinical practice. Yet even with this review, results from this study suggest that a high percentage of plans could be clinically acceptable without any human input during contouring or plan optimization, thereby improving efficiency without compromising on quality.
Manual contouring for prostate radiotherapy is labor-intensive and time-consuming. For clinics in low socio-economic areas that experience high patient load but suffer from staffing challenges due to limited resources, patients may face extended wait time between simulation and treatments. Moderate to high-risk patients may face an undue risk if their treatments are delayed3,4. In this study, we demonstrated that no statistically significant difference was observed from the plans created based on the radiation oncologist's manual contours versus Radformation or Syngo.Via auto-contours. The fully automated workflow with autosegmentation and knowledge-based planning has the potential to address the planning efficiency bottleneck and reduce treatment latency.
One limitation of this work is the lack of direct comparisons between manual contours and commercial software contours in terms of segmentation metrics such as the Dice similarity score and mean Hausdorff distance. However, there are a growing number of papers that benchmark the contouring accuracy of commercial auto-segmentation algorithms [11,12]. It is expected that the contouring accuracy of this study is similar to previous studies on this topic. Another potential factor that has not been taken into account in this study is the differing contouring preferences among physicians. This could lead to increased contouring inaccuracy. Both of these limitations warrant further studies on this topic.

Conclusions
This study has evaluated plans generated by an automated workflow powered by commercial autosegmentation and KBP software. The results indicate that auto-segmented contours can be directly used to generate plans with KBP without sacrificing plan quality. However, small but consistent differences in contouring preferences may lead to subtle differences in planning results. Therefore, the clinical implementation of auto-contouring still needs to be carefully validated.

Additional Information Disclosures
Human subjects: Consent was obtained or waived by all participants in this study. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.