Auto-delineation of treatment target volume for radiation therapy using large language model-aided multimodal learning.

PURPOSE: Artificial intelligence (AI)-aided methods have made significant progress in the auto-delineation of normal tissues. However, these approaches struggle with the auto-contouring of radiotherapy target volume. Our goal is to model the delineation of target volume as a clinical decision-making problem, resolved by leveraging large language model-aided multimodal learning approaches. METHODS AND MATERIALS: A vision-language model, termed Medformer, has been developed, employing the hierarchical vision transformer as its backbone, and incorporating large language models to extract text-rich features. The contextually embedded linguistic features are seamlessly integrated into visual features for language-aware visual encoding through the visual language attention module. Metrics, including Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95), were used to quantitatively evaluate the performance of our model. The evaluation was conducted on an in-house prostate cancer dataset and a public oropharyngeal carcinoma (OPC) dataset, totaling 668 subjects. RESULTS: Our Medformer achieved a DSC of 0.81 ± 0.10 versus 0.72 ± 0.10, IOU of 0.73 ± 0.12 versus 0.65 ± 0.09, and HD95 of 9.86 ± 9.77 mm versus 19.13 ± 12.96 mm for delineation of gross tumor volume (GTV) on the prostate cancer dataset. Similarly, on the OPC dataset, it achieved a DSC of 0.77 ± 0.11 versus 0.72 ± 0.09, IOU of 0.70 ± 0.09 versus 0.65 ± 0.07, and HD95 of 7.52 ± 4.8 mm versus 13.63 ± 7.13 mm, representing significant improvements (p < 0.05). For delineating the clinical target volume (CTV), Medformer achieved a DSC of 0.91 ± 0.04, IOU of 0.85 ± 0.05, and HD95 of 2.98 ± 1.60 mm, comparable to other state-of-the-art algorithms. CONCLUSIONS: Auto-delineation of the treatment target based on multimodal learning outperforms conventional approaches that rely purely on visual features. Our method could be adopted into routine practice to rapidly contour CTV/GTV.

Automated radiotherapy treatment planning guided by GPT-4Vision.

Radiotherapy treatment planning is a time-consuming and potentially subjective process that requires the iterative adjustment of model parameters to balance multiple conflicting objectives. Recent advancements in large foundation models offer promising avenues for addressing the challenges in planning and clinical decision-making. This study introduces GPT-RadPlan, a fully automated treatment planning framework that harnesses prior radiation oncology knowledge encoded in multi-modal large language models, such as GPT-4Vision (GPT-4V) from OpenAI. GPT-RadPlan is made aware of planning protocols as context and acts as an expert human planner, capable of guiding a treatment planning process. Via in-context learning, we incorporate clinical protocols for various disease sites as prompts to enable GPT-4V to acquire treatment planning domain knowledge. The resulting GPT-RadPlan agent is integrated into our in-house inverse treatment planning system through an API. The efficacy of the automated planning system is showcased using multiple prostate and head & neck cancer cases, where we compared GPT-RadPlan results to clinical plans. In all cases, GPT-RadPlan either outperformed or matched the clinical plans, demonstrating superior target coverage and organ-at-risk sparing. Consistently satisfying the dosimetric objectives in the clinical protocol, GPT-RadPlan represents the first multimodal large language model agent that mimics the behaviors of human planners in radiation oncology clinics, achieving remarkable results in automating the treatment planning process without the need for additional training.

Photoselectively modulating main products by changing the wavelength of visible light over D-π-A-D conjugated polymers.

The diversity of catalytic products determines the difficulty of selective product modulation, which usually relies on adjusting the catalyst and reaction conditions to obtain different main products selectively. Herein, we synthesized D-π-A-D conjugated organic polymers (TH-COP) using cyclotriphosphonitrile, alkyne, 2H-benzimidazole, and sulfur units as electron donors, π bridges, electron acceptors, and electron donors, respectively. TH-COP exhibited excellent photoinduced carrier separation and redox ability under different visible light wavelengths, and the main products of its CO2 reduction are CH4 (1000.0 µmol g-1) and CO (837.0 µmol g-1) under 400-420 nm and 420-560 nm, respectively. In addition, TH-COP could completely convert phenylmethyl sulfide to methyl phenyl sulfone at 400-420 nm and diphenyl disulfide at 480-485 nm in yields up to 95 %. This study presents a novel strategy for the targeted fabrication of various main products using conjugated polymers by simply changing the wavelength range of visible light.

Automated contouring, treatment planning, and quality assurance for VMAT craniospinal irradiation (VMAT-CSI).

Purpose: Create a comprehensive automated solution for pediatric and adult VMAT-CSI including contouring, planning, and plan check to reduce planning time and improve plan quality. Methods: Seventy-seven previously treated CSI patients (age, 2-67 years) were used for creation of an auto-contouring model to segment 25 organs at risk (OARs). The auto-contoured OARs were evaluated using the Dice Similarity Coefficient (DSC), 95% Hausdorff Distance (HD95), and a qualitative ranking by one physician and one physicist (scale: 1-acceptable, 2-minor edits, 3-major edits). The auto-planning script was developed using the Varian Eclipse Scripting API and tested with 20 patients previously treated with either low-dose VMAT-CSI (12 Gy) or high-dose VMAT-CSI (36 Gy + 18 Gy boost). Clinically relevant metrics, planning time, and blinded physician review were used to evaluate significance of differences between the auto and manual plans. Finally, the plan preparation for treatment and plan check processes were automated to improve efficiency and safety of VMAT-CSI. Results: The auto-contours achieved an average DSC of 0.71 ± 0.15, HD95 of 4.81 ± 4.68, and reviewers' ranking of 1.22 ± 0.39, indicating close to "acceptable-as-is" contours. Compared to the manual CSI plans, the auto-plans for both dose regimens achieved statistically significant reductions in body V50% and Dmean for parotids, submandibular, and thyroid glands. The variance in the dosimetric parameters decreased for the auto-plans as compared to the manual plans indicating better plan consistency. From the blinded review, the auto-plans were marked as equivalent or superior to the manual-plans 88.3% of the time. The required time for the auto-contouring and planning was consistently between 1-2 hours compared to an estimated 5-6 hours for manual contouring and planning. Conclusions: Reductions in contouring and planning time without sacrificing plan quality were obtained using the developed auto-planning process. The auto-planning scripts and documentation will be made freely available to other institutions and clinics.