Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields.

Virtual surgical training is crucial for enhancing minimally invasive surgical skills. Traditional geometric reconstruction methods based on medical CT/MRI images often fall short in providing color information, which is typically generated through pseudo-coloring or artistic rendering. To simultaneously reconstruct both the geometric shape and appearance information of organs, we propose a novel organ model reconstruction network called Endoscope-NeSRF. This network jointly leverages neural radiance fields and Signed Distance Function (SDF) to reconstruct a textured geometric model of the organ of interest from multi-view photometric images acquired by an endoscope. The prior knowledge of the inverse correlation between the distance from the light source to the object and the radiance improves the real physical properties of the organ. The dilated mask further refines the appearance and geometry at the organ's edges. We also proposed a highlight adaptive optimization strategy to remove highlights caused by the light source during the acquisition process, thereby preventing the reconstruction results in areas previously affected by highlights from turning white. Finally, the real-time realistic rendering of the organ model is achieved by combining the inverse rendering and Bidirectional Reflectance Distribution Function (BRDF) rendering methods. Experimental results show that our method closely matches the Instant-NGP method in appearance reconstruction, outperforming other state-of-the-art methods, and stands as the superior method in terms of geometric reconstruction. Our method obtained a detailed geometric model and realistic appearance, providing a realistic visual sense for virtual surgical simulation, which is important for medical training.

Neural radiance fields-based multi-view endoscopic scene reconstruction for surgical simulation.

PURPOSE: In virtual surgery, the appearance of 3D models constructed from CT images lacks realism, leading to potential misunderstandings among residents. Therefore, it is crucial to reconstruct realistic endoscopic scene using multi-view images captured by an endoscope. METHODS: We propose an Endoscope-NeRF network for implicit radiance fields reconstruction of endoscopic scene under non-fixed light source, and synthesize novel views using volume rendering. Endoscope-NeRF network with multiple MLP networks and a ray transformer network represents endoscopic scene as implicit field function with color and volume density at continuous 5D vectors (3D position and 2D direction). The final synthesized image is obtained by aggregating all sampling points on each ray of the target camera using volume rendering. Our method considers the effect of distance from the light source to the sampling point on the scene radiance. RESULTS: Our network is validated on the lung, liver, kidney and heart of pig collected by our device. The results show that the novel views of endoscopic scene synthesized by our method outperform existing methods (NeRF and IBRNet) in terms of PSNR, SSIM, and LPIPS metrics. CONCLUSION: Our network can effectively learn a radiance field function with generalization ability. Fine-tuning the pre-trained model on a new endoscopic scene to further optimize the neural radiance fields of the scene, which can provide more realistic, high-resolution rendered images for surgical simulation.

Efficacy of immune checkpoint inhibitors in SMARCA4-deficient and TP53 mutant undifferentiated lung cancer.

The present study was conducted to characterize the clinicopathologic characteristics, immunohistochemical staining results, and immune checkpoint inhibitors (ICIs) efficacy in patients with SMARCA4-deficient/TP53 mutant lung cancer. Patients diagnosed with advanced or metastatic undifferentiated lung cancer harboring SMARCA4-deficient and TP53 mutations, however, without targetable sensitive mutations were retrieved from the electronic medical record system. Descriptive statistics were used to describe the baseline characteristics and clinical features including age, gender, eastern cooperative oncology group performance status, disease stage, smoking status, chief complaint, site of the primary mass, tumor size, gross type, symptoms, local invasion, and metastatic sizes. Immunological markers and potential drive genes were detected by immunohistochemical staining and next generation sequencing. Efficacy and safety profile of ICIs in included patients was evaluated with progression-free survival and overall survival. Between January 2019 and September 2022, there were 4 patients included within the inclusion criteria in the present study. Biomarkers including CK, CK7, and integrase interactor 1 were detected positive, however, other immunological markers including CK20, CD56, P63, P40, NapsinA, TTF-1, CgA, Syn, BRG1, or PD-L1 were detected negative among them. Results of next generation sequencing panel were failed to discover any targetable sensitive mutations. A total of 4 mutation types of TP53, including p.C141Y, p.S240G, p.E339X (terminator acquired), and p.L130F detected for the patients, respectively. Microsatellite stability status, as well as low tumor mutation burden was identified among all the patients. Median progression-free survival for ICIs as first line treatment and median overall survival were 3.25 months (range from 1.3 to 6.8 months), and 6.0 months (range from 2.7 to 9.6 months), respectively. Our results indicated that advanced lung cancer patients harboring co-occurring SMARCA4-deficient/TP53 mutations might respond to ICIs treatment, though within negative programmed cell death-ligand 1 expression or low tumor mutation burden. However, hyperprogressive disease by ICIs may also happen for such patients. The mutation types of TP53 might play a role during the exposure of ICIs, however, need further identification in basic experiments.

Neutrophil to lymphocyte ratio may predict efficacy of anti-PD-1 inhibitors in advanced EGFR-mutant non-small cell lung cancer: retrospective cohort study.

This study aimed to investigate the associations between the clinical characteristics and effectiveness of anti-PD-1 inhibitors in patients with EGFR-sensitive mutations, aiming to identify the potential subgroup of patients who might benefit from anti-PD-1 inhibitor treatment. Patients with advanced non-small cell lung cancer (NSCLC) harboring epidermal growth factor receptor (EGFR)-sensitive mutations who received subsequent anti-PD-1 inhibitors in combination with chemotherapy/antiangiogenic agents or alone after progression to tyrosine kinase inhibitors (TKIs) were screened. Clinical characteristics, including hematological parameters, were investigated for potential correlations with clinical outcomes. Subgroup and multivariate analyses were used for further confirmation of the relationship. Kaplan-Meier curves and Cox survival regression models using the log-rank test were used for progression-free survival (PFS) and overall survival (OS) assessments between the groups. Multiple regression analysis was performed using the standard regression coefficient values. The Wilcoxon test was used for the analysis of the variation in NLR. P ≤ 0.05 was considered to indicate statistical significance. This study was a retrospective study. Twenty-two patients met the inclusion criteria and were included in the study. The median PFS was 3.05 months (95% CI, 2.9-10.2 months). The median OS was 7.30 months (95% CI, 5.2-18.1 months). PFS in low neutrophil to lymphocyte ratio (NLR ≤ 4) was significantly longer than high NLR (NLR > 4, 5.7 months versus 2.0 months, HR, 0.35, 95% CI, 0.08-0.63, P = 0.0083). The OS in the low NLR group was also significantly better than that in the high NLR group (OS, 21.3 months versus 5.0 months, HR, 0.33; 95% CI, 0.09-0.74; P = 0.0163). In the multivariate analysis, NLR was the only significant factor for OS benefits (ß = 3.535, 95% CI, 1.175-10.636, P = 0.025). Further investigation revealed that front-line TKIs exposure may contribute to the elevation or decrease of NLR, and finally lead to different efficacy outcomes by anti-PD-1 inhibitors. The findings suggest that a portion of advanced NSCLC patients with low NLR characteristics (NLR ≤ 4), even those harboring EGFR-sensitive mutations, could benefit from anti-PD-1 inhibitors as further line treatment after progression to EGFR-TKIs.