3D reconstruction from endoscopy images: A survey.

Three-dimensional reconstruction of images acquired through endoscopes is playing a vital role in an increasing number of medical applications. Endoscopes used in the clinic are commonly classified as monocular endoscopes and binocular endoscopes. We have reviewed the classification of methods for depth estimation according to the type of endoscope. Basically, depth estimation relies on feature matching of images and multi-view geometry theory. However, these traditional techniques have many problems in the endoscopic environment. With the increasing development of deep learning techniques, there is a growing number of works based on learning methods to address challenges such as inconsistent illumination and texture sparsity. We have reviewed over 170 papers published in the 10 years from 2013 to 2023. The commonly used public datasets and performance metrics are summarized. We also give a taxonomy of methods and analyze the advantages and drawbacks of algorithms. Summary tables and result atlas are listed to facilitate the comparison of qualitative and quantitative performance of different methods in each category. In addition, we summarize commonly used scene representation methods in endoscopy and speculate on the prospects of deep estimation research in medical applications. We also compare the robustness performance, processing time, and scene representation of the methods to facilitate doctors and researchers in selecting appropriate methods based on surgical applications.

The negative effects of the excessive nitrite accumulation raised by anaerobic bioaugmentation on bioremediation of PAH-contaminated soil.

Nitrite accumulation in anaerobic bioaugmentation and its side effects on remediation efficiency of polycyclic aromatic hydrocarbon (PAH)-contaminated soil were investigated in this study. Four gradient doses of PAH-degrading inoculum (10^4, 10^5, 10^6 and 10^7 cells/g soil) were separately supplied to the actual PAH-contaminated soil combining with nitrate as the biostimulant. Although bioaugmented with higher dose of inoculum could effectively improve the biodegradation efficiencies in the initial stage than sole nitrate addition but also accelerated the accumulation of nitrite in soil. The inhibition effects of nitrite swiftly occurred following the rapid accumulation of nitrite in each experiment group, restraining the PAH-degrading functionality by inhibiting the growth of total biomass and denitrifying functions in soil. This study revealed the side effects of nitrite accumulation raised by bioaugmentation on soil microorganisms, contributing to further improving the biodegrading efficiencies in the actual site restoration.

Self-Supervised Lightweight Depth Estimation in Endoscopy Combining CNN and Transformer.

In recent years, an increasing number of medical engineering tasks, such as surgical navigation, pre-operative registration, and surgical robotics, rely on 3D reconstruction techniques. Self-supervised depth estimation has attracted interest in endoscopic scenarios because it does not require ground truth. Most existing methods depend on expanding the size of parameters to improve their performance. There, designing a lightweight self-supervised model that can obtain competitive results is a hot topic. We propose a lightweight network with a tight coupling of convolutional neural network (CNN) and Transformer for depth estimation. Unlike other methods that use CNN and Transformer to extract features separately and then fuse them on the deepest layer, we utilize the modules of CNN and Transformer to extract features at different scales in the encoder. This hierarchical structure leverages the advantages of CNN in texture perception and Transformer in shape extraction. In the same scale of feature extraction, the CNN is used to acquire local features while the Transformer encodes global information. Finally, we add multi-head attention modules to the pose network to improve the accuracy of predicted poses. Experiments demonstrate that our approach obtains comparable results while effectively compressing the model parameters on two datasets.

Augmented-reality-based surgical navigation for endoscope retrograde cholangiopancreatography: A phantom study.

BACKGROUND: Endoscope retrograde cholangiopancreatography is a standard surgical treatment for gallbladder and pancreatic diseases. However, surgeons is at high risk and require sufficient surgical experience and skills. METHODS: (1) The simultaneous localisation and mapping technique to reconstruct the surgical environment. (2) The preoperative 3D model is transformed into the intraoperative video environment to implement the multi-modal fusion. (3) A framework for virtual-to-real projection based on hand-eye alignment. For the purpose of projecting the 3D model onto the imaging plane of the camera, it uses position data from electromagnetic sensors. RESULTS: Our AR-assisted navigation system can accurately guide physicians, which means a distance of registration error to be restricted to under 5 mm and a projection error of 5.76 ± 2.13, and the intubation procedure is done at 30 frames per second. CONCLUSIONS: Coupled with clinical validation and user studies, both the quantitative and qualitative results indicate that our navigation system has the potential to be highly useful in clinical practice.

NEO212, temozolomide conjugated to NEO100, exerts superior therapeutic activity over temozolomide in preclinical chemoradiation models of glioblastoma.

Background: The chemotherapeutic standard of care for patients with glioblastoma (GB) is radiation therapy (RT) combined with temozolomide (TMZ). However, during the twenty years since its introduction, this so-called Stupp protocol has revealed major drawbacks, because nearly half of all GBs harbor intrinsic treatment resistance mechanisms. Prime among these are the increased expression of the DNA repair protein O6-guanine-DNA methyltransferase (MGMT) and cellular deficiency in DNA mismatch repair (MMR). Patients with such tumors receive very little, if any, benefit from TMZ. We are developing a novel molecule, NEO212 (TMZ conjugated to NEO100), that harbors the potential to overcome these limitations. Methods: We used mouse models that were orthotopically implanted with GB cell lines or primary, radioresistant human GB stem cells, representing different treatment resistance mechanisms. Animals received NEO212 (or TMZ for comparison) without or with RT. Overall survival was recorded, and histology studies quantified DNA damage, apoptosis, microvessel density, and impact on bone marrow. Results: In all tumor models, replacing TMZ with NEO212 in a schedule designed to mimic the Stupp protocol achieved a strikingly superior extension of survival, especially in TMZ-resistant and RT-resistant models. While NEO212 displayed pronounced radiation-sensitizing, DNA-damaging, pro-apoptotic, and anti-angiogenic effects in tumor tissue, it did not cause bone marrow toxicity. Conclusions: NEO212 is a candidate drug to potentially replace TMZ within the standard Stupp protocol. It has the potential to become the first chemotherapeutic agent to significantly extend overall survival in TMZ-resistant patients when combined with radiation.

Predicting the effective compressive modulus of human cancellous bone using the convolutional neural network method.

The efficient prediction of biomechanical properties of bone plays an important role in the assessment of bone quality. However, the present techniques are either of low accuracy or of high complexity for the clinical application. The present study aims to investigate the predictive ability of the evolving convolutional neural network (CNN) technique in predicting the effective compressive modulus of porous bone structures. The T11/T12/L1 segments of thirty-five female cadavers were scanned using the HR-pQCT scanner and the images obtained from it were used to generate 10896 2 D bone samples, in which only the cancellous bony parts were processed and investigated. The corresponding 10896 heterogeneous finite-element (FE) models were generated, and then a CNN model was constructed and trained using the predictions of the FE analysis as the ground truths. Then the remaining 260 bone samples generated from the initial HR-pQCT images were used to test the predictive power of the CNN model. The results show that the coefficient of the determinant (R2) from the linear correlation between the CNN and FE predicted elastic modulus is 0.95, which is much higher than that from the correlation between the BMD and the FE predictions (R2 = 0.65). Furthermore, the 95th and 50th percentiles of relative prediction error are below 0.28 and 0.09, respectively. In the conclusion, the CNN model can efficiently predict the effective compressive modulus of human cancellous bone and can be used as a promising and clinically applicable method to evaluate the mechanical quality of porous bone.

Scene-graph-driven semantic feature matching for monocular digestive endoscopy.

BACKGROUND AND OBJECTIVE: Registration of the preoperative 3D model with the video of the digestive tract is the key task in endoscopy surgical navigation. Accurate 3D reconstruction of soft tissue surfaces is essential to complete registration. However, existing feature matching methods still fall short of desirable performance, due to the soft tissue deformation and smooth but less-textured surface. METHODS: In this paper, we present a new semantic description based on the scene graph to integrate contour features and SIFT features. Firstly, we construct the semantic feature descriptor using the SIFT features and dense points in the contour regions to obtain more dense point feature matching. Secondly, we design a clustering algorithm based on the proposed semantic feature descriptor. Finally, we apply the semantic description to the structure from motion (SfM) reconstruction framework. RESULTS: Our techniques are validated by the phantom tests and real surgery videos. We compare our approaches with other typical methods in contour extraction, feature matching, and SfM reconstruction. On average, the feature matching accuracy reaches 75.6% and improves 16.6% in pose estimation. In addition, 39.8% of sparse points are increased in SfM results, and 35.31% more valid points are obtained for the DenseDescriptorNet training in 3D reconstruction. CONCLUSIONS: The new semantic feature description has the potential to reveal more accurate and dense feature correspondence and provides local semantic information in feature matching. Our experiments on the clinical dataset demonstrate the effectiveness and robustness of the novel approach.

Influence of the parameters of the convolutional neural network model in predicting the effective compressive modulus of porous structure.

In recent years, the convolutional neural network (CNN) technique has emerged as an efficient new method for designing porous structure, but a CNN model generally contains a large number of parameters, each of which could influence the predictive ability of the CNN model. Furthermore, there is no consensus on the setting of each parameter in the CNN model. Therefore, the present study aimed to investigate the sensitivity of the parameters in the CNN model for the prediction of the mechanical property of porous structures. 10,500 samples of porous structure were randomly generated, and their effective compressive moduli obtained from finite element analysis were used as the ground truths to construct and train a CNN model. 8,000 of the samples were used to train the CNN model, 2000 samples were used for the cross-validation of the CNN model and the remaining 500 new structures, which did not participate in the CNN training process, were used to test the predictive power of the CNN model. The sensitivity of the number of convolutional layers, the number of convolution kernels, the number of pooling layers, the number of fully connected layers and the optimizer in the CNN model were then investigated. The results showed that the optimizer has the largest influence on the training speed, while the fully connected layer has the least impact on the training speed. Additionally, the pooling layer has the largest impact on the predictive ability while the optimizer has the least impact on the predictive ability. In conclusion, the parameters of the CNN model play an important role in the performance of the CNN model and the parameter sensitivity analysis can help optimize the CNN model to increase the computational efficiency.

Designing anisotropic porous bone scaffolds using a self-learning convolutional neural network model.

The design of bionic bone scaffolds to mimic the behaviors of native bone tissue is crucial in clinical application, but such design is very challenging due to the complex behaviors of native bone tissues. In the present study, bionic bone scaffolds with the anisotropic mechanical properties similar to those of native bone tissues were successfully designed using a novel self-learning convolutional neural network (CNN) framework. The anisotropic mechanical property of bone was first calculated from the CT images of bone tissues. The CNN model constructed was trained and validated using the predictions from the heterogonous finite element (FE) models. The CNN model was then used to design the scaffold with the elasticity matrix matched to that of the replaced bone tissues. For the comparison, the bone scaffold was also designed using the conventional method. The results showed that the mechanical properties of scaffolds designed using the CNN model are closer to those of native bone tissues. In conclusion, the self-learning CNN framework can be used to design the anisotropic bone scaffolds and has a great potential in the clinical application.

NEO212, a Perillyl Alcohol-Temozolomide Conjugate, Triggers Macrophage Differentiation of Acute Myeloid Leukemia Cells and Blocks Their Tumorigenicity.

Many patients with acute myeloid leukemia (AML) are still dying from this disease. In the past, the alkylating agent temozolomide (TMZ) has been investigated for AML and found to be partially effective; however, the presence of O6-methylguanine DNA methyltransferase (MGMT; a DNA repair enzyme) in tumor cells confers profound treatment resistance against TMZ. We are developing a novel anticancer compound, called NEO212, where TMZ was covalently conjugated to perillyl alcohol (a naturally occurring monoterpene). NEO212 has revealed robust therapeutic activity in a variety of preclinical cancer models, including AML. In the current study, we investigated its impact on a panel of human AML cell lines and found that it exerted cytotoxic potency even against MGMT-positive cells that were highly resistant to TMZ. Furthermore, NEO212 strongly stimulated the expression of a large number of macrophage-associated marker genes, including CD11b/ITGAM. This latter effect could not be mimicked when cells were treated with TMZ or an equimolar mix of individual agents, TMZ plus perillyl alcohol. The superior cytotoxic impact of NEO212 appeared to involve down-regulation of MGMT protein levels. In a mouse model implanted with TMZ-resistant, MGMT-positive AML cells, two 5-day cycles of 25 mg/kg NEO212 achieved an apparent cure, as mice survived >300 days without any signs of disease. In parallel toxicity studies with rats, a 5-day cycle of 200 mg/kg NEO212 was well tolerated by these animals, whereas animals that were given 200 mg/kg TMZ all died due to severe leukopenia. Together, our results show that NEO212 exerts pleiotropic effects on AML cells that include differentiation, proliferation arrest, and eventual cell death. In vivo, NEO212 was well tolerated even at dosages that far exceed the therapeutic need, indicating a large therapeutic window. These results present NEO212 as an agent that should be considered for development as a therapeutic agent for AML.

Anaerobic phenanthrene biodegradation by a new salt-tolerant/halophilic and nitrate-reducing Virgibacillus halodenitrificans strain PheN4 and metabolic processes exploration.

The biodegradation of polycyclic aromatic hydrocarbons (PAHs) under hypersaline environments has received increasing attention, whereas the study of anaerobic PAH biodegradation under hypersaline environments is still lacking. Here, we found a pure culture designated PheN4, which was affiliated with Virgibacillus halodenitrificans and could degrade phenanthrene with nitrate as the terminal electron acceptor and a wide range of salinities (from 0.3% to 20%) under anaerobic environments. The optimal salinity for biodegradation of phenanthrene by PheN4 was 5%, which could degrade 93.5% of 0.62 ± 0.04 mM phenanthrene within 10 days with the initial inoculum of 0.01 gVSS/L. Meanwhile, an increased microbial amount could efficiently promote the phenanthrene biodegradation rate. The metabolic processes of anaerobic phenanthrene biodegradation under hypersaline conditions by PheN4 were proposed based on intermediates and genome analyses. Phenanthrene was initially activated via methylation to form 2-methylphenanthrene. Next, fumarate addition and ß-oxidation or direct oxidation of the methyl group, ring reduction and ring cleavage were identified as the midstream and downstream steps. In addition, PheN4 could utilize benzene, naphthalene, and anthracene as carbon sources, but Benz[a]anthracene, pyrene, and Benzo[a]pyrene could not be biodegraded by PheN4. This study could provide some guidance for the bioremediation of PAH pollutants in anaerobic and hypersaline zones.

Comparison of the design maps of TPMS based bone scaffolds using a computational modeling framework simultaneously considering various conditions.

In recent years, the triply periodic minimal surface (TPMS)-based scaffolds have been served as one of the crucial types of structures for biological replacements, the energy absorber, etc. Meanwhile, the development of additive manufacturing (AM) has facilitated the production of TPMS scaffolds with complex microstructures. However, the design maps of TPMS scaffolds, especially considering the AM constraints, remain unclear, which has hindered the design and application of TPMS scaffolds. The aims of the present study were to develop an efficient computational modeling framework for investigating the design maps of TPMS scaffolds simultaneously considering the AM constraints, the biological requirements, and the structural anisotropy. To demonstrate the computational framework, five widely-used topologies of the TPMS-based scaffolds (i.e. the Diamond, the Gyroid, the Fischer-Koch S, the F-RD, and the Schwarz P) were used, whose design maps for the surface-to-volume ratio and the effective elastic modulus were also investigated. The results showed that as the porosities increase, the design ranges of the surface-to-volume ratios decreases for all the structures. Compared with the effect of the constraint for the pore size, the minimal structural thickness for AM constraint has a greater effect on the surface-to-volume ratio. Regarding the elastic modulus, in the region of low porosity (approximately 0.5-0.7), the range for the effective elastic modulus of Schwarz P is the widest (approximately 2.24-32.6 GPa), but the Gyroid can achieve both high porosity and low effective elastic modulus (e.g. 0.61 GPa at the porosity of 0.90). These results and the method developed in the present study provided important basis and guidance for the design and application of the TPMS-based porous structures.

Relationship between the morphological, mechanical and permeability properties of porous bone scaffolds and the underlying microstructure.

Bone scaffolds are widely used as one of the main bone substitute materials. However, many bone scaffold microstructure topologies exist and it is still unclear which topology to use when designing scaffold for a specific application. The aim of the present study was to reveal the mechanism of the microstructure-driven performance of bone scaffold and thus to provide guideline on scaffold design. Finite element (FE) models of five TPMS (Diamond, Gyroid, Schwarz P, Fischer-Koch S and F-RD) and three traditional (Cube, FD-Cube and Octa) scaffolds were generated. The effective compressive and shear moduli of scaffolds were calculated from the mechanical analysis using the FE unit cell models with the periodic boundary condition. The scaffold permeability was calculated from the computational fluid dynamics (CFD) analysis using the 4×4×4 FE models. It is revealed that the surface-to-volume ratio of the Fischer-Koch S-based scaffold is the highest among the scaffolds investigated. The mechanical analysis revealed that the bending deformation dominated structures (e.g., the Diamond, the Gyroid, the Schwarz P) have higher effective shear moduli. The stretching deformation dominated structures (e.g., the Schwarz P, the Cube) have higher effective compressive moduli. For all the scaffolds, when the same amount of change in scaffold porosity is made, the corresponding change in the scaffold relative shear modulus is larger than that in the relative compressive modulus. The CFD analysis revealed that the structures with the simple and straight pores (e.g., Cube) have higher permeability than the structures with the complex pores (e.g., Fischer-Koch S). The main contribution of the present study is that the relationship between scaffold properties and the underlying microstructure is systematically investigated and thus some guidelines on the design of bone scaffolds are provided, for example, in the scenario where a high surface-to-volume ratio is required, it is suggested to use the Fischer-Koch S based scaffold.

Quantifying the discrepancies in the geometric and mechanical properties of the theoretically designed and additively manufactured scaffolds.

In recent years, the triply periodic minimal surface (TPMS) has emerged as a new method for producing open cell porous scaffolds because of the superior properties, such as the high surface-to-volume ratio, the zero curvature, etc. On the other hand, the additive manufacturing (AM) technique has made feasible the design and development of TPMS scaffolds with complex microstructures. However, neither the discrepancy between the theoretically designed and the additively manufactured TPMS scaffolds nor the underlying mechanisms is clear so far. The aims of the present study were to quantify the discrepancies between the theoretically designed and the AM produced TPMS scaffolds and to reveal the underlying mechanisms, e.g., the effect of building orientation on the discrepancy. 24 Gyroid scaffolds were produced along the height and width directions of the scaffold using the selective laser melting (SLM) technique (i.e., 12 scaffolds produced in each direction). The discrepancies in the geometric and mechanical properties of the TPMS scaffolds were quantified. Regarding the geometric properties, the discrepancies in the porosity, the dimension and the three-dimensional (3D) geometry of the scaffolds were quantified. Regarding the mechanical properties, the discrepancies in the effective compressive modulus and the mechanical environment (strain energy density) of the scaffolds were evaluated. It is revealed that the porosity in the AM produced scaffold is approximately 12% lower than the designed value. There are approximately 68.1 ± 8.6% added materials in the AM produced scaffolds and the added materials are mostly distributed in the places opposite to the building orientation. The building orientation has no effect on the discrepancy in the scaffold porosity and no effect on the distribution of the added materials (p > 0.05). Regarding the mechanical properties, the compressive moduli of the scaffolds are 24.4% (produced along the height direction) and 14.6% (produced along the width direction) lower than the designed value and are 49.1% and 43.6% lower than the µFE counterparts, indicating that the imperfect bonding and the partially melted powders have a large contribution to the discrepancy in the compressive modulus of the scaffolds. Compared to the values in the theoretically designed scaffold, the strain energy densities have shifted towards the higher values in the AM produced scaffolds. The findings in the present study provide important information for the design and additive manufacturing of TPMS scaffolds.

Comparison of Biomechanical Performance of Five Different Treatment Approaches for Fixing Posterior Pelvic Ring Injury.

Background: A large number of pelvic injuries are seriously unstable, with mortality rates reaching 19%. Approximately 60% of pelvic injuries are related to the posterior pelvic ring. However, the selection of a fixation method for a posterior pelvic ring injury remains a challenging problem for orthopedic surgeons. The aim of the present study is to investigate the biomechanical performance of five different fixation approaches for posterior pelvic ring injury and thus provide guidance on the choice of treatment approach in a clinical setting. Methods: A finite element (FE) model, including the L3-L5 lumbar vertebrae, sacrum, and full pelvis, was created from CT images of a healthy adult. Tile B and Tile C types of pelvic fractures were created in the model. Five different fixation methods for fixing the posterior ring injury (PRI) were simulated: TA1 (conservative treatment), TA2 (S1 screw fixation), TA3 (S1 + S2 screw fixation), TA4 (plate fixation), and TA5 (modified triangular osteosynthesis). Based on the fixation status (fixed or nonfixed) of the anterior ring and the fixation method for PRI, 20 different FE models were created. An upright standing loading scenario was simulated, and the resultant displacements at the sacroiliac joint were compared between different models. Results: When TA5 was applied, the resultant displacements at the sacroiliac joint were the smallest (1.5 mm, 1.6 mm, 1.6 mm, and 1.7 mm) for all the injury cases. The displacements induced by TA3 and TA2 were similar to those induced by TA5. TA4 led to larger displacements at the sacroiliac joint (2.3 mm, 2.4 mm, 4.8 mm, and 4.9 mm), and TA1 was the worst case (3.1 mm, 3.2 mm, 6.3 mm, and 6.5 mm). Conclusions: The best internal fixation method for PRI is the triangular osteosynthesis approach (TA5), followed by S1 + S2 screw fixation (TA3), S1 screw fixation (TA2), and plate fixation (TA4).

A critical review on the three-dimensional finite element modelling of the compression therapy for chronic venous insufficiency.

Compression therapy is an adjuvant physical intervention providing the benefits of calibrated compression and controlled stretch and consequently is increasingly applied for the treatment of chronic venous insufficiency. However, the mechanism of the compression therapy for chronic venous insufficiency is still unclear. To elaborate the mechanism of compression therapy, in recent years, the computational modelling technique, especially the finite element modelling method, has been widely used. However, there are still many unclear issues regarding the finite element modelling of compression therapy, for example, the selection of appropriate material models, the validation of the finite element predictions, the post-processing of the results. To shed light on these unclear issues, this study provides a state-of-the-art review on the application of finite element modelling technique in the compression therapy for chronic venous insufficiency. The aims of the present study are as follows: (1) to provide guidance on the application of the finite element technique in healthcare and relevant fields, (2) to enhance the understanding of the mechanism of compression therapy and (3) to foster the collaborations among different disciplines. To achieve these aims, the following parts are reviewed: (1) the background on chronic venous insufficiency and the computational modelling approach, (2) the acquisition of medical images and the procedure for generating the finite element model, (3) the definition of material models in the finite element model, (4) the methods for validating the finite element predictions, (5) the post-processing of the finite element results and (6) future challenges in the finite element modelling of compression therapy.

Total glucosides of paeony attenuates animal psoriasis induced inflammatory response through inhibiting STAT1 and STAT3 phosphorylation.

ETHNOPHARMACOLOGICAL RELEVANCE: Psoriasis is an immune system meditated disease, especially T cells. It disturbed many people around the world and hard to therapy. Paeonia lactiflora Pall has been used as a medicine in china for thousands of years. Recent studies found that the main component of Paeonia lactiflora Pall can alleviates the immune response in many diseases. In this study, we researched the effects and possible mechanisms of total glucosides of paeony (TGP) on animal psoriasis. AIM OF THE STUDY: To study the therapeutic effects and mechanisms of TGP in 5% propranolol cream-induced psoriasis in guinea pigs and Imiquimod (IMQ) cream-induced psoriasis in mice. MATERIALS AND METHODS: The effect of TGP was evaluated using a psoriasis-like model of guinea pigs and mice. Ear thickness was accessed, and pathology injury was observed by H&E staining. The levels of serum IL-1ß, IL-6, IL-12, IL-17, IL-23, TNF-α, and IFN-γ, skin IL-17A, IL-22 and orphan nuclear receptor (RORγt) mRNA expression, proliferating cell nuclear antigen (PCNA), total or phosphorylated signal transducers and activators of transcription (STAT1, STAT3) were determined by enzyme linked immunosorbent assays (ELISAs), real time PCR, immunohistochemical staining, and western blotting, respectively. RESULTS: Compared with model group, TGP treatment decreased the ear thickness, improved pathology of psoriasis, alleviated IMQ-induced keratinocyte proliferation, reduced the inflammatory cytokine, and downregulated IL-17A, IL-22, and RORγt mRNA in mice. Further study indicated that TGP inhibited STAT1 and STAT3 phosphorylation in lesion skins of psoriasis-like mice. CONCLUSIONS: TGP alleviates the symptoms of psoriasis-like guinea pigs and mice, and the possible mechanism may relate to inhibit T helper 17 (TH17) cell differentiation and keratinocytes proliferation by inhibiting STAT1 and STAT3 phosphorylation.