Investigating keypoint descriptors for camera relocalization in endoscopy surgery.

PURPOSE: Recent advances in computer vision and machine learning have resulted in endoscopic video-based solutions for dense reconstruction of the anatomy. To effectively use these systems in surgical navigation, a reliable image-based technique is required to constantly track the endoscopic camera's position within the anatomy, despite frequent removal and re-insertion. In this work, we investigate the use of recent learning-based keypoint descriptors for six degree-of-freedom camera pose estimation in intraoperative endoscopic sequences and under changes in anatomy due to surgical resection. METHODS: Our method employs a dense structure from motion (SfM) reconstruction of the preoperative anatomy, obtained with a state-of-the-art patient-specific learning-based descriptor. During the reconstruction step, each estimated 3D point is associated with a descriptor. This information is employed in the intraoperative sequences to establish 2D-3D correspondences for Perspective-n-Point (PnP) camera pose estimation. We evaluate this method in six intraoperative sequences that include anatomical modifications obtained from two cadaveric subjects. RESULTS: Show that this approach led to translation and rotation errors of 3.9 mm and 0.2 radians, respectively, with 21.86% of localized cameras averaged over the six sequences. In comparison to an additional learning-based descriptor (HardNet++), the selected descriptor can achieve a better percentage of localized cameras with similar pose estimation performance. We further discussed potential error causes and limitations of the proposed approach. CONCLUSION: Patient-specific learning-based descriptors can relocalize images that are well distributed across the inspected anatomy, even where the anatomy is modified. However, camera relocalization in endoscopic sequences remains a persistently challenging problem, and future research is necessary to increase the robustness and accuracy of this technique.

Mixed Reality Interfaces for Achieving Desired Views with Robotic X-ray Systems.

Robotic X-ray C-arm imaging systems can precisely achieve any position and orientation relative to the patient. Informing the system, however, what pose exactly corresponds to a desired view is challenging. Currently these systems are operated by the surgeon using joysticks, but this interaction paradigm is not necessarily effective because users may be unable to efficiently actuate more than a single axis of the system simultaneously. Moreover, novel robotic imaging systems, such as the Brainlab Loop-X, allow for independent source and detector movements, adding even more complexity. To address this challenge, we consider complementary interfaces for the surgeon to command robotic X-ray systems effectively. Specifically, we consider three interaction paradigms: (1) the use of a pointer to specify the principal ray of the desired view relative to the anatomy, (2) the same pointer, but combined with a mixed reality environment to synchronously render digitally reconstructed radiographs from the tool's pose, and (3) the same mixed reality environment but with a virtual X-ray source instead of the pointer. Initial human-in-the-loop evaluation with an attending trauma surgeon indicates that mixed reality interfaces for robotic X-ray system control are promising and may contribute to substantially reducing the number of X-ray images acquired solely during "fluoro hunting" for the desired view or standard plane.

Nonlinearly scaled prior image-controlled frequency split for high-frequency metal artifact reduction in computed tomography.

PURPOSE: This paper introduces a new approach for the dedicated reduction of high-frequency metal artifacts, which applies a nonlinear scaling (NLS) transfer function on the high-frequency projection domain to reduce artifacts, while preserving edge information and anatomic detail by incorporating prior image information. METHODS: An NLS function is applied to suppress high-frequency streak artifacts, but to restrict the correction to metal projections only, scaling is performed in the sinogram domain. Anatomic information should be preserved and is excluded from scaling by incorporating a prior image from tissue classification. The corrected high-frequency sinogram is reconstructed and combined with the low-frequency component of a normalized metal artifact reduction (NMAR) image. Scans of different anthropomorphic phantoms were acquired (unilateral hip, bilateral hip, dental implants, and embolization coil). Multiple regions of interest (ROIs) were drawn around the metal implants and hounsfield unit (HU) deviations were analyzed. Clinical data sets including single image slices of dental fillings, a bilateral hip implant, spinal fixation screws, and an aneurysm coil were reconstructed and assessed. RESULTS: The prior image-controlled NLS can remove streak artifacts while preserving anatomic detail within the bone and soft tissue. The qualitative analysis of clinical cases showed a tremendous enhancement within dental fillings and neuro coils, and a significant enhancement within spinal screws or hip implants. The phantom scan measurements support this observation. In all phantom setups, the NLS-corrected result showed lowest HU derivation and the best visualization of the data. CONCLUSIONS: The prior image-controlled NLS provides a method to reduce high-frequency streaks in metal-corrupted computed tomography (CT) data.

Embryology of a planktonic tunicate reveals traces of sessility.

A key problem in understanding deuterostome evolution has been the origin of the chordate body plan. A biphasic life cycle with a sessile adult and a free-swimming larva is traditionally considered ancestral in chordates with subsequent neotenic loss of the sessile adult stage. Molecular phylogenies challenged this view, suggesting that the primitive life cycle in chordates was entirely free-living as in modern day larvaceans. Here, we report the precise cell lineage and fate map in the normal embryo of the larvacean Oikopleura dioica, using 4D microscopy technique and transmission electron microscopy. We document the extraordinary rapidity of cleavage and morphogenetic events until hatching and demonstrate that--compared with ascidians--fate restriction occurs considerably earlier in O. dioica and that clonal organization of the cell lineage is more tightly coupled to tissue fate. We show that epidermal cells in the trunk migrate through 90 degrees, reminiscent of events in ascidian metamorphosis and that the axis of bilateral symmetry in the tail rotates in relation to the trunk. We argue that part of the tail muscle cells are ectomesodermal, because they are more closely associated with prospective epidermis than with other tissues in the cell lineage. Cladistic comparison with other deuterostomes suggests that these traits are derived within tunicates strengthening the hypothesis that the last common ancestor of tunicates had a sessile adult and thus support traditional morphology-derived scenarios. Our results allow hypothesizing that molecular developmental mechanisms known from ascidian models are restricted to fewer, yet identifiable, cells in O. dioica.

Thermally induced degradation pathways of three different antibody-based drug development candidates.

Protein-based medicinal products are prone to undergo a variety of chemical and physical degradation pathways. One of the most important exogenous stress condition to consider during manufacturing, transport and storage processes is temperature, because antibody-based therapeutics are only stable in a limited temperature range. In this study, three different formats of antibody-based molecules (IgG1, a bispecific scFv and a fab fragment) were exposed to thermal stress conditions occurring during transport and storage. For evaluation, an analytical platform was developed for the detection and characterization of relevant degradation pathways of different antibody-based therapeutics. The effect of thermal stress conditions on the stability of the three antibody-based formats was therefore investigated using visual inspection, different spectroscopic measurements, dynamic light scattering (DLS), differential scanning calorimetry (DSC), electrophoresis, asymmetric flow field-flow fractionation (AF4) and surface plasmon resonance technology (SPR). In summary, thermal stress led to heterogeneous chemical and physical degradation pathways of all three antibody-based formats used. In addition, identical exogenous stress conditions resulted in different kinds and levels of aggregates and fragmentation products. This knowledge is fundamental for a systematic and successful stabilization of protein-based therapeutics by the use of formulation additives.

Structural and functional characterization of alternative transmembrane domain conformations in VEGF receptor 2 activation.

Transmembrane signaling by receptor tyrosine kinases (RTKs) entails ligand-mediated dimerization and structural rearrangement of the extracellular domains. RTK activation also depends on the specific orientation of the transmembrane domain (TMD) helices, as suggested by pathogenic, constitutively active RTK mutants. Such mutant TMDs carry polar amino acids promoting stable transmembrane helix dimerization, which is essential for kinase activation. We investigated the effect of polar amino acids introduced into the TMD of vascular endothelial growth factor receptor 2, regulating blood vessel homeostasis. Two mutants showed constitutive kinase activity, suggesting that precise TMD orientation is mandatory for kinase activation. Nuclear magnetic resonance spectroscopy revealed that TMD helices in activated constructs were rotated by 180° relative to the interface of the wild-type conformation, confirming that ligand-mediated receptor activation indeed results from transmembrane helix rearrangement. A molecular dynamics simulation confirmed the transmembrane helix arrangement of wild-type and mutant TMDs revealed by nuclear magnetic resonance spectroscopy.

A new bioassay for the immunocytokine L19-IL2 for simultaneous analysis of both functional moieties.

Currently, cancer directed new biological entities (NBEs) in the pharmaceutical R&D pipelines are derived from monoclonal antibodies in various formats, such as immunocytokines. Generally, immunocytokines are bi-functional molecules that consist of a specific targeting antibody-based portion and a linked cytokine. To confirm the quality of the drug product both moieties have to be characterized using appropriate techniques. Until now, the binding capacity of antibodies is usually examined by ligand binding assays whereas the biological activity of the linked cytokine is determined by cell-based potency assays. However, the simultaneous analysis of both functional moieties in a single assay format has not been described so far. In this paper we present a newly designed bioassay format for the anti-cancer immunocytokine L19-IL2, comprising of the human vascular targeting single-chain Fv L19 and human interleukin 2 (IL2). This new potency assay allows simultaneous analysis of both moieties, thus specific L19 binding capacity and the ability of IL2 to induce the proliferation of the detector cytotoxic T-cell line CTLL-2. Assay development was performed with special focus on application of different fitting models for the sigmoid dose-response curves to evaluate the influence of model optimization on the validity of assay results. For assay validation generally accepted characteristics were determined. Assay specificity was shown by testing L19-IL2 related compounds. All other validation parameters were derived from 25 batch runs using five nominal L19-IL2 concentrations, covering a range from 60% to 140% of the standard's potency. Accuracy ranged from -3.4% to -6.9% relative error (%RE). Interbatch precision ranged from 6.1% to 10.6% coefficient of variation (%CV). For assay linearity a coefficient of determination (R(2)) of 0.9992 was found. Assay robustness was shown with L19-IL2 samples after three freeze-thaw cycles and also with different cell passages of the used cytotoxic T-cell line. Based on the data, we conclude that this assay is valid for potency estimation of the immunocytokine L19-IL2. Moreover, this format represents a major improvement compared to other approaches which only allow potency evaluation of both functional moieties in separate assays. In general the underlying assay principle described seems suitable for potency determination of other immunocytokines.