Divergent G-protein selectivity across melanopsins from mice and humans.

Melanopsin is an opsin photopigment and light-activated G-protein-coupled receptor; it is expressed in photoreceptive retinal ganglion cells (mRGCs) and can be employed as an optogenetic tool. Mammalian melanopsins can signal via Gq/11 and Gi/o/t heterotrimeric G proteins, but aspects of the mRGC light response appear incompatible with either mode of signalling. We use live-cell reporter assays in HEK293T cells to show that melanopsins from mice and humans can also signal via Gs. We subsequently show that this mode of signalling is substantially divergent between species. The two established structural isoforms of mouse melanopsin (which differ in the length of their C-terminal tail) both signalled strongly through all three G-protein classes (Gq/11, Gi/o and Gs), whereas human melanopsin showed weaker signalling through Gs. Our data identify Gs as a new mode of signalling for mammalian melanopsins and reveal diversity in G-protein selectivity across mammalian melanopsins.

The mouse suprachiasmatic nucleus encodes irradiance via a diverse population of neurons monotonically tuned to different ranges of intensity.

Many neurons of the mammalian master circadian oscillator in the suprachiasmatic nuclei (SCN) respond to light pulses with irradiance-dependent changes in firing. Here, we set out to better understand this irradiance coding ability by considering how the SCN tracks more continuous changes in irradiance at both population and single unit level. To this end, we recorded extracellular activity in the SCN of anaesthetised mice presented with up + down irradiance staircase stimuli covering moonlight to daylight conditions and incorporating epochs with steady light or superimposed higher frequency modulations (temporal white noise (WN) and frequency/contrast chirps). Single unit activity was extracted by spike sorting. The population response of SCN units to this stimulus was a progressive increase in firing rate at higher irradiances. This relationship was symmetrical for up vs. down phases of the ramp in the presence of white noise or chirps but exhibited hysteresis for steady light, with firing systematically higher during increasing irradiance. Single units also showed a monotonic relationship between firing and irradiance but exhibited diversity not only in response polarity (increases vs. decreases in firing), but also in the sensitivity (EC50 ) and slope of fitted functions. These data show that individual SCN neurons exhibit monotonic relationships between irradiance and firing rate but differ in the irradiance range over which they respond. This property may help the SCN to encode the large differences in irradiance found in nature using neurons with a constrained range of firing rates. KEY POINTS: Daily changes in environmental light (irradiance) entrain the suprachiasmatic nucleus (SCN) circadian clock. The mouse SCN shows graded increases in neurophysiological activity with light pulses of increasing irradiance. We show that this monotonic relationship between firing rate and irradiance is retained at population and single unit level when probed with more naturalistic staircase increases and decreases in irradiance. The irradiance response is more reliable in the presence of ongoing higher temporal frequency modulations in light intensity than under steady light. Single units varied in sensitivity allowing the population to cover a wide range of irradiances. Irradiance coding in the SCN has characteristics of a sparse code with individual neurons tracking different portions of the natural irradiance range. This property may address the challenge of encoding a 109 -fold day:night difference in irradiance within the constrained range of firing rates available to individual neurons.

Modulating signalling lifetime to optimise a prototypical animal opsin for optogenetic applications.

Animal opsins are light activated G-protein-coupled receptors, capable of optogenetic control of G-protein signalling for research or therapeutic applications. Animal opsins offer excellent photosensitivity, but their temporal resolution can be limited by long photoresponse duration when expressed outside their native cellular environment. Here, we explore methods for addressing this limitation for a prototypical animal opsin (human rod opsin) in HEK293T cells. We find that the application of the canonical rhodopsin kinase (GRK1)/visual arrestin signal termination mechanism to this problem is complicated by a generalised suppressive effect of GRK1 expression. This attenuation can be overcome using phosphorylation-independent mutants of arrestin, especially when these are tethered to the opsin protein. We further show that point mutations targeting the Schiff base stability of the opsin can also reduce signalling lifetime. Finally, we apply one such mutation (E122Q) to improve the temporal fidelity of restored visual responses following ectopic opsin expression in the inner retina of a mouse model of retinal degeneration (rd1). Our results reveal that these two strategies (targeting either arrestin binding or Schiff-base hydrolysis) can produce more time-delimited opsin signalling under heterologous expression and establish the potential of this approach to improve optogenetic performance.

Using a bistable animal opsin for switchable and scalable optogenetic inhibition of neurons.

There is no consensus on the best inhibitory optogenetic tool. Since Gi/o signalling is a native mechanism of neuronal inhibition, we asked whether Lamprey Parapinopsin ("Lamplight"), a Gi/o-coupled bistable animal opsin, could be used for optogenetic silencing. We show that short (405 nm) and long (525 nm) wavelength pulses repeatedly switch Lamplight between stable signalling active and inactive states, respectively, and that combining these wavelengths can be used to achieve intermediate levels of activity. These properties can be applied to produce switchable neuronal hyperpolarisation and suppression of spontaneous spike firing in the mouse hypothalamic suprachiasmatic nucleus. Expressing Lamplight in (predominantly) ON bipolar cells can photosensitise retinas following advanced photoreceptor degeneration, with 405 and 525 nm stimuli producing responses of opposite sign in the output neurons of the retina. We conclude that bistable animal opsins can co-opt endogenous signalling mechanisms to allow optogenetic inhibition that is scalable, sustained and reversible.

Sensor-Based Automated Detection of Electrosurgical Cautery States.

In computer-assisted surgery, it is typically required to detect when the tool comes into contact with the patient. In activated electrosurgery, this is known as the energy event. By continuously tracking the electrosurgical tools' location using a navigation system, energy events can help determine locations of sensor-classified tissues. Our objective was to detect the energy event and determine the settings of electrosurgical cautery-robustly and automatically based on sensor data. This study aims to demonstrate the feasibility of using the cautery state to detect surgical incisions, without disrupting the surgical workflow. We detected current changes in the wires of the cautery device and grounding pad using non-invasive current sensors and an oscilloscope. An open-source software was implemented to apply machine learning on sensor data to detect energy events and cautery settings. Our methods classified each cautery state at an average accuracy of 95.56% across different tissue types and energy level parameters altered by surgeons during an operation. Our results demonstrate the feasibility of automatically identifying energy events during surgical incisions, which could be an important safety feature in robotic and computer-integrated surgery. This study provides a key step towards locating tissue classifications during breast cancer operations and reducing the rate of positive margins.

Chimeric human opsins as optogenetic light sensitisers.

Human opsin-based photopigments have great potential as light-sensitisers, but their requirement for phototransduction cascade-specific second messenger proteins may restrict their functionality in non-native cell types. In this study, eight chimeric human opsins were generated consisting of a backbone of either a rhodopsin (RHO) or long-wavelength-sensitive (LWS) opsin and intracellular domains from Gq/11-coupled human melanopsin. Rhodopsin/melanopsin chimeric opsins coupled to both Gi and Gq/11 pathways. Greater substitution of the intracellular surface with corresponding melanopsin domains generally showed greater Gq/11 activity with a decrease in Gi activation. Unlike melanopsin, rhodopsin and rhodopsin/melanopsin chimeras were dependent upon exogenous chromophore to function. By contrast, wild-type LWS opsin and LWS opsin/melanopsin chimeras showed only weak Gi activation in response to light, whilst Gq/11 pathway activation was not detected. Immunocytochemistry (ICC) demonstrated that chimeric opsins with more intracellular domains of melanopsin were less likely to be trafficked to the plasma membrane. This study demonstrates the importance of Gα coupling efficiency to the speed of cellular responses and created human opsins with a unique combination of properties to expand the range of customised optogenetic biotools for basic research and translational therapies.

Viral Transduction of Human Rod Opsin or Channelrhodopsin Variants to Mouse ON Bipolar Cells Does Not Impact Retinal Anatomy or Cause Measurable Death in the Targeted Cells.

The viral gene delivery of optogenetic actuators to the surviving inner retina has been proposed as a strategy for restoring vision in advanced retinal degeneration. We investigated the safety of ectopic expression of human rod opsin (hRHO), and two channelrhodopsins (enhanced sensitivity CoChR-3M and red-shifted ReaChR) by viral gene delivery in ON bipolar cells of the mouse retina. Adult Grm6Cre mice were bred to be retinally degenerate or non-retinally degenerate (homozygous and heterozygous for the rd1Pde6b mutation, respectively) and intravitreally injected with recombinant adeno-associated virus AAV2/2(quad Y-F) serotype containing a double-floxed inverted transgene comprising one of the opsins of interest under a CMV promoter. None of the opsins investigated caused changes in retinal thickness; induced apoptosis in the retina or in transgene expressing cells; or reduced expression of PKCα (a specific bipolar cell marker). No increase in retinal inflammation at the level of gene expression (IBA1/AIF1) was found within the treated mice compared to controls. The expression of hRHO, CoChR or ReaChR under a strong constitutive promoter in retinal ON bipolar cells following intravitreal delivery via AAV2 does not cause either gross changes in retinal health, or have a measurable impact on the survival of targeted cells.

Defining the impact of melanopsin missense polymorphisms using in vivo functional rescue.

Melanopsin (OPN4) is an opsin photopigment expressed within intrinsically photosensitive retinal ganglion cells (ipRGCs) that mediate non-image forming (NIF) responses to light. Two single-nucleotide polymorphisms (SNPs) in human melanopsin (hOPN4), Pro10Leu and Thr394Ile, have recently been associated with abnormal NIF responses to light, including seasonal affective disorder. It has been suggested these behavioural changes are due to altered melanopsin signalling. However, there is currently no direct evidence to support this. Here we have used ipRGC-specific delivery of hOPN4 wild-type (WT), Pro10Leu or Thr394Ile adeno-associated viruses (AAV) to determine the functional consequences of hOPN4 SNPs on melanopsin-driven light responses and associated behaviours. Immunohistochemistry confirmed hOPN4 AAVs exclusively transduced mouse ipRGCs. Behavioural phenotyping performed before and after AAV injection demonstrated that both hOPN4 Pro10Leu and Thr394Ile could functionally rescue pupillary light responses and circadian photoentrainment in Opn4-/- mice, with no differences in NIF behaviours detected for animals expressing either SNP compared to hOPN4 WT. Multi-electrode array recordings revealed that ipRGCs expressing hOPN4 Thr394Ile exhibit melanopsin-driven light responses with significantly attenuated response amplitude, decreased sensitivity and faster offset kinetics compared to hOPN4 WT. IpRGCs expressing hOpn4 Pro10Leu also showed reduced response amplitude. Collectively these data suggest Thr394Ile and Pro10Leu may be functionally significant SNPs, which result in altered melanopsin signalling. To our knowledge, this study provides the first direct evidence for the effects of hOPN4 polymorphisms on melanopsin-driven light responses and NIF behaviours in vivo, providing further insight into the role of these SNPs in melanopsin function and human physiology.

Spectral sensitivity of cone vision in the diurnal murid Rhabdomys pumilio.

An animal's temporal niche - the time of day at which it is active - is known to drive a variety of adaptations in the visual system. These include variations in the topography, spectral sensitivity and density of retinal photoreceptors, and changes in the eye's gross anatomy and spectral transmission characteristics. We have characterised visual spectral sensitivity in the murid rodent Rhabdomys pumilio (the four-striped grass mouse), which is in the same family as (nocturnal) mice and rats but exhibits a strong diurnal niche. As is common in diurnal species, the R. pumilio lens acts as a long-pass spectral filter, providing limited transmission of light <400 nm. Conversely, we found strong sequence homologies with the R. pumilio SWS and MWS opsins and those of related nocturnal species (mice and rats) whose SWS opsins are maximally sensitive in the near-UV. We continued to assess in vivo spectral sensitivity of cone vision using electroretinography and multi-channel recordings from the visual thalamus. These revealed that responses across the human visible range could be adequately described by those of a single pigment (assumed to be MWS opsin) maximally sensitive at ∼500 nm, but that sensitivity in the near-UV required inclusion of a second pigment whose peak sensitivity lay well into the UV range (λmax<400 nm, probably ∼360 nm). We therefore conclude that, despite the UV-filtering effects of the lens, R. pumilio retains an SWS pigment with a UV-A λmax In effect, this somewhat paradoxical combination of long-pass lens and UV-A λmax results in narrow-band sensitivity for SWS cone pathways in the UV-A range.

Functional characterisation of naturally occurring mutations in human melanopsin.

Melanopsin is a blue light-sensitive opsin photopigment involved in a range of non-image forming behaviours, including circadian photoentrainment and the pupil light response. Many naturally occurring genetic variants exist within the human melanopsin gene (OPN4), yet it remains unclear how these variants affect melanopsin protein function and downstream physiological responses to light. Here, we have used bioinformatic analysis and in vitro expression systems to determine the functional phenotypes of missense human OPN4 variants. From 1242 human OPN4 variants collated in the NCBI Short Genetic Variation database (dbSNP), we identified 96 that lead to non-synonymous amino acid substitutions. These 96 missense mutations were screened using sequence alignment and comparative approaches to select 16 potentially deleterious variants for functional characterisation using calcium imaging of melanopsin-driven light responses in HEK293T cells. We identify several previously uncharacterised OPN4 mutations with altered functional properties, including attenuated or abolished light responses, as well as variants demonstrating abnormal response kinetics. These data provide valuable insight into the structure-function relationships of human melanopsin, including several key functional residues of the melanopsin protein. The identification of melanopsin variants with significantly altered function may serve to detect individuals with disrupted melanopsin-based light perception, and potentially highlight those at increased risk of sleep disturbance, circadian dysfunction, and visual abnormalities.

A live cell assay of GPCR coupling allows identification of optogenetic tools for controlling Go and Gi signaling.

BACKGROUND: Animal opsins are light-sensitive G-protein-coupled receptors (GPCRs) that enable optogenetic control over the major heterotrimeric G-protein signaling pathways in animal cells. As such, opsins have potential applications in both biomedical research and therapy. Selecting the opsin with the best balance of activity and selectivity for a given application requires knowing their ability to couple to a full range of relevant Gα subunits. We present the GsX assay, a set of tools based on chimeric Gs subunits that transduce coupling of opsins to diverse G proteins into increases in cAMP levels,  measured with a real-time reporter in living cells. We use this assay to compare coupling to Gi/o/t across a panel of natural and chimeric opsins selected for potential application in gene therapy for retinal degeneration. RESULTS: Of the opsins tested, wild-type human rod opsin had the highest activity for chimeric Gs proxies for Gi and Gt (Gsi and Gst) and was matched in Go proxy (Gso) activity only by a human rod opsin/scallop opsin chimera. Rod opsin drove roughly equivalent responses via Gsi, Gso, and Gst, while cone opsins showed much lower activities with Gso than Gsi or Gst, and a human rod opsin/amphioxus opsin chimera demonstrated higher activity with Gso than with Gsi or Gst. We failed to detect activity for opsin chimeras bearing three intracellular fragments of mGluR6, and observed unexpectedly complex response profiles for scallop and amphioxus opsins thought to be specialized for Go. CONCLUSIONS: These results identify rod opsin as the most potent non-selective Gi/o/t-coupled opsin, long-wave sensitive cone opsin as the best for selectively activating Gi/t over Go, and a rod opsin/amphioxus opsin chimera as the best choice for selectively activating Go over Gi/t.

The hypothalamic photoreceptors regulating seasonal reproduction in birds: a prime role for VA opsin.

Extraretinal photoreceptors located within the medio-basal hypothalamus regulate the photoperiodic control of seasonal reproduction in birds. An action spectrum for this response describes an opsin photopigment with a λmax of ∼ 492 nm. Beyond this however, the specific identity of the photopigment remains unresolved. Several candidates have emerged including rod-opsin; melanopsin (OPN4); neuropsin (OPN5); and vertebrate ancient (VA) opsin. These contenders are evaluated against key criteria used routinely in photobiology to link orphan photopigments to specific biological responses. To date, only VA opsin can easily satisfy all criteria and we propose that this photopigment represents the prime candidate for encoding daylength and driving seasonal breeding in birds. We also show that VA opsin is co-expressed with both gonadotropin-releasing hormone (GnRH) and arginine-vasotocin (AVT) neurons. These new data suggest that GnRH and AVT neurosecretory pathways are endogenously photosensitive and that our current understanding of how these systems are regulated will require substantial revision.

On the role of suppression in spatial attention: evidence from negative BOLD in human subcortical and cortical structures.

There is clear evidence that spatial attention increases neural responses to attended stimuli in extrastriate visual areas and, to a lesser degree, in earlier visual areas. Other evidence shows that neurons representing unattended locations can also be suppressed. However, the extent to which enhancement and suppression is observed, their stimulus dependence, and the stages of the visual system at which they are expressed remains poorly understood. Using fMRI we set out to characterize both the task and stimulus dependence of neural responses in the lateral geniculate nucleus (LGN), primary visual cortex (V1), and visual motion area (V5) in humans to determine where suppressive and facilitatory effects of spatial attention are expressed. Subjects viewed a lateralized drifting grating stimulus, presented at multiple stimulus contrasts, and performed one of three tasks designed to alter the spatial location of their attention. In retinotopic representations of the stimulus location, we observed increasing attention-dependent facilitation and decreasing dependence on stimulus contrast moving up the visual hierarchy from the LGN to V5. However, in the representations of unattended locations of the LGN and V1, we observed suppression, which was not significantly dependent on the attended stimulus contrast. These suppressive effects were also found in the pulvinar, which has been frequently associated with attention. We provide evidence, therefore, for a spatially selective suppressive mechanism that acts at a subcortical level.

Reconfiguration of the visual code and retinal cell type complement in closely related diurnal and nocturnal mice.

How does evolution act on neuronal populations to match computational characteristics to functional demands? We address this problem by comparing visual code and retinal cell composition in closely related murid species with different behaviours. Rhabdomys pumilio are diurnal and have substantially thicker inner retina and larger visual thalamus than nocturnal Mus musculus. High-density electrophysiological recordings of visual response features in the dorsal lateral geniculate nucleus (dLGN) reveals that Rhabdomys attains higher spatiotemporal acuity both by denser coverage of the visual scene and a selective expansion of elements of the code characterised by non-linear spatiotemporal summation. Comparative analysis of single cell transcriptomic cell atlases reveals that realignment of the visual code is associated with increased relative abundance of bipolar and ganglion cell types supporting OFF and ON-OFF responses. These findings demonstrate how changes in retinal cell complement can reconfigure the coding of visual information to match changes in visual needs.

Functional integrity of visual coding following advanced photoreceptor degeneration.

Photoreceptor degeneration sufficient to produce severe visual loss often spares the inner retina. This raises hope for vision restoration treatments using optogenetics or electrical stimulation, which generate a replacement light input signal in surviving neurons. The success of these approaches is dependent on the capacity of surviving circuits of the visual system to generate and propagate an appropriate visual code in the face of neuroanatomical remodeling. To determine whether retinally degenerate animals possess this capacity, we generated a transgenic mouse model expressing the optogenetic actuator ReaChR in ON bipolar cells (second-order neurons in the visual projection). After crossing this with the rd1 model of photoreceptor degeneration, we compared ReaChR-derived responses with photoreceptor-driven responses in wild-type (WT) mice at the level of retinal ganglion cells and the visual thalamus. The ReaChR-driven responses in rd1 animals showed low photosensitivity, but in other respects generated a visual code that was very similar to the WT. ReaChR rd1 responses had high trial-to-trial reproducibility and showed sensitivity normalization to code contrast across background intensities. At the single unit level, ReaChR-derived responses exhibited broadly similar variations in response polarity, contrast sensitivity, and temporal frequency tuning as the WT. Units from the WT and ReaChR rd1 mice clustered together when subjected to unsupervised community detection based on stimulus-response properties. Our data reveal an impressive ability for surviving circuitry to recreate a rich visual code following advanced retinal degeneration and are promising for regenerative medicine in the central nervous system.

Quantitative characterisation of ipRGCs in retinal degeneration using a computation platform for extracting and reconstructing single neurons in 3D from a multi-colour labeled population.

Light has a profound impact on mammalian physiology and behavior. Intrinsically photosensitive retinal ganglion cells (ipRGCs) express the photopigment melanopsin, rendering them sensitive to light, and are involved in both image-forming vision and non-image forming responses to light such as circadian photo-entrainment and the pupillary light reflex. Following outer photoreceptor degeneration, the death of rod and cone photoreceptors results in global re-modeling of the remnant neural retina. Although ipRGCs can continue signaling light information to the brain even in advanced stages of degeneration, it is unknown if all six morphologically distinct subtypes survive, or how their dendritic architecture may be affected. To answer these questions, we generated a computational platform-BRIAN (Brainbow Analysis of individual Neurons) to analyze Brainbow labeled tissues by allowing objective identification of voxels clusters in Principal Component Space, and their subsequent extraction to produce 3D images of single neurons suitable for analysis with existing tracing technology. We show that BRIAN can efficiently recreate single neurons or individual axonal projections from densely labeled tissue with sufficient anatomical resolution for subtype quantitative classification. We apply this tool to generate quantitative morphological information about ipRGCs in the degenerate retina including soma size, dendritic field size, dendritic complexity, and stratification. Using this information, we were able to identify cells whose characteristics match those reported for all six defined subtypes of ipRGC in the wildtype mouse retina (M1-M6), including the rare and complex M3 and M6 subtypes. This indicates that ipRGCs survive outer retinal degeneration with broadly normal morphology. We additionally describe one cell in the degenerate retina which matches the description of the Gigantic M1 cell in Humans which has not been previously identified in rodent.

Dosimetry of a sonolucent material for an ultrasound-compatible gynecologic high-dose-rate brachytherapy cylinder using Monte Carlo simulation and radiochromic film.

PURPOSE: he purpose of this study was to study the dosimetric characterization of sonolucent material "TPX" to be used toward gynecologic high-dose-rate brachytherapy treatments using ultrasound-compatible cylinders in non-model-based dose calculation workflows. METHODS: Monte Carlo simulations were performed using EGSnrc application egs_brachy in cylinders of polymethylpentene (TPX) plastic, water, and PMMA. Simulations were performed of five 192Ir sources placed longitudinally in ∼3.7 cm diameter, 5.0 cm length cylinders (matching physical cylinders used in film measurements). TPX and PMMA dose distributions and percentage depth dose curves were compared relative to water. Film measurements were performed to validate egs_brachy simulations. TPX and PMMA cylinders were placed in a water tank using 3D-printed supports to position film radially and touching the surface of the cylinders. The same five 192Ir dwell positions were delivered as simulated in egs_brachy. RESULTS: The egs_brachy and film percentage depth doses agreed within film uncertainties. The egs_brachy relative dose difference between TPX and water was (0.74 ± 0.09)% and between PMMA and water was (-0.79 ± 0.09)% over the dose scoring phantom. Dose differences for TPX and PMMA relative to water were less than ± 1% within 5 cm of the cylinder surface. CONCLUSIONS: In a solid sonolucent sheath of TPX, the dosimetric differences are comparable with PMMA and other applicator materials in clinical use. No additional uncertainty to dose calculation is introduced when treating through TPX cylinders compared with current applicator materials, and therefore, it is acceptable to perform gynecologic brachytherapy treatments with a sonolucent sheath inserted during radiation delivery.

Feasibility of fusing three-dimensional transabdominal and transrectal ultrasound images for comprehensive intraoperative visualization of gynecologic brachytherapy applicators.

PURPOSE: In this study, we propose combining three-dimensional (3D) transrectal ultrasound (TRUS) and 3D transabdominal ultrasound (TAUS) images of gynecologic brachytherapy applicators to leverage the advantages of each imaging perspective, providing a broader field-of-view and allowing previously obscured features to be recovered. The aim of this study was to evaluate the feasibility of fusing these 3D ultrasound (US) perspectives based on the applicator geometry in a phantom prior to clinical implementation. METHODS: In proof-of-concept experiments, 3D US images of application-specific multimodality pelvic phantoms were acquired with tandem-and-ring and tandem-and-ovoids applicators using previously validated imaging systems. Two TRUS images were acquired at different insertion depths and manually fused based on the position of the ring/ovoids to broaden the TRUS field-of-view. The phantom design allowed "abdominal thickness" to be modified to represent different body habitus and TAUS images were acquired at three thicknesses for each applicator. The merged TRUS images were then combined with TAUS images by rigidly aligning applicator components and manually refining the registration using the positions of source channels and known tandem length, as well as the ring diameter for the tandem-and-ring applicator. Combined 3D US images were manually, rigidly registered to images from a second modality (magnetic resonance (MR) imaging for the tandem-and-ring applicator and X-ray computed tomography (CT) for the tandem-and-ovoids applicator (based on applicator compatibility)) to assess alignment. Four spherical fiducials were used to calculate target registration errors (TREs), providing a metric for validating registrations, where TREs were computed using root-mean-square distances to describe the alignment of manually identified corresponding fiducials. An analysis of variance (ANOVA) was used to identify statistically significant differences (p < 0.05) between the TREs for the three abdominal thicknesses for each applicator type. As an additional indicator of geometric accuracy, the bladder was segmented in the 3D US and corresponding MR/CT images, and volumetric differences and Dice similarity coefficients (DSCs) were calculated. RESULTS: For both applicator types, the combination of 3D TRUS with 3D TAUS images allowed image information obscured by the shadowing artifacts under single imaging perspectives to be recovered. For the tandem-and-ring applicator, the mean ± one standard deviation (SD) TREs from the images with increasing thicknesses were 1.37 ± 1.35 mm, 1.84 ± 1.22 mm, and 1.60 ± 1.00 mm. Similarly, for the tandem-and-ovoids applicator, the mean ± SD TREs from the images with increasing thicknesses were 1.37 ± 0.35 mm, 1.95 ± 0.90 mm, and 1.61 ± 0.76 mm. No statistically significant difference was detected in the TREs for the three thicknesses for either applicator type. The mean volume differences for the bladder segmentations were 3.14% and 2.33% and mean DSCs were 87.8% and 87.7% for the tandem-and-ring and tandem-and-ovoids applicators, respectively. CONCLUSIONS: In this proof-of-concept study, we demonstrated the feasibility of fusing 3D TRUS and 3D TAUS images based on the geometry of tandem-and-ring and tandem-and-ovoids applicators. This represents a step toward an accessible and low-cost 3D imaging method for gynecologic brachytherapy, with the potential to extend this approach to other intracavitary configurations and hybrid applicators.

Deep learning segmentation of general interventional tools in two-dimensional ultrasound images.

PURPOSE: Many interventional procedures require the precise placement of needles or therapy applicators (tools) to correctly achieve planned targets for optimal diagnosis or treatment of cancer, typically leveraging the temporal resolution of ultrasound (US) to provide real-time feedback. Identifying tools in two-dimensional (2D) images can often be time-consuming with the precise position difficult to distinguish. We have developed and implemented a deep learning method to segment tools in 2D US images in near real-time for multiple anatomical sites, despite the widely varying appearances across interventional applications. METHODS: A U-Net architecture with a Dice similarity coefficient (DSC) loss function was used to perform segmentation on input images resized to 256 × 256 pixels. The U-Net was modified by adding 50% dropouts and the use of transpose convolutions in the decoder section of the network. The proposed approach was trained with 917 images and manual segmentations from prostate/gynecologic brachytherapy, liver ablation, and kidney biopsy/ablation procedures, as well as phantom experiments. Real-time data augmentation was applied to improve generalizability and doubled the dataset for each epoch. Postprocessing to identify the tool tip and trajectory was performed using two different approaches, comparing the largest island with a linear fit to random sample consensus (RANSAC) fitting. RESULTS: Comparing predictions from 315 unseen test images to manual segmentations, the overall median [first quartile, third quartile] tip error, angular error, and DSC were 3.5 [1.3, 13.5] mm, 0.8 [0.3, 1.7]°, and 73.3 [56.2, 82.3]%, respectively, following RANSAC postprocessing. The predictions with the lowest median tip and angular errors were observed in the gynecologic images (median tip error: 0.3 mm; median angular error: 0.4°) with the highest errors in the kidney images (median tip error: 10.1 mm; median angular error: 2.9°). The performance on the kidney images was likely due to a reduction in acoustic signal associated with oblique insertions relative to the US probe and the increased number of anatomical interfaces with similar echogenicity. Unprocessed segmentations were performed with a mean time of approximately 50 ms per image. CONCLUSIONS: We have demonstrated that our proposed approach can accurately segment tools in 2D US images from multiple anatomical locations and a variety of clinical interventional procedures in near real-time, providing the potential to improve image guidance during a broad range of diagnostic and therapeutic cancer interventions.

A semiautomatic segmentation method for interstitial needles in intraoperative 3D transvaginal ultrasound images for high-dose-rate gynecologic brachytherapy of vaginal tumors.

PURPOSE: The purpose of this study was to evaluate the use of a semiautomatic algorithm to simultaneously segment multiple high-dose-rate (HDR) gynecologic interstitial brachytherapy (ISBT) needles in three-dimensional (3D) transvaginal ultrasound (TVUS) images, with the aim of providing a clinically useful tool for intraoperative implant assessment. METHODS AND MATERIALS: A needle segmentation algorithm previously developed for HDR prostate brachytherapy was adapted and extended to 3D TVUS images from gynecologic ISBT patients with vaginal tumors. Two patients were used for refining/validating the modified algorithm and five patients (8-12 needles/patient) were reserved as an unseen test data set. The images were filtered to enhance needle edges, using intensity peaks to generate feature points, and leveraged the randomized 3D Hough transform to identify candidate needle trajectories. Algorithmic segmentations were compared against manual segmentations and calculated dwell positions were evaluated. RESULTS: All 50 test data set needles were successfully segmented with 96% of algorithmically segmented needles having angular differences <3° compared with manually segmented needles and the maximum Euclidean distance was <2.1 mm. The median distance between corresponding dwell positions was 0.77 mm with 86% of needles having maximum differences <3 mm. The mean segmentation time using the algorithm was <30 s/patient. CONCLUSIONS: We successfully segmented multiple needles simultaneously in intraoperative 3D TVUS images from gynecologic HDR-ISBT patients with vaginal tumors and demonstrated the robustness of the algorithmic approach to image artifacts. This method provided accurate segmentations within a clinically efficient timeframe, providing the potential to be translated into intraoperative clinical use for implant assessment.