Geometric lightguide for near-eye light field displays.

Most near-eye displays with one fixed focal plane suffer from the vergence-accommodation conflict and cause visual discomfort to users. In contrast, light field displays can provide natural and comfortable 3D visual sensation to users without the conflict. This paper presents a near-eye light field display consisting of a geometric lightguide and a light field generator, along with a collimator to ensure the light rays propagating in the lightguide are collimated. Unlike most lightguides, which reduce thickness by employing total internal reflection that can easily generate stray light, our lightguide directly propagates light rays without total internal reflection. The partially reflective mirrors of the lightguide expand the exit pupil to achieve an eyebox of 13m m(h o r i z o n t a l)×6.5m m(v e r t i c a l) with an eye relief of 18 mm. The collimator and the light field generator, both having effective focal lengths different in the horizontal and vertical directions, are designed to provide a 40-deg diagonal field of view. The working range of the light field generator, which is 30 cm to infinity, is verified qualitatively and quantitatively by experiments. We optimize the illuminance uniformity and analyze the illuminance variation across the eyebox. Further, we minimize the ghost artifact (referring to the split-up of light fields replicated by the partially reflective mirrors) by orienting the partially reflective mirrors at slightly different angles to enhance the image quality for short-range applications such as medical surgery.

High-resolution full-field optical coherence tomography microscope for the evaluation of freshly excised skin specimens during Mohs surgery: A feasibility study.

Histopathology for tumor margin assessment is time-consuming and expensive. High-resolution full-field optical coherence tomography (FF-OCT) images fresh tissues rapidly at cellular resolution and potentially facilitates evaluation. Here, we define FF-OCT features of normal and neoplastic skin lesions in fresh ex vivo tissues and assess its diagnostic accuracy for malignancies. For this, normal and neoplastic tissues were obtained from Mohs surgery, imaged using FF-OCT, and their features were described. Two expert OCT readers conducted a blinded analysis to evaluate their diagnostic accuracies, using histopathology as the ground truth. A convolutional neural network was built to distinguish and outline normal structures and tumors. Of the 113 tissues imaged, 95 (84%) had a tumor (75 basal cell carcinomas [BCCs] and 17 squamous cell carcinomas [SCCs]). The average reader diagnostic accuracy was 88.1%, with a sensitivity of 93.7%, and a specificity of 58.3%. The artificial intelligence (AI) model achieved a diagnostic accuracy of 87.6 ± 5.9%, sensitivity of 93.2 ± 2.1%, and specificity of 81.2 ± 9.2%. A mean intersection-over-union of 60.3 ± 10.1% was achieved when delineating the nodular BCC from normal structures. Limitation of the study was the small sample size for all tumors, especially SCCs. However, based on our preliminary results, we envision FF-OCT to rapidly image fresh tissues, facilitating surgical margin assessment. AI algorithms can aid in automated tumor detection, enabling widespread adoption of this technique.

Training With Uncertain Annotations for Semantic Segmentation of Basal Cell Carcinoma From Full-Field OCT Images.

Semantic segmentation of basal cell carcinoma (BCC) from full-field optical coherence tomography (FF-OCT) images of human skin has received considerable attention in medical imaging. However, it is challenging for dermatopathologists to annotate the training data due to OCT's lack of color specificity. Very often, they are uncertain about the correctness of the annotations they made. In practice, annotations fraught with uncertainty profoundly impact the effectiveness of model training and hence the performance of BCC segmentation. To address this issue, we propose an approach to model training with uncertain annotations. The proposed approach includes a data selection strategy to mitigate the uncertainty of training data, a class expansion to consider sebaceous gland and hair follicle as additional classes to enhance the performance of BCC segmentation, and a self-supervised pre-training procedure to improve the initial weights of the segmentation model parameters. Furthermore, we develop three post-processing techniques to reduce the impact of speckle noise and image discontinuities on BCC segmentation. The mean Dice score of BCC of our model reaches 0.503±0.003, which, to the best of our knowledge, is the best performance to date for semantic segmentation of BCC from FF-OCT images.

Toward cell nuclei precision between OCT and H&E images translation using signal-to-noise ratio cycle-consistency.

Medical image-to-image translation is often difficult and of limited effectiveness due to the differences in image acquisition mechanisms and the diverse structure of biological tissues. This work presents an unpaired image translation model between in-vivo optical coherence tomography (OCT) and ex-vivo Hematoxylin and eosin (H&E) stained images without the need for image stacking, registration, post-processing, and annotation. The model can generate high-quality and highly accurate virtual medical images, and is robust and bidirectional. Our framework introduces random noise to (1) blur redundant features, (2) defend against self-adversarial attacks, (3) stabilize inverse conversion, and (4) mitigate the impact of OCT speckles. We also demonstrate that our model can be pre-trained and then fine-tuned using images from different OCT systems in just a few epochs. Qualitative and quantitative comparisons with traditional image-to-image translation models show the robustness of our proposed signal-to-noise ratio (SNR) cycle-consistency method.

Acquiring 360° Light Field by a Moving Dual-Fisheye Camera.

In this paper, we propose an efficient deep learning pipeline for light field acquisition using a back-to-back dual-fisheye camera. The proposed pipeline generates a light field from a sequence of 360° raw images captured by the dual-fisheye camera. It has three main components: a convolutional network (CNN) that enforces a spatiotemporal consistency constraint on the subviews of the 360° light field, an equirectangular matching cost that aims at increasing the accuracy of disparity estimation, and a light field resampling subnet that produces the 360° light field based on the disparity information. Ablation tests are conducted to analyze the performance of the proposed pipeline using the HCI light field datasets with five objective assessment metrics (MSE, MAE, PSNR, SSIM, and GMSD). We also use real data obtained from a commercially available dual-fisheye camera to quantitatively and qualitatively test the effectiveness, robustness, and quality of the proposed pipeline. Our contributions include: 1) a novel spatiotemporal consistency loss that enforces the subviews of the 360° light field to be consistent, 2) an equirectangular matching cost that combats severe projection distortion of fisheye images, and 3) a light field resampling subnet that retains the geometric structure of spherical subviews while enhancing the angular resolution of the light field.

Isolated Sixth Nerve Palsies in a Child With Familial Hemophagocytic Lymphohistiocytosis Type 2.

ABSTRACT: A previously healthy 2-year-old boy presented with a left sixth cranial nerve palsy. There was a family history of multiple sclerosis and optic neuritis. Neuroimaging showed multiple foci of T2/FLAIR hyperintense signal abnormality in both cerebral hemispheres and in the brainstem. The initial diagnosis was suspicious for demyelinating disease. However, there was no clinical improvement after a course of corticosteroids, and there was no change in his follow-up MRI. He later developed bilateral sixth nerve palsies, with esotropia addressed with bilateral medial rectus botulinum toxin injections. A brain biopsy was planned. However, his 3-month-old sister was separately admitted for fever and pancytopenia. She had markedly elevated ferritin, D-dimer, triglycerides, sIL-2R, CXCL9, and IL-18 and low fibrinogen. Her bone marrow biopsy showed hemophagocytosis. Genetic testing of both siblings revealed biallelic mutations in the PRF1 locus. The final diagnosis of familial hemophagocytic lymphohistiocytosis Type 2 was made. Both siblings underwent chemotherapy. The boy's sixth nerve palsies and MRI abnormalities resolved. Both siblings then went on to undergo bone marrow transplant.

Efficient and Accurate Stitching for 360° Dual-Fisheye Images and Videos.

Back-to-back dual-fisheye cameras are the most cost-effective devices to capture 360° visual content. However, image and video stitching for such cameras often suffer from the effect of fisheye distortion, photometric inconsistency between the two views, and non-collocated optical centers. In this paper, we present algorithms for geometric calibration, photometric compensation, and seamless stitching to address these issues for back-to-back dual-fisheye cameras. Specifically, we develop a co-centric trajectory model for geometric calibration to characterize both intrinsic and extrinsic parameters of the fisheye camera to fifth-order precision, a photometric correction model for intensity and color compensation to provide efficient and accurate local color transfer, and a mesh deformation model along with an adaptive seam carving method for image stitching to reduce geometric distortion and ensure optimal spatiotemporal alignment. The stitching algorithm and the compensation algorithm can run efficiently for 1920×960 images. Quantitative evaluation of geometric distortion, color discontinuity, jitter, and ghost artifact of the resulting image and video shows that our solution outperforms the state-of-the-art techniques.

Time-Division Multiplexing Light Field Display with Learned Coded Aperture.

Conventional stereoscopic displays suffer from vergence-accommodation conflict and cause visual fatigue. Integral-imaging-based displays resolve the problem by directly projecting the sub-aperture views of a light field into the eyes using a microlens array or a similar structure. However, such displays have an inherent trade-off between angular and spatial resolutions. In this paper, we propose a novel coded time-division multiplexing technique that projects encoded sub-aperture views to the eyes of a viewer with correct cues for vergence-accommodation reflex. Given sparse light field sub-aperture views, our pipeline can provide a perception of high-resolution refocused images with minimal aliasing by jointly optimizing the sub-aperture views for display and the coded aperture pattern. This is achieved via deep learning in an end-to-end fashion by simulating light transport and image formation with Fourier optics. To our knowledge, this work is among the first that optimize the light field display pipeline with deep learning. We verify our idea with objective image quality metrics (PSNR, SSIM, and LPIPS) and perform an extensive study on various customizable design variables in our display pipeline. Experimental results show that light fields displayed using the proposed technique indeed have higher quality than that of baseline display designs.

Identification of Sex and Age from Macular Optical Coherence Tomography and Feature Analysis Using Deep Learning.

PURPOSE: To develop deep learning models for identification of sex and age from macular optical coherence tomography (OCT) and to analyze the features for differentiation of sex and age. DESIGN: Algorithm development using database of macular OCT. METHODS: We reviewed 6147 sets of macular OCT images from the healthy eyes of 3134 individuals from a single eye center in Taiwan. Deep learning-based algorithms were used to develop models for the identification of sex and age, and 10-fold cross-validation was applied. Gradient-weighted class activation mapping was used for feature analysis. RESULTS: The accuracy for sex prediction using deep learning from macular OCT was 85.6% ± 2.1% compared with accuracy of 61.9% using macular thickness and 61.4% ± 4.0% using deep learning from infrared fundus photography (P < .001 for both). The mean absolute error for age prediction using deep learning from macular OCT was 5.78 ± 0.29 years. A thorough analysis of the prediction accuracy and the gradient-weighted class activation mapping showed that the cross-sectional foveal contour lead to a better sex distinction than macular thickness or fundus photography, and the age-related characteristics of macula were on the whole layers of retina rather than the choroid. CONCLUSIONS: Sex and age could be identified from macular OCT using deep learning with good accuracy. The main sexual difference of macula lies in the foveal contour, and the whole layers of retina differ with aging. These novel findings provide useful information for further investigation in the pathogenesis of sex- and age-related macular structural diseases.

Classification of squamous cell carcinoma from FF-OCT images: Data selection and progressive model construction.

We investigate the speed and performance of squamous cell carcinoma (SCC) classification from full-field optical coherence tomography (FF-OCT) images based on the convolutional neural network (CNN). Due to the unique characteristics of SCC features, the high variety of CNN, and the high volume of our 3D FF-OCT dataset, progressive model construction is a time-consuming process. To address the issue, we develop a training strategy for data selection that makes model training 16 times faster by exploiting the dependency between images and the knowledge of SCC feature distribution. The speedup makes progressive model construction computationally feasible. Our approach further refines the regularization, channel attention, and optimization mechanism of SCC classifier and improves the accuracy of SCC classification to 87.12% at the image level and 90.10% at the tomogram level. The results are obtained by testing the proposed approach on an FF-OCT dataset with over one million mouse skin images.

Delayed diagnosis of endometriosis disadvantages women in ART: a retrospective population linked data study.

STUDY QUESTION: Do the outcomes and use of ART differ between women with and without endometriosis? SUMMARY ANSWER: ART use and outcome do not appear to differ for women with and without endometriosis, as long as endometriosis is diagnosed prior to commencing ART. WHAT IS KNOWN ALREADY: Approximately 40% of women with endometriosis have infertility and ART is the recommended treatment option for these women. However, diagnosis of endometriosis can be complex and lengthy, and a delay in diagnosis can reduce the likelihood of achieving a live birth. STUDY DESIGN, SIZE, DURATION: This retrospective national cohort study used longitudinal self-report data (collected 1996-2018) from women born in 1973-1978 who are participants in the Australian Longitudinal Study on Women's Health (ALSWH). The study also used linked administrative data on Endometriosis (1970-2018), ART (1996-2020) and births (1996-2018). PARTICIPANTS/MATERIALS, SETTING, METHODS: The outcome measures were: age at first ART cycle; use of ART treatments (IVF only; IUI only/and IVF); number of ART cycles (1-3; 4-10; 11-36); and births after first ART (no; yes) (note that births could not be tied to ART). MAIN RESULTS AND THE ROLE OF CHANCE: One in three (34.7%, n = 459/1322) women using ART had endometriosis, with 65.6% of these diagnosed before first ART and 34.4% after. Adjusted regression analyses showed women with endometriosis diagnosed before first ART were not significantly different to women without endometriosis on any outcome. However, women with endometriosis diagnosed after first ART were more likely to use IUI (adjusted odds ratio (aOR) 2.14, 95% CI 1.48, 3.09) and do more cycles (11-36 cycles: aOR 4.09, 95% CI 2.41, 6.95), and less likely to report a birth (aOR 0.67, 95% CI 0.45, 0.99), compared to women without endometriosis, despite no significant difference in starting age (coefficient = -0.62, 95% CI -1.36, 0.13). LIMITATIONS, REASONS FOR CAUTION: We did not have information on the severity of endometriosis, or the reasons for using ART, which can influence treatment and outcomes. We were not able to reliably link births with ART treatment. Finally, it is possible that some of the women in our 'no endometriosis' group did have endometriosis and were unaware of it, although prevalence rates match population estimates. WIDER IMPLICATIONS OF THE FINDINGS: These findings support previous studies that have found no difference in outcome of ART for women with endometriosis, but add the new insight that this is only true if endometriosis is diagnosed prior to commencing ART. A delayed diagnosis can create disadvantage during ART treatment. Early recourse to IVF may be advantageous for pregnancy prospects for women with endometriosis. STUDY FUNDING/COMPETING INTEREST(S): The ALSWH is funded by the Australian Government Department of Health. G.D.M. is supported by an NHMRC Principal Research Fellowship (APP11218449). The authors have no conflicts of interest to declare. TRIAL REGISTRATION NUMBER: N/A.

Efficient Face Detection in the Fisheye Image Domain.

Significant progress has been made for face detection from normal images in recent years; however, accurate and fast face detection from fisheye images remains a challenging issue because of serious fisheye distortion in the peripheral region of the image. To improve face detection accuracy, we propose a light-weight location-aware network to distinguish the peripheral region from the central region in the feature learning stage. To match the face detector, the shape and scale of the anchor (bounding box) is made location dependent. The overall face detection system performs directly in the fisheye image domain without rectification and calibration and hence is agnostic of the fisheye projection parameters. Experiments on Wider-360 and real-world fisheye images using a single CPU core indeed show that our method is superior to the state-of-the-art real-time face detector RFB Net.

Premature ovarian ageing following heterozygous loss of Senataxin.

Premature loss of ovarian activity before 40 years of age is known as primary ovarian insufficiency (POI) and occurs in ∼1% of women. A more subtle decline in ovarian activity, known as premature ovarian ageing (POA), occurs in ∼10% of women. Despite the high prevalence of POA, very little is known regarding its genetic causation. Senataxin (SETX) is an RNA/DNA helicase involved in repair of oxidative stress-induced DNA damage. Homozygous mutation of SETX leads to the neurodegenerative disorder, ataxia oculomotor apraxia type 2 (AOA2). There have been reports of POI in AOA2 females suggesting a link between SETX and ovarian ageing. Here, we studied female mice lacking either one (Setx+/-) or both (Setx-/-) copies of SETX over a 12- to 14-month period. We find that DNA damage is increased in oocytes from 8-month-old Setx+/- and Setx-/- females compared with Setx+/+ oocytes leading to a marked reduction in all classes of ovarian follicles at least 4 months earlier than typically occurs in female mice. Furthermore, during a 12-month long mating trial, Setx+/- and Setx-/- females produced significantly fewer pups than Setx+/+ females from 7 months of age onwards. These data show that SETX is critical for preventing POA in mice, likely by preserving DNA integrity in oocytes. Intriguingly, heterozygous Setx loss causes an equally severe impact on ovarian ageing as homozygous Setx loss. Because heterozygous SETX disruption is less likely to produce systemic effects, SETX compromise could underpin some cases of insidious POA.

Dermal epidermal junction detection for full-field optical coherence tomography data of human skin by deep learning.

Full-field optical coherence tomography (FF-OCT) has been developed to obtain three-dimensional (3D) OCT data of human skin for early diagnosis of skin cancer. Detection of dermal epidermal junction (DEJ), where melanomas and basal cell carcinomas originate, is an essential step for skin cancer diagnosis. However, most existing DEJ detection methods consider each cross-sectional frame of the 3D OCT data independently, leaving the relationship between neighboring frames unexplored. In this paper, we exploit the continuity of 3D OCT data to enhance DEJ detection. In particular, we propose a method for noise reduction of the training data and a multi-directional convolutional neural network to predict the probability of epidermal pixels in the 3D OCT data, which is more stable than one-directional convolutional neural network for DEJ detection. Our crosscheck refinement method also exploits the domain knowledge to generate a smooth DEJ surface. The average mean error of the entire DEJ detection system is approximately 6 µm.

Deep Face Rectification for 360° Dual-Fisheye Cameras.

Rectilinear face recognition models suffer from severe performance degradation when applied to fisheye images captured by 360° back-to-back dual fisheye cameras. We propose a novel face rectification method to combat the effect of fisheye image distortion on face recognition. The method consists of a classification network and a restoration network specifically designed to handle the non-linear property of fisheye projection. The classification network classifies an input fisheye image according to its distortion level. The restoration network takes a distorted image as input and restores the rectilinear geometric structure of the face. The performance of the proposed method is tested on an end-to-end face recognition system constructed by integrating the proposed rectification method with a conventional rectilinear face recognition system. The face verification accuracy of the integrated system is 99.18% when tested on images in the synthetic Labeled Faces in the Wild (LFW) dataset and 95.70% for images in a real image dataset, resulting in an average accuracy improvement of 6.57% over the conventional face recognition system. For face identification, the average improvement over the conventional face recognition system is 4.51%.

Detecting mouse squamous cell carcinoma from submicron full-field optical coherence tomography images by deep learning.

The standard medical practice for cancer diagnosis requires histopathology, which is an invasive and time-consuming procedure. Optical coherence tomography (OCT) is an alternative that is relatively fast, noninvasive, and able to capture three-dimensional structures of epithelial tissue. Unlike most previous OCT systems, which cannot capture crucial cellular-level information for squamous cell carcinoma (SCC) diagnosis, the full-field OCT (FF-OCT) technology used in this paper is able to produce images at sub-micron resolution and thereby facilitates the development of a deep learning algorithm for SCC detection. Experimental results show that the SCC detection algorithm can achieve a classification accuracy of 80% for mouse skin. Using the sub-micron FF-OCT imaging system, the proposed SCC detection algorithm has the potential for in-vivo applications.

Enhancement and Speedup of Photometric Compensation for Projectors by Reducing Inter-Pixel Coupling and Calibration Patterns.

For a procam to preserve the color appearance of an image projected on a color surface, the photometric distortion introduced by the color surface has to be properly compensated. The performance of such photometric compensation relies on an accurate estimation of the projector nonlinearity. In this paper, we improve the accuracy of projector nonlinearity estimation by taking inter-pixel coupling into consideration. In addition, to respond quickly to the change of projection area due to projector movement, we reduce the number of calibration patterns from six to one and use the projected image as the calibration pattern. This greatly improves the computational efficiency of re-calibration that needs to be performed on the fly during a multimedia presentation without breaking its continuity. Both objective and subjective results are provided to illustrate the effectiveness of the proposed method for color compensation.

Light Field Synthesis by Training Deep Network in the Refocused Image Domain.

Light field imaging, which captures spatial-angular information of light incident on image sensors, enables many interesting applications such as image refocusing and augmented reality. However, due to the limited sensor resolution, a trade-off exists between the spatial and angular resolutions. To increase the angular resolution, view synthesis techniques have been adopted to generate new views from existing views. However, traditional learning-based view synthesis mainly considers the image quality of each view of the light field and neglects the quality of the refocused images. In this paper, we propose a new loss function called refocused image error (RIE) to address the issue. The main idea is that the image quality of the synthesized light field should be optimized in the refocused image domain because it is where the light field is viewed. We analyze the behavior of RIE in the spectral domain and test the performance of our approach against previous approaches on both real (INRIA) and software-rendered (HCI) light field datasets using objective assessment metrics such as MSE, MAE, PSNR, SSIM, and GMSD. Experimental results show that the light field generated by our method results in better refocused images than previous methods.

The 2019 PGDIS position statement on transfer of mosaic embryos within a context of new information on PGT-A.

BACKGROUND: A recently published Position Statement (PS) by the Preimplantation Genetics Diagnosis International Society (PGDIS) regarding utilization of preimplantation genetic testing for aneuploidy (PGT-A) in association with in vitro fertilization (IVF) contained inaccuracies and misrepresentations. Because opinions issued by the PGDIS have since 2016 determined worldwide IVF practice, corrections appear of importance. METHODS: The International Do No Harm Group in IVF (IDNHG-IVF) is a spontaneously coalesced body of international investigators, concerned with increasing utilization of add-ons to IVF. It is responsible for the presented consensus statement, which as a final document was reached after review of the pertinent literature and again revised after the recent publication of the STAR trial and related commentaries. RESULTS: In contrast to the PGDIA-PS, we recommend restrictions to the increasing, and by IVF centers now often even mandated, utilization of PGT-A in IVF cycles. While PGT-A has been proposed as a tool for achieving enhanced singleton livebirth outcomes through embryo selection, continued false-positive rates and increasing evidence for embryonic self-correction downstream from the testing stage, has led IDNHG-IVF to conclude that currently available data are insufficient to impose overreaching recommendations for PGT-A utilization. DISCUSSION: Here presented consensus offers an alternative to the 2019 PGDIS position statement regarding utilization of preimplantation genetic testing for aneuploidy (PGT-A) in association with in vitro fertilization (IVF). Mindful of what appears to offer best outcomes for patients, and in full consideration of patient autonomy, here presented opinion is based on best available evidence, with the goal of improving safety and efficacy of IVF and minimizing wastage of embryos with potential for healthy births. CONCLUSIONS: As the PGDIS never suggested restrictions on clinical utilization of PGT-A in IVF, here presented rebuttal represents an act of self-regulation by parts of the IVF community in attempts to control increasing utilization of different unproven recent add-ons to IVF.