Real-time dense-view imaging for three-dimensional light-field display based on image color calibration and self-supervised view synthesis.

Three-Dimensional (3D) light-field display has achieved promising improvement in recent years. However, since the dense-view images cannot be collected fast in real-world 3D scenes, the real-time 3D light-field display is still challenging to achieve in real scenes, especially at the high-resolution 3D display. Here, a real-time 3D light-field display method with dense-view is proposed based on image color correction and self-supervised optical flow estimation, and a high-quality and high frame rate of 3D light-field display can be realized simultaneously. A sparse camera array is firstly used to capture sparse-view images in the proposed method. To eliminate the color deviation of the sparse views, the imaging process of the camera is analyzed, and a practical multi-layer perception (MLP) network is proposed to perform color calibration. Given sparse views with consistent color, the optical flow can be estimated by a lightweight convolutional neural network (CNN) at high speed, which uses the input image pairs to learn the optical flow in a self-supervised manner. With inverse warp operation, dense-view images can be synthesized in the end. Quantitative and qualitative experiments are performed to evaluate the feasibility of the proposed method. Experimental results show that over 60 dense-view images at a resolution of 1024 × 512 can be generated with 11 input views at a frame rate over 20 fps, which is 4× faster than previous optical flow estimation methods PWC-Net and LiteFlowNet3. Finally, large viewing angles and high-quality 3D light-field display at 3840 × 2160 resolution can be achieved in real-time.

Self-supervised stereo depth estimation based on bi-directional pixel-movement learning.

Stereo depth estimation is an efficient method to perceive three-dimensional structures in real scenes. In this paper, we propose a novel self-supervised method, to the best of our knowledge, to extract depth information by learning bi-directional pixel movement with convolutional neural networks (CNNs). Given left and right views, we use CNNs to learn the task of middle-view synthesis for perceiving bi-directional pixel movement from left-right views to the middle view. The information of pixel movement will be stored in the features after CNNs are trained. Then we use several convolutional layers to extract the information of pixel movement for estimating a depth map of the given scene. Experiments show that our proposed method can significantly provide a high-quality depth map using only a color image as a supervisory signal.

Analysis and removal of crosstalk in a time-multiplexed light-field display.

Time-multiplexed light-field displays (TMLFDs) can provide natural and realistic three-dimensional (3D) performance with a wide 120° viewing angle, which provides broad potential applications in 3D electronic sand table (EST) technology. However, current TMLFDs suffer from severe crosstalk, which can lead to image aliasing and the distortion of the depth information. In this paper, the mechanisms underlying the emergence of crosstalk in TMLFD systems are identified and analyzed. The results indicate that the specific structure of the slanted lenticular lens array (LLA) and the non-uniformity of the emergent light distribution in the lens elements are the two main factors responsible for the crosstalk. In order to produce clear depth perception and improve the image quality, a novel ladder-type LCD sub-pixel arrangement and a compound lens with three aspheric surfaces are proposed and introduced into a TMLFD to respectively reduce the two types of crosstalk. Crosstalk simulation experiments demonstrate the validity of the proposed methods. Structural similarity (SSIM) simulation experiments and light-field reconstruction experiments also indicate that aliasing is effectively reduced and the depth quality is significantly improved over the entire viewing range. In addition, a tabletop 3D EST based on the proposed TMLFD is presented. The proposed approaches to crosstalk reduction are also compatible with other lenticular lens-based 3D displays.

Virtual view synthesis for 3D light-field display based on scene tower blending.

Three-dimensional (3D) light-field display has achieved a great improvement. However, the collection of dense viewpoints in the real 3D scene is still a bottleneck. Virtual views can be generated by unsupervised networks, but the quality of different views is inconsistent because networks are separately trained on each posed view. Here, a virtual view synthesis method for the 3D light-field display based on scene tower blending is presented, which can synthesize high quality virtual views with correct occlusions by blending all tower results, and dense viewpoints on 3D light-field display can be provided with smooth motion parallax. Posed views are combinatorially input into diverse unsupervised CNNs to predict respective input-view towers, and towers of the same viewpoint are fused together. All posed-view towers are blended as a scene color tower and a scene selection tower, so that 3D scene distributions at different depth planes can be accurately estimated. Blended scene towers are soft-projected to synthesize virtual views with correct occlusions. A denoising network is used to improve the image quality of final synthetic views. Experimental results demonstrate the validity of the proposed method, which shows outstanding performances under various disparities. PSNR of the virtual views are about 30 dB and SSIM is above 0.91. We believe that our view synthesis method will be helpful for future applications of the 3D light-field display.

Dense-view synthesis for three-dimensional light-field display based on unsupervised learning.

Three-dimensional (3D) light field display, as a potential future display method, has attracted considerable attention. However, there still exist certain issues to be addressed, especially the capture of dense views in real 3D scenes. Using sparse cameras associated with view synthesis algorithm has become a practical method. Supervised convolutional neural network (CNN) is used to synthesize virtual views. However, such a large amount of training target views is sometimes difficult to be obtained and the training position is relatively fixed. Novel views can also be synthesized by unsupervised network MPVN, but the method has strict requirements on capturing multiple uniform horizontal viewpoints, which is not suitable in practice. Here, a method of dense-view synthesis based on unsupervised learning is presented, which can synthesize arbitrary virtual views with multiple free-posed views captured in the real 3D scene based on unsupervised learning. Multiple posed views are reprojected to the target position and input into the neural network. The network outputs a color tower and a selection tower indicating the scene distribution along the depth direction. A single image is yielded by the weighted summation of two towers. The proposed network is end-to-end trained based on unsupervised learning by minimizing errors during reconstructions of posed views. A virtual view can be predicted in a high quality by reprojecting posed views to the desired position. Additionally, a sequence of dense virtual views can be generated for 3D light-field display by repeated predictions. Experimental results demonstrate the validity of our proposed network. PSNR of synthesized views are around 30dB and SSIM are over 0.90. Since multiple cameras are supported to be placed in free-posed positions, there are not strict physical requirements and the proposed method can be flexibly used for the real scene capture. We believe this approach will contribute to the wide applications of 3D light-field display in the future.

[Application of CT scan in diagnosis of pathological type and origin of metastatic ovarian tumors].

OBJECTIVE: To evaluate the application of CT scan in diagnosis of pathological types and origins of metastatic ovarian tumors. METHODS: Clinical data, histopathological results and CT images of 43 patients with pathologically-proved metastatic ovarian tumor were retrospectively analyzed. Diagnostic values of CT imaging for pathological type and origin of metastatic ovarian tumors were evaluated. RESULTS: The pathological types of metastatic ovarian tumor were related to the size of the lesion (P<0.01), while not related to the sites of lesion (unilateral or bilateral), the cystic-solid and mixed lesions with or without separation (all P>0.05). Metastatic ovarian tumors of colorectal origin were usually unilateral lesions, and showed cystic or cystic-solid masses, while those of gastric origin were usually bilateral lesions, and showed solid or solid-based masses. CONCLUSIONS: CT imaging may be of value in diagnosis of pathological types and origin of metastatic ovarian tumor.

Novel mutations in EVC cause aberrant splicing in Ellis-van Creveld syndrome.

Ellis-van Creveld syndrome (EvC) is a rare autosomal recessive disorder characterized by disproportionate chondrodysplasia, postaxial polydactyly, nail dystrophy, dental abnormalities and in a proportion of patients, congenital cardiac malformations. Weyers acrofacial dysostosis (Weyers) is another dominantly inherited disorder allelic to EvC syndrome but with milder phenotypes. Both disorders can result from loss-of-function mutations in either EVC or EVC2 gene, and phenotypes associated with the two gene mutations are clinically indistinguishable. We present here a clinical and molecular analysis of a Chinese family manifested specific features of EvC syndrome. Sequencing of both EVC and EVC2 identified two novel heterozygous splice site mutations c.384+5G>C in intron 3 and c.1465-1G>A in intron 10 in EVC, which were inherited from mother and father, respectively. In vitro minigene expression assay, RT-PCR and sequencing analysis demonstrated that c.384+5G>C mutation abolished normal splice site and created a new cryptic acceptor site within exon 4, whereas c.1465-1G>A mutation affected consensus splice junction site and resulted in full exon 11 skipping. These two aberrant pre-mRNA splicing processes both produced in-frame abnormal transcripts that possibly led to abolishment of important functional domains. To our knowledge, this is the first report of EVC mutations that cause EvC syndrome in Chinese population. Our data revealed that EVC splice site mutations altered splicing pattern and helped elucidate the pathogenesis of EvC syndrome.

HOXB1 founder mutation in humans recapitulates the phenotype of Hoxb1-/- mice.

Members of the highly conserved homeobox (HOX) gene family encode transcription factors that confer cellular and tissue identities along the antero-posterior axis of mice and humans. We have identified a founder homozygous missense mutation in HOXB1 in two families from a conservative German American population. The resulting phenotype includes bilateral facial palsy, hearing loss, and strabismus and correlates extensively with the previously reported Hoxb1(-/-) mouse phenotype. The missense variant is predicted to result in the substitution of a cysteine for an arginine at amino acid residue 207 (Arg207Cys), which corresponds to the highly conserved Arg5 of the homeodomain. Arg5 interacts with thymine in the minor groove of DNA through hydrogen bonding and electrostatic attraction. Molecular modeling and an in vitro DNA-protein binding assay predict that the mutation would disrupt these interactions, destabilize the HOXB1:PBX1:DNA complex, and alter HOXB1 transcriptional activity.

MiR-376a and histone deacetylation 9 form a regulatory circuitry in hepatocellular carcinoma.

BACKGROUND/AIMS: Our previous study has demonstrated that down-regulation of miR-376a might contribute to the development of hepatocellular carcinoma (HCC), but the mechanism underlying this down-regulation remains obscure. METHODS/RESULTS: histone deacetylase (HDAC) inhibitor increased the level of miR-376a in L02 and Huh7 cells by up-regulating the acetylation level of histone 3 at the Maternally expressed 3 (Meg3) differentially methylated region (DMR). Interestingly, HDAC9, a histone deacetylase responsible for deacetylating lysine 18 of histone 3 (H3K18), was identified as the target of miR-376a. In addition, HDAC9 siRNA increased the expression of miR-376a by up-regulating the global histone H3K18 acetylation level, with Meg3 DMR included. Finally, miR-376a and HDAC9 were inversely correlated in HCC. CONCLUSION: HDAC9 plays an important role both as effects and targets of miR-376a.

[Association study between candidate genes on transforming growth factor-ß signaling pathway and the risk of non-syndromic cleft lip with or without cleft palate in Chinese populations].

OBJECTIVE: To explore the association between 10 candidate genes on transforming growth factor-ß (TGFB) signaling pathway and non-syndromic cleft lip with or without cleft palate (NSCL/P) among Chinese populations, and to study the gene-environment interaction. METHODS: A total of 806 Chinese Han NSCL/P trios were ascertained from an international consortium, which conducted a genome-wide association study using a case-parent trio design to investigate the genes affecting risk to NSCL/P. The transmission disequilibrium test (TDT) was used to test for effects of 343 single nucleotide polymorphisms (SNPs) in 10 genes on TGFB signaling pathway including DCN, TGFB1, TGFB2, TGFB3, TGFBR1, TGFBR2, BAMBI, SMAD2, SMAD3 and SMAD4. The conditional regression models were used to test for gene-environment interaction. RESULTS: For TDT, although 19 SNPs showed nominal significant association with NSCL/P, no significant evidence of association was seen for all SNPs in 806 NSCL/P trios after Bonferroni correction. The interactions between genes and maternal smoking, environmental tobacco smoke, alcohol consumption and multi-vitamin supplementation during pregnancy did not attain statistical significance after correction for multiple comparisons. CONCLUSION: No evidence for SNP effect of genes on TGFB signaling pathway and significant gene-environment interaction was seen in our data.

The LIF-mediated molecular signature regulating murine embryo implantation.

The establishment of a receptive uterus is the prime requirement for embryo implantation. In mice, the E2-induced cytokine leukemia inhibitory factor (LIF) is essential in switching the uterine luminal epithelium (LE) from a nonreceptive to a receptive state. Here we define the LIF-mediated switch using array analysis and informatics to identify LIF-induced changes in gene expression and annotated signaling pathways specific to the LE. We compare gene expression profiles at 0, 1, 3, and 6 h, following LIF treatment. During the first hour, the JAK-STAT signaling pathway is activated and the expression of 54 genes declines, primarily affecting LE cytoskeletal and chromatin organization as well as a transient reduction in the progesterone, TGFbetaR1, and ACVR1 receptors. Simultaneously 256 genes increase expression, of which 42 are transcription factors, including Sox, Kfl, Hes, Hey, and Hox families. Within 3 h, the expression of 3987 genes belonging to more than 25 biological process pathways was altered. We confirmed the mRNA and protein distribution of key genes from 10 pathways, including the Igf-1, Vegf, Toll-like receptors, actin cytoskeleton, ephrin, integrins, TGFbeta, Wnt, and Notch pathways. These data identify novel LIF-activated pathways in the LE and define the molecular basis between the refractory and receptive uterine phases. More broadly, these findings highlight the staggering capacity of a single cytokine to induce a dynamic and complex network of changes in a simple epithelium essential to mammalian reproduction and provide a basis for identifying new routes to regulating female reproduction.

Probing cerebellar circuit dysfunction in rodent models of spinocerebellar ataxia by means of in vivo two-photon calcium imaging.

Purkinje neuron degeneration characterizes spinocerebellar ataxia type 1, yet the comprehension of the impact on the broader cerebellar circuit remains incomplete. We here detail simultaneous in vivo two-photon calcium imaging of diverse neuronal populations in the cerebellar cortex of Sca1 mice while they are navigating a virtual environment. We outline surgical procedures and protocols to chronically record from identical neurons, and we detail data post-processing and analysis to delineate disease-related alterations in neuronal activity and sensorimotor-driven response properties. For complete details on the use and execution of this protocol, please refer to Pilotto et al.1.

Cortical hyperexcitability in mouse models and patients with amyotrophic lateral sclerosis is linked to noradrenaline deficiency.

Amyotrophic lateral sclerosis (ALS) is a devastating neurodegenerative disease, characterized by the death of upper (UMN) and lower motor neurons (LMN) in the motor cortex, brainstem, and spinal cord. Despite decades of research, ALS remains incurable, challenging to diagnose, and of extremely rapid progression. A unifying feature of sporadic and familial forms of ALS is cortical hyperexcitability, which precedes symptom onset, negatively correlates with survival, and is sufficient to trigger neurodegeneration in rodents. Using electrocorticography in the Sod1G86R and FusΔNLS/+ ALS mouse models and standard electroencephalography recordings in patients with sporadic ALS, we demonstrate a deficit in theta-gamma phase-amplitude coupling (PAC) in ALS. In mice, PAC deficits started before symptom onset, and in patients, PAC deficits correlated with the rate of disease progression. Using mass spectrometry analyses of CNS neuropeptides, we identified a presymptomatic reduction of noradrenaline (NA) in the motor cortex of ALS mouse models, further validated by in vivo two-photon imaging in behaving SOD1G93A and FusΔNLS/+ mice, that revealed pronounced reduction of locomotion-associated NA release. NA deficits were also detected in postmortem tissues from patients with ALS, along with transcriptomic alterations of noradrenergic signaling pathways. Pharmacological ablation of noradrenergic neurons with DSP-4 reduced theta-gamma PAC in wild-type mice and administration of a synthetic precursor of NA augmented theta-gamma PAC in ALS mice. Our findings suggest theta-gamma PAC as means to assess and monitor cortical dysfunction in ALS and warrant further investigation of the NA system as a potential therapeutic target.

Examining markers in 8q24 to explain differences in evidence for association with cleft lip with/without cleft palate between Asians and Europeans.

In a recent genome-wide association study (GWAS) from an international consortium, evidence of linkage and association in chr8q24 was much stronger among nonsyndromic cleft lip/palate (CL/P) case-parent trios of European ancestry than among trios of Asian ancestry. We examined marker information content and haplotype diversity across 13 recruitment sites (from Europe, United States, and Asia) separately, and conducted principal components analysis (PCA) on parents. As expected, PCA revealed large genetic distances between Europeans and Asians, and a north-south cline from Korea to Singapore in Asia, with Filipino parents forming a somewhat distinct Southeast Asian cluster. Hierarchical clustering of SNP heterozygosity revealed two major clades consistent with PCA results. All genotyped SNPs giving P < 10(-6) in the allelic transmission disequilibrium test (TDT) showed higher heterozygosity in Europeans than Asians. On average, European ancestry parents had higher haplotype diversity than Asians. Imputing additional variants across chr8q24 increased the strength of statistical evidence among Europeans and also revealed a significant signal among Asians (although it did not reach genome-wide significance). Tests for SNP-population interaction were negative, indicating the lack of strong signal for 8q24 in families of Asian ancestry was not due to any distinct genetic effect, but could simply reflect low power due to lower allele frequencies in Asians.

The second deletion mutation in exon 8 of EDA gene in an XLHED pedigree.

BACKGROUND: X-linked hypohidrotic ectodermal dysplasia (XLHED) is characterized by hypodontia, hypohidrosis, sparse hair and characteristic facial features and is caused by mutation in the ectodysplasin A (EDA) gene. OBJECTIVE: In this study we report on a large Chinese XLHED family and investigate the molecular genetics of the defect. METHODS: All individuals of the family were examined by clinical and radiographic examinations. The EDA gene was sequenced in the whole family and in 150 controls. RESULTS: Three male patients had classic XLHED phenotype. A novel one-nucleotide deletion mutation (c.855delG) in exon 8 which caused premature termination of the polypeptide at amino acid 307 was confirmed. The mutant lost parts of the TNF domain may prevent transmission of the intracellular downstream signal. This was the second deletion mutation in exon 8 that was reported in a Chinese individual. CONCLUSIONS: Our findings suggested deletion mutations in exon 8 might be specific to the Chinese population.

X-linked markers in the Duchenne muscular dystrophy gene associated with oral clefts.

As part of an international consortium, case-parent trios were collected for a genome-wide association study of isolated, non-syndromic oral clefts, including cleft lip (CL), cleft palate (CP), and cleft lip and palate (CLP). Non-syndromic oral clefts have a complex and heterogeneous etiology. Risk is influenced by genes and environmental factors, and differs markedly by gender. Family-based association tests (FBAT) were used on 14,486 single nucleotide polymorphisms (SNPs) spanning the X chromosome, stratified by type of cleft and racial group. Significant results, even after multiple-comparisons correction, were obtained for the Duchenne muscular dystrophy (DMD) gene, the largest single gene in the human genome, among CL/P (i.e., both CL and CLP combined) trios. When stratified into groups of European and Asian ancestry, stronger signals were obtained for Asian subjects. Although conventional sliding-window haplotype analysis showed no increase in significance, selected combinations of the 25 most significant SNPs in the DMD gene identified four SNPs together that attained genome-wide significance among Asian CL/P trios, raising the possibility of interaction between distant SNPs within the DMD gene.

[Association study for gene polymorphism of folic acid/homocysteine metabolic pathway and nonsyndromic cleft lip with or without cleft palate in Chinese populations].

OBJECTIVE: To explore the association between 18 candidate genes encoding enzymes on the folate/homocysteine metabolism pathway and non-syndromic cleft lip with or without cleft palate (NSCL/P) in Chinese populations. METHODS: A total of 806 NSCL/P trios were drawn by an international consortium, which conducted a genome-wide association study using a case-parent trio design to investigate genes affecting risks to NSCL/P. The transmission disequilibrium test (TDT) was used for deviation from Mendelian expectations for 257 SNPs in 18 folate/homocysteine metabolism-related genes. The interactions between markers in these gene and environmental risk factors were also tested using conditional Logistic regressions. RESULTS: Although four SNPs (rs6428977, rs12060264, rs7730643 and rs4920037) showed nominal significant association with NSCL/P in the TDT on 806 NSCL/P trios (P<0.05), no significant evidence of linkage and association remained in all the SNPs after Bonferroni correction. Similar tests for interactions between genes and maternal smoking, environmental tobacco smoke, alcohol consumption and multi-vitamin supplementation during pregnancy did not attain statistical significance after correction for multiple comparisons. CONCLUSION: Folate/homocysteine metabolism-related genes could not influence the risk of NSCL/P.

Real-time High-Quality Computer-Generated Hologram Using Complex-Valued Convolutional Neural Network.

Holographic displays are ideal display technologies for virtual and augmented reality because all visual cues are provided. However, real-time high-quality holographic displays are difficult to achieve because the generation of high-quality computer-generated hologram (CGH) is inefficient in existing algorithms. Here, complex-valued convolutional neural network (CCNN) is proposed for phase-only CGH generation. The CCNN-CGH architecture is effective with a simple network structure based on the character design of complex amplitude. A holographic display prototype is set up for optical reconstruction. Experiments verify that state-of-the-art performance is achieved in terms of quality and generation speed in existing end-to-end neural holography methods using the ideal wave propagation model. The generation speed is three times faster than HoloNet and one-sixth faster than Holo-encoder, and the Peak Signal to Noise Ratio (PSNR) is increased by 3 dB and 9 dB, respectively. Real-time high-quality CGHs are generated in 1920×1072 and 3840×2160 resolutions for dynamic holographic displays.

Early molecular layer interneuron hyperactivity triggers Purkinje neuron degeneration in SCA1.

Toxic proteinaceous deposits and alterations in excitability and activity levels characterize vulnerable neuronal populations in neurodegenerative diseases. Using in vivo two-photon imaging in behaving spinocerebellar ataxia type 1 (Sca1) mice, wherein Purkinje neurons (PNs) degenerate, we identify an inhibitory circuit element (molecular layer interneurons [MLINs]) that becomes prematurely hyperexcitable, compromising sensorimotor signals in the cerebellum at early stages. Mutant MLINs express abnormally elevated parvalbumin, harbor high excitatory-to-inhibitory synaptic density, and display more numerous synaptic connections on PNs, indicating an excitation/inhibition imbalance. Chemogenetic inhibition of hyperexcitable MLINs normalizes parvalbumin expression and restores calcium signaling in Sca1 PNs. Chronic inhibition of mutant MLINs delayed PN degeneration, reduced pathology, and ameliorated motor deficits in Sca1 mice. Conserved proteomic signature of Sca1 MLINs, shared with human SCA1 interneurons, involved the higher expression of FRRS1L, implicated in AMPA receptor trafficking. We thus propose that circuit-level deficits upstream of PNs are one of the main disease triggers in SCA1.

Evidence for gene-environment interaction in a genome wide study of nonsyndromic cleft palate.

Nonsyndromic cleft palate (CP) is a common birth defect with a complex and heterogeneous etiology involving both genetic and environmental risk factors. We conducted a genome-wide association study (GWAS) using 550 case-parent trios, ascertained through a CP case collected in an international consortium. Family-based association tests of single nucleotide polymorphisms (SNP) and three common maternal exposures (maternal smoking, alcohol consumption, and multivitamin supplementation) were used in a combined 2 df test for gene (G) and gene-environment (G × E) interaction simultaneously, plus a separate 1 df test for G × E interaction alone. Conditional logistic regression models were used to estimate effects on risk to exposed and unexposed children. While no SNP achieved genome-wide significance when considered alone, markers in several genes attained or approached genome-wide significance when G × E interaction was included. Among these, MLLT3 and SMC2 on chromosome 9 showed multiple SNPs resulting in an increased risk if the mother consumed alcohol during the peri-conceptual period (3 months prior to conception through the first trimester). TBK1 on chr. 12 and ZNF236 on chr. 18 showed multiple SNPs associated with higher risk of CP in the presence of maternal smoking. Additional evidence of reduced risk due to G × E interaction in the presence of multivitamin supplementation was observed for SNPs in BAALC on chr. 8. These results emphasize the need to consider G × E interaction when searching for genes influencing risk to complex and heterogeneous disorders, such as nonsyndromic CP.