Single-Image SVBRDF Estimation Using Auxiliary Renderings as Intermediate Targets.

Recently, single-image SVBRDF capture is formulated as a regression problem, which uses a network to infer four SVBRDF maps from a flash-lit image. However, the accuracy is still not satisfactory since previous approaches usually adopt endto-end inference strategies. To mitigate the challenge, we propose "auxiliary renderings" as the intermediate regression targets, through which we divide the original end-to-end regression task into several easier sub-tasks, thus achieving better inference accuracy. Our contributions are threefold. First, we design three (or two pairs of) auxiliary renderings and summarize the motivations behind the designs. By our design, the auxiliary images are bumpiness-flattened or highlight-removed, containing disentangled visual cues about the final SVBRDF maps and can be easily transformed to the final maps. Second, to help estimate the auxiliary targets from the input image, we propose two mask images including a bumpiness mask and a highlight mask. Our method thus first infers mask images, then with the help of the mask images infers auxiliary renderings, and finally transforms the auxiliary images to SVBRDF maps. Third, we propose backbone UNets to infer mask images, and gated deformable UNets for estimating auxiliary targets. Thanks to the well designed networks and intermediate images, our method outputs better SVBRDF maps than previous approaches, validated by the extensive comparisonal and ablation experiments.

Discriminating Tensor Spectral Clustering for High-Dimension-Low-Sample-Size Data.

Tensor spectral clustering (TSC) is a recently proposed approach to robustly group data into underlying clusters. Unlike the traditional spectral clustering (SC), which merely uses pairwise similarities of data in an affinity matrix, TSC aims at exploring their multiwise similarities in an affinity tensor to achieve better performance. However, the performance of TSC highly relies on the design of multiwise similarities, and it remains unclear especially for high-dimension-low-sample-size (HDLSS) data. To this end, this article has proposed a discriminating TSC (DTSC) for HDLSS data. Specifically, DTSC uses the proposed discriminating affinity tensor that encodes the pair-to-pair similarities, which are particularly constructed by the anchor-based distance. HDLSS asymptotic analysis shows that the proposed affinity tensor can explicitly differentiate samples from different clusters when the feature dimension is large. This theoretical property allows DTSC to improve the clustering performance on HDLSS data. Experimental results on synthetic and benchmark datasets demonstrate the effectiveness and robustness of the proposed method in comparison to several baseline methods.

Dual-stream multi-dependency graph neural network enables precise cancer survival analysis.

Histopathology image-based survival prediction aims to provide a precise assessment of cancer prognosis and can inform personalized treatment decision-making in order to improve patient outcomes. However, existing methods cannot automatically model the complex correlations between numerous morphologically diverse patches in each whole slide image (WSI), thereby preventing them from achieving a more profound understanding and inference of the patient status. To address this, here we propose a novel deep learning framework, termed dual-stream multi-dependency graph neural network (DM-GNN), to enable precise cancer patient survival analysis. Specifically, DM-GNN is structured with the feature updating and global analysis branches to better model each WSI as two graphs based on morphological affinity and global co-activating dependencies. As these two dependencies depict each WSI from distinct but complementary perspectives, the two designed branches of DM-GNN can jointly achieve the multi-view modeling of complex correlations between the patches. Moreover, DM-GNN is also capable of boosting the utilization of dependency information during graph construction by introducing the affinity-guided attention recalibration module as the readout function. This novel module offers increased robustness against feature perturbation, thereby ensuring more reliable and stable predictions. Extensive benchmarking experiments on five TCGA datasets demonstrate that DM-GNN outperforms other state-of-the-art methods and offers interpretable prediction insights based on the morphological depiction of high-attention patches. Overall, DM-GNN represents a powerful and auxiliary tool for personalized cancer prognosis from histopathology images and has great potential to assist clinicians in making personalized treatment decisions and improving patient outcomes.

3D Snapshot: Invertible Embedding of 3D Neural Representations in a Single Image.

3D neural rendering enables photo-realistic reconstruction of a specific scene by encoding discontinuous inputs into a neural representation. Despite the remarkable rendering results, the storage of network parameters is not transmission-friendly and not extendable to metaverse applications. In this paper, we propose an invertible neural rendering approach that enables generating an interactive 3D model from a single image (i.e., 3D Snapshot). Our idea is to distill a pre-trained neural rendering model (e.g., NeRF) into a visualizable image form that can then be easily inverted back to a neural network. To this end, we first present a neural image distillation method to optimize three neural planes for representing the original neural rendering model. However, this representation is noisy and visually meaningless. We thus propose a dynamic invertible neural network to embed this noisy representation into a plausible image representation of the scene. We demonstrate promising reconstruction quality quantitatively and qualitatively, by comparing to the original neural rendering model, as well as video-based invertible methods. On the other hand, our method can store dozens of NeRFs with a compact restoration network (5MB), and embedding each 3D scene takes up only 160KB of storage. More importantly, our approach is the first solution that allows embedding a neural rendering model into image representations, which enables applications like creating an interactive 3D model from a printed image in the metaverse.

Multiview Tensor Spectral Clustering via Co-regularization.

Graph-based multi-view clustering encodes multi-view data into sample affinities to find consensus representation, effectively overcoming heterogeneity across different views. However, traditional affinity measures tend to collapse as the feature dimension expands, posing challenges in estimating a unified alignment that reveals both crossview and inner relationships. To tackle this challenge, we propose to achieve multi-view uniform clustering via consensus representation coregularization. First, the sample affinities are encoded by both popular dyadic affinity and recent high-order affinities to comprehensively characterize spatial distributions of the HDLSS data. Second, a fused consensus representation is learned through aligning the multi-view lowdimensional representation by co-regularization. The learning of the fused representation is modeled by a high-order eigenvalue problem within manifold space to preserve the intrinsic connections and complementary correlations of original data. A numerical scheme via manifold minimization is designed to solve the high-order eigenvalue problem efficaciously. Experiments on eight HDLSS datasets demonstrate the effectiveness of our proposed method in comparison with the recent thirteen benchmark methods.

Cryo-EM structures of adenosine receptor A3AR bound to selective agonists.

The adenosine A3 receptor (A3AR), a key member of the G protein-coupled receptor family, is a promising therapeutic target for inflammatory and cancerous conditions. The selective A3AR agonists, CF101 and CF102, are clinically significant, yet their recognition mechanisms remained elusive. Here we report the cryogenic electron microscopy structures of the full-length human A3AR bound to CF101 and CF102 with heterotrimeric Gi protein in complex at 3.3-3.2 Å resolution. These agonists reside in the orthosteric pocket, forming conserved interactions via their adenine moieties, while their 3-iodobenzyl groups exhibit distinct orientations. Functional assays reveal the critical role of extracellular loop 3 in A3AR's ligand selectivity and receptor activation. Key mutations, including His3.37, Ser5.42, and Ser6.52, in a unique sub-pocket of A3AR, significantly impact receptor activation. Comparative analysis with the inactive A2AAR structure highlights a conserved receptor activation mechanism. Our findings provide comprehensive insights into the molecular recognition and signaling of A3AR, paving the way for designing subtype-selective adenosine receptor ligands.

Decreased structural pathways mediating functional connectivity in obstructive sleep apnea.

BACKGROUND: Obstructive sleep apnea (OSA) is a common sleep breathing disorder that is often accompanied by changes in structural connectivity (SC) and functional connectivity (FC). However, the current understanding of the interaction between SC and FC in OSA is still limited. METHODS: The aim of this study is to integrate complementary neuroimaging modalities into a unified framework using multi-layer network analysis methods and to reveal their complex interrelationships. We introduce a new graph metric called SC-FC bandwidth, which measures the throughput of SC mediating FC in a multi-layer network. The bandwidth differences between two groups are evaluated using the network-based statistics (NBS) method. Additionally, we traced and analyzed the SC pathways corresponding to the abnormal bandwidth. RESULTS: In both the healthy control and patients with OSA, the majority offunctionally synchronized nodes were connected via SC paths of length 2. With the NBS method, we observed significantly lower bandwidth between the right Posterior cingulate gyrus and right Cuneus, bilateral Middle frontal gyrus, bilateral Gyrus rectus in OSA patients. By tracing the high-proportion SC pathways, it was found that OSA patients typically exhibit a decrease in direct SC-FC, SC-FC triangles, and SC-FC quads intra- and inter-networks. CONCLUSION: Complex interrelationship changes have been observed between the SC and FC in patients with OSA, which might leads to abnormal information transmission and communication in the brain network.

Deep Tensor Spectral Clustering Network via Ensemble of Multiple Affinity Tensors.

Tensor spectral clustering (TSC) is an emerging approach that explores multi-wise similarities to boost learning. However, two key challenges have yet to be well addressed in the existing TSC methods: (1) The construction and storage of high-order affinity tensors to encode the multi-wise similarities are memory-intensive and hampers their applicability, and (2) they mostly employ a two-stage approach that integrates multiple affinity tensors of different orders to learn a consensus tensor spectral embedding, thus often leading to a suboptimal clustering result. To this end, this paper proposes a tensor spectral clustering network (TSC-Net) to achieve one-stage learning of a consensus tensor spectral embedding, while reducing the memory cost. TSC-Net employs a deep neural network that learns to map the input samples to the consensus tensor spectral embedding, guided by a TSC objective with multiple affinity tensors. It uses stochastic optimization to calculate a small part of the affinity tensors, thereby avoiding loading the whole affinity tensors for computation, thus significantly reducing the memory cost. Through using an ensemble of multiple affinity tensors, the TSC can dramatically improve clustering performance. Empirical studies on benchmark datasets demonstrate that TSC-Net outperforms the recent baseline methods.

Improving Cancer Survival Prediction via Graph Convolutional Neural Network Learning on Protein-Protein Interaction Networks.

Cancer is one of the most challenging health problems worldwide. Accurate cancer survival prediction is vital for clinical decision making. Many deep learning methods have been proposed to understand the association between patients' genomic features and survival time. In most cases, the gene expression matrix is fed directly to the deep learning model. However, this approach completely ignores the interactions between biomolecules, and the resulting models can only learn the expression levels of genes to predict patient survival. In essence, the interaction between biomolecules is the key to determining the direction and function of biological processes. Proteins are the building blocks and principal undertakings of life activities, and as such, their complex interaction network is potentially informative for deep learning methods. Therefore, a more reliable approach is to have the neural network learn both gene expression data and protein interaction networks. We propose a new computational approach, termed CRESCENT, which is a protein-protein interaction (PPI) prior knowledge graph-based convolutional neural network (GCN) to improve cancer survival prediction. CRESCENT relies on the gene expression networks rather than gene expression levels to predict patient survival. The performance of CRESCENT is evaluated on a large-scale pan-cancer dataset consisting of 5991 patients from 16 different types of cancers. Extensive benchmarking experiments demonstrate that our proposed method is competitive in terms of the evaluation metric of the time-dependent concordance index( Ctd) when compared with several existing state-of-the-art approaches. Experiments also show that incorporating the network structure between genomic features effectively improves cancer survival prediction.

Specifying the Starting Point for S1 Iliosacral Screw Placement in the Dysmorphic Sacrum.

BACKGROUND: Sacral dysmorphism is not uncommon and complicates S1 iliosacral screw placement partially because of the difficulty of determining the starting point accurately on the sacral lateral view. We propose a method of specifying the starting point. METHODS: The starting point for the S1 iliosacral screw into the dysmorphic sacrum was specifically set at a point where the ossification of the S1/S2 intervertebral disc (OSID) intersected the posterior vertebral cortical line (PVCL) on the sacral lateral view, followed by guidewire manipulation and screw placement on the pelvic outlet and inlet views. Computer-simulated virtual surgical procedures based on pelvic computed tomography (CT) data on 95 dysmorphic sacra were performed to determine whether the starting point was below the iliac cortical density (ICD) and in the S1 oblique osseous corridor and to evaluate the accuracy of screw placement (with 1 screw being used, in the left hemipelvis). Surgical procedures on 17 patients were performed to verify the visibility of the OSID and PVCL, to check the location of the starting point relative to the ICD, and to validate the screw placement safety as demonstrated with postoperative CT scans. RESULTS: In the virtual surgical procedures, the starting point was consistently below the ICD and in the oblique osseous corridor in all patients and all screws were Grade 1. In the clinical surgical procedures, the OSID and PVCL were consistently visible and the starting point was always below the ICD in all patients; overall, 21 S1 iliosacral screws were placed in these 17 patients without malpositioning or iatrogenic injury. CONCLUSIONS: On the lateral view of the dysmorphic sacrum, the OSID and PVCL are visible and intersect at a point that is consistently below the ICD and in the oblique osseous corridor, and thus they can be used to identify the starting point. LEVEL OF EVIDENCE: Therapeutic Level III . See Instructions for Authors for a complete description of levels of evidence.

Realize Generative Yet Complete Latent Representation for Incomplete Multi-View Learning.

In multi-view environment, it would yield missing observations due to the limitation of the observation process. The most current representation learning methods struggle to explore complete information by lacking either cross-generative via simply filling in missing view data, or solidative via inferring a consistent representation among the existing views. To address this problem, we propose a deep generative model to learn a complete generative latent representation, namely Complete Multi-view Variational Auto-Encoders (CMVAE), which models the generation of the multiple views from a complete latent variable represented by a mixture of Gaussian distributions. Thus, the missing view can be fully characterized by the latent variables and is resolved by estimating its posterior distribution. Accordingly, a novel variational lower bound is introduced to integrate view-invariant information into posterior inference to enhance the solidative of the learned latent representation. The intrinsic correlations between views are mined to seek cross-view generality, and information leading to missing views is fused by view weights to reach solidity. Benchmark experimental results in clustering, classification, and cross-view image generation tasks demonstrate the superiority of CMVAE, while time complexity and parameter sensitivity analyses illustrate the efficiency and robustness. Additionally, application to bioinformatics data exemplifies its practical significance.

Differentiating ChatGPT-Generated and Human-Written Medical Texts: Quantitative Study.

BACKGROUND: Large language models, such as ChatGPT, are capable of generating grammatically perfect and human-like text content, and a large number of ChatGPT-generated texts have appeared on the internet. However, medical texts, such as clinical notes and diagnoses, require rigorous validation, and erroneous medical content generated by ChatGPT could potentially lead to disinformation that poses significant harm to health care and the general public. OBJECTIVE: This study is among the first on responsible artificial intelligence-generated content in medicine. We focus on analyzing the differences between medical texts written by human experts and those generated by ChatGPT and designing machine learning workflows to effectively detect and differentiate medical texts generated by ChatGPT. METHODS: We first constructed a suite of data sets containing medical texts written by human experts and generated by ChatGPT. We analyzed the linguistic features of these 2 types of content and uncovered differences in vocabulary, parts-of-speech, dependency, sentiment, perplexity, and other aspects. Finally, we designed and implemented machine learning methods to detect medical text generated by ChatGPT. The data and code used in this paper are published on GitHub. RESULTS: Medical texts written by humans were more concrete, more diverse, and typically contained more useful information, while medical texts generated by ChatGPT paid more attention to fluency and logic and usually expressed general terminologies rather than effective information specific to the context of the problem. A bidirectional encoder representations from transformers-based model effectively detected medical texts generated by ChatGPT, and the F1 score exceeded 95%. CONCLUSIONS: Although text generated by ChatGPT is grammatically perfect and human-like, the linguistic characteristics of generated medical texts were different from those written by human experts. Medical text generated by ChatGPT could be effectively detected by the proposed machine learning algorithms. This study provides a pathway toward trustworthy and accountable use of large language models in medicine.

AIDMAN: An AI-based object detection system for malaria diagnosis from smartphone thin-blood-smear images.

Malaria is a significant public health concern, with ∼95% of cases occurring in Africa, but accurate and timely diagnosis is problematic in remote and low-income areas. Here, we developed an artificial intelligence-based object detection system for malaria diagnosis (AIDMAN). In this system, the YOLOv5 model is used to detect cells in a thin blood smear. An attentional aligner model (AAM) is then applied for cellular classification that consists of multi-scale features, a local context aligner, and multi-scale attention. Finally, a convolutional neural network classifier is applied for diagnosis using blood-smear images, reducing interference caused by false positive cells. The results demonstrate that AIDMAN handles interference well, with a diagnostic accuracy of 98.62% for cells and 97% for blood-smear images. The prospective clinical validation accuracy of 98.44% is comparable to that of microscopists. AIDMAN shows clinically acceptable detection of malaria parasites and could aid malaria diagnosis, especially in areas lacking experienced parasitologists and equipment.

Brain Network Classification for Accurate Detection of Alzheimer's Disease via Manifold Harmonic Discriminant Analysis.

Mounting evidence shows that Alzheimer's disease (AD) manifests the dysfunction of the brain network much earlier before the onset of clinical symptoms, making its early diagnosis possible. Current brain network analyses treat high-dimensional network data as a regular matrix or vector, which destroys the essential network topology, thereby seriously affecting diagnosis accuracy. In this context, harmonic waves provide a solid theoretical background for exploring brain network topology. However, the harmonic waves are originally intended to discover neurological disease propagation patterns in the brain, which makes it difficult to accommodate brain disease diagnosis with high heterogeneity. To address this challenge, this article proposes a network manifold harmonic discriminant analysis (MHDA) method for accurately detecting AD. Each brain network is regarded as an instance drawn on a Stiefel manifold. Every instance is represented by a set of orthonormal eigenvectors (i.e., harmonic waves) derived from its Laplacian matrix, which fully respects the topological structure of the brain network. An MHDA method within the Stiefel space is proposed to identify the group-dependent common harmonic waves, which can be used as group-specific references for downstream analyses. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method in stratifying cognitively normal (CN) controls, mild cognitive impairment (MCI), and AD.

A survey of drug-target interaction and affinity prediction methods via graph neural networks.

The tasks of drug-target interaction (DTI) and drug-target affinity (DTA) prediction play important roles in the field of drug discovery. However, biological experiment-based methods are time-consuming and expensive. Recently, computational-based approaches have accelerated the process of drug-target relationship prediction. Drug and target features are represented in structure-based, sequence-based, and graph-based ways. Although some achievements have been made regarding structure-based representations and sequence-based representations, the acquired feature information is not sufficiently rich. Molecular graph-based representations are some of the more popular approaches, and they have also generated a great deal of interest. In this article, we provide an overview of the DTI prediction and DTA prediction tasks based on graph neural networks (GNNs). We briefly discuss the molecular graphs of drugs, the primary sequences of target proteins, and the graph reSLBpresentations of target proteins. Meanwhile, we conducted experiments on various fundamental datasets to substantiate the plausibility of DTI and DTA utilizing graph neural networks.

Learning pyramidal multi-scale harmonic wavelets for identifying the neuropathology propagation patterns of Alzheimer's disease.

Previous studies have established that neurodegenerative disease such as Alzheimer's disease (AD) is a disconnection syndrome, where the neuropathological burdens often propagate across the brain network to interfere with the structural and functional connections. In this context, identifying the propagation patterns of neuropathological burdens sheds new light on understanding the pathophysiological mechanism of AD progression. However, little attention has been paid to propagation pattern identification by fully considering the intrinsic properties of brain-network organization, which plays an important role in improving the interpretability of the identified propagation pathways. To this end, we propose a novel harmonic wavelet analysis approach to construct a set of region-specific pyramidal multi-scale harmonic wavelets, it allows us to characterize the propagation patterns of neuropathological burdens from multiple hierarchical modules across the brain network. Specifically, we first extract underlying hub nodes through a series of network centrality measurements on the common brain network reference generated from a population of minimum spanning tree (MST) brain networks. Then, we propose a manifold learning method to identify the region-specific pyramidal multi-scale harmonic wavelets corresponding to hub nodes by seamlessly integrating the hierarchically modular property of the brain network. We estimate the statistical power of our proposed harmonic wavelet analysis approach on synthetic data and large-scale neuroimaging data from ADNI. Compared with the other harmonic analysis techniques, our proposed method not only effectively predicts the early stage of AD but also provides a new window to capture the underlying hub nodes and the propagation pathways of neuropathological burdens in AD.

Estimating Outlier-Immunized Common Harmonic Waves for Brain Network Analyses on the Stiefel Manifold.

Since brain network organization is essentially governed by the harmonic waves derived from the Eigen-system of the underlying Laplacian matrix, discovering the harmonic-based alterations provides a new window to understand the pathogenic mechanism of Alzheimer's disease (AD) in a unified reference space. However, current reference (common harmonic waves) estimation studies over the individual harmonic waves are often sensitive to outliers, which are obtained by averaging the heterogenous individual brain networks. To address this challenge, we propose a novel manifold learning approach to identify a set of outlier-immunized common harmonic waves. The backbone of our framework is calculating the geometric median of all individual harmonic waves on the Stiefel manifold, instead of Fréchet mean, thus improving the robustness of learned common harmonic waves to the outliers. A manifold optimization scheme with theoretically guaranteed convergence is tailored to solve our method. The experimental results on synthetic data and real data demonstrate that the common harmonic waves learned by our approach are not only more robust to the outliers than the state-of-the-art methods, but also provide a putative imaging biomarker to predict the early stage of AD.

An Online Mammography Database with Biopsy Confirmed Types.

Breast carcinoma is the second largest cancer in the world among women. Early detection of breast cancer has been shown to increase the survival rate, thereby significantly increasing patients' lifespan. Mammography, a noninvasive imaging tool with low cost, is widely used to diagnose breast disease at an early stage due to its high sensitivity. Although some public mammography datasets are useful, there is still a lack of open access datasets that expand beyond the white population as well as missing biopsy confirmation or with unknown molecular subtypes. To fill this gap, we build a database containing two online breast mammographies. The dataset named by Chinese Mammography Database (CMMD) contains 3712 mammographies involved 1775 patients, which is divided into two branches. The first dataset CMMD1 contains 1026 cases (2214 mammographies) with biopsy confirmed type of benign or malignant tumors. The second dataset CMMD2 includes 1498 mammographies for 749 patients with known molecular subtypes. Our database is constructed to enrich the diversity of mammography data and promote the development of relevant fields.

Evolutionary route of nasopharyngeal carcinoma metastasis and its clinical significance.

It is critical to understand factors associated with nasopharyngeal carcinoma (NPC) metastasis. To track the evolutionary route of metastasis, here we perform an integrative genomic analysis of 163 matched blood and primary, regional lymph node metastasis and distant metastasis tumour samples, combined with single-cell RNA-seq on 11 samples from two patients. The mutation burden, gene mutation frequency, mutation signature, and copy number frequency are similar between metastatic tumours and primary and regional lymph node tumours. There are two distinct evolutionary routes of metastasis, including metastases evolved from regional lymph nodes (lymphatic route, 61.5%, 8/13) and from primary tumours (hematogenous route, 38.5%, 5/13). The hematogenous route is characterised by higher IFN-γ response gene expression and a higher fraction of exhausted CD8+ T cells. Based on a radiomics model, we find that the hematogenous group has significantly better progression-free survival and PD-1 immunotherapy response, while the lymphatic group has a better response to locoregional radiotherapy.