Radioport: a radiomics-reporting network for interpretable deep learning in BI-RADS classification of mammographic calcification.

Objective.Generally, due to a lack of explainability, radiomics based on deep learning has been perceived as a black-box solution for radiologists. Automatic generation of diagnostic reports is a semantic approach to enhance the explanation of deep learning radiomics (DLR).Approach.In this paper, we propose a novel model called radiomics-reporting network (Radioport), which incorporates text attention. This model aims to improve the interpretability of DLR in mammographic calcification diagnosis. Firstly, it employs convolutional neural networks to extract visual features as radiomics for multi-category classification based on breast imaging reporting and data system. Then, it builds a mapping between these visual features and textual features to generate diagnostic reports, incorporating an attention module for improved clarity.Main results.To demonstrate the effectiveness of our proposed model, we conducted experiments on a breast calcification dataset comprising mammograms and diagnostic reports. The results demonstrate that our model can: (i) semantically enhance the interpretability of DLR; and, (ii) improve the readability of generated medical reports.Significance.Our interpretable textual model can explicitly simulate the mammographic calcification diagnosis process.

Image collection and annotation platforms to establish a multi-source database of oral lesions.

OBJECTIVE: To describe the development of a platform for image collection and annotation that resulted in a multi-sourced international image dataset of oral lesions to facilitate the development of automated lesion classification algorithms. MATERIALS AND METHODS: We developed a web-interface, hosted on a web server to collect oral lesions images from international partners. Further, we developed a customised annotation tool, also a web-interface for systematic annotation of images to build a rich clinically labelled dataset. We evaluated the sensitivities comparing referral decisions through the annotation process with the clinical diagnosis of the lesions. RESULTS: The image repository hosts 2474 images of oral lesions consisting of oral cancer, oral potentially malignant disorders and other oral lesions that were collected through MeMoSA® UPLOAD. Eight-hundred images were annotated by seven oral medicine specialists on MeMoSA® ANNOTATE, to mark the lesion and to collect clinical labels. The sensitivity in referral decision for all lesions that required a referral for cancer management/surveillance was moderate to high depending on the type of lesion (64.3%-100%). CONCLUSION: This is the first description of a database with clinically labelled oral lesions. This database could accelerate the improvement of AI algorithms that can promote the early detection of high-risk oral lesions.

DeepIPR: Deep Neural Network Ownership Verification With Passports.

With substantial amount of time, resources and human (team) efforts invested to explore and develop successful deep neural networks (DNN), there emerges an urgent need to protect these inventions from being illegally copied, redistributed, or abused without respecting the intellectual properties of legitimate owners. Following recent progresses along this line, we investigate a number of watermark-based DNN ownership verification methods in the face of ambiguity attacks, which aim to cast doubts on the ownership verification by forging counterfeit watermarks. It is shown that ambiguity attacks pose serious threats to existing DNN watermarking methods. As remedies to the above-mentioned loophole, this paper proposes novel passport-based DNN ownership verification schemes which are both robust to network modifications and resilient to ambiguity attacks. The gist of embedding digital passports is to design and train DNN models in a way such that, the DNN inference performance of an original task will be significantly deteriorated due to forged passports. In other words, genuine passports are not only verified by looking for the predefined signatures, but also reasserted by the unyielding DNN model inference performances. Extensive experimental results justify the effectiveness of the proposed passport-based DNN ownership verification schemes. Code is available at https://github.com/kamwoh/DeepIPR.

Utility and usability of an automated COVID-19 symptom monitoring system (CoSMoS) in primary care during COVID-19 pandemic: A qualitative feasibility study.

BACKGROUND: COVID-19 telemonitoring applications have been developed and used in primary care to monitor patients quarantined at home. There is a lack of evidence on the utility and usability of telemonitoring applications from end-users' perspective. OBJECTIVES: This study aimed to evaluate the feasibility of a COVID-19 symptom monitoring system (CoSMoS) by exploring its utility and usability with end-users. METHODS: This was a qualitative study using in-depth interviews. Patients with suspected COVID-19 infection who used CoSMoS Telegram bot to monitor their COVID-19 symptoms and doctors who conducted the telemonitoring via CoSMoS dashboard were recruited. Universal sampling was used in this study. We stopped the recruitment when data saturation was reached. Patients and doctors shared their experiences using CoSMoS, its utility and usability for COVID-19 symptoms monitoring. Data were coded and analysed using thematic analysis. RESULTS: A total of 11 patients and 4 doctors were recruited into this study. For utility, CoSMoS was useful in providing close monitoring and continuity of care, supporting patients' decision making, ensuring adherence to reporting, and reducing healthcare workers' burden during the pandemic. In terms of usability, patients expressed that CoSMoS was convenient and easy to use. The use of the existing social media application for symptom monitoring was acceptable for the patients. The content in the Telegram bot was easy to understand, although revision was needed to keep the content updated. Doctors preferred to integrate CoSMoS into the electronic medical record. CONCLUSION: CoSMoS is feasible and useful to patients and doctors in providing remote monitoring and teleconsultation during the COVID-19 pandemic. The utility and usability evaluation enables the refinement of CoSMoS to be a patient-centred monitoring system.

Relating molecular descriptors to frontier orbital energy levels, singlet and triplet excited states of fused tricyclics using machine learning.

Fused tricyclic organic compounds are an important class of organic electronic materials. In designing molecules for organic electronics, knowing what chemical structure that be used to tune the molecular property is one of the keys that can help to improve the material performance. In this research, we applied machine learning and data analytic approaches in addressing this problem. The energy states (Lowest Unoccupied Molecular Orbital (HOMO), Highest Occupied Molecular Orbitals (LUMO), singlet (Es) and triplet (ET) energy) of more than 10 thousand fused tricyclics are calculated. Corresponding descriptors are also generated. We find that the Coulomb matrix is a poorer descriptor than high-level descriptors in a multilayer perceptron neural network. Correlations as high as 0.95 is obtained using a multilayer perceptron neural network with Mean Absolute Error as low as 0.08 eV. The descriptors that are important in tuning the energy levels are revealed using the Random Forest algorithm. Correlations of such descriptors are also plotted. We found that the higher the number of tertiary amines, the deeper are the HOMO and LUMO levels. The presence of NN in the aromatic rings can be used to tune the ES. However, there is no single dominant descriptor that can be correlated with the ET. A collection of descriptors is found to give a far better correlation with ET. This research demonstrated that machine learning and data analytics in guiding how certain chemical substructures correlate with the molecule energy states.

Semi-supervised GAN-based Radiomics Model for Data Augmentation in Breast Ultrasound Mass Classification.

BACKGROUND AND OBJECTIVE: The capability of deep learning radiomics (DLR) to extract high-level medical imaging features has promoted the use of computer-aided diagnosis of breast mass detected on ultrasound. Recently, generative adversarial network (GAN) has aided in tackling a general issue in DLR, i.e., obtaining a sufficient number of medical images. However, GAN methods require a pair of input and labeled images, which require an exhaustive human annotation process that is very time-consuming. The aim of this paper is to develop a radiomics model based on a semi-supervised GAN method to perform data augmentation in breast ultrasound images. METHODS: A total of 1447 ultrasound images, including 767 benign masses and 680 malignant masses were acquired from a tertiary hospital. A semi-supervised GAN model was developed to augment the breast ultrasound images. The synthesized images were subsequently used to classify breast masses using a convolutional neural network (CNN). The model was validated using a 5-fold cross-validation method. RESULTS: The proposed GAN architecture generated high-quality breast ultrasound images, verified by two experienced radiologists. The improved performance of semi-supervised learning increased the quality of the synthetic data produced in comparison to the baseline method. We achieved more accurate breast mass classification results (accuracy 90.41%, sensitivity 87.94%, specificity 85.86%) with our synthetic data augmentation compared to other state-of-the-art methods. CONCLUSION: The proposed radiomics model has demonstrated a promising potential to synthesize and classify breast masses on ultrasound in a semi-supervised manner.

An Automated Patient Self-Monitoring System to Reduce Health Care System Burden During the COVID-19 Pandemic in Malaysia: Development and Implementation Study.

BACKGROUND: During the COVID-19 pandemic, there was an urgent need to develop an automated COVID-19 symptom monitoring system to reduce the burden on the health care system and to provide better self-monitoring at home. OBJECTIVE: This paper aimed to describe the development process of the COVID-19 Symptom Monitoring System (CoSMoS), which consists of a self-monitoring, algorithm-based Telegram bot and a teleconsultation system. We describe all the essential steps from the clinical perspective and our technical approach in designing, developing, and integrating the system into clinical practice during the COVID-19 pandemic as well as lessons learned from this development process. METHODS: CoSMoS was developed in three phases: (1) requirement formation to identify clinical problems and to draft the clinical algorithm, (2) development testing iteration using the agile software development method, and (3) integration into clinical practice to design an effective clinical workflow using repeated simulations and role-playing. RESULTS: We completed the development of CoSMoS in 19 days. In Phase 1 (ie, requirement formation), we identified three main functions: a daily automated reminder system for patients to self-check their symptoms, a safe patient risk assessment to guide patients in clinical decision making, and an active telemonitoring system with real-time phone consultations. The system architecture of CoSMoS involved five components: Telegram instant messaging, a clinician dashboard, system administration (ie, back end), a database, and development and operations infrastructure. The integration of CoSMoS into clinical practice involved the consideration of COVID-19 infectivity and patient safety. CONCLUSIONS: This study demonstrated that developing a COVID-19 symptom monitoring system within a short time during a pandemic is feasible using the agile development method. Time factors and communication between the technical and clinical teams were the main challenges in the development process. The development process and lessons learned from this study can guide the future development of digital monitoring systems during the next pandemic, especially in developing countries.

Improved ArtGAN for Conditional Synthesis of Natural Image and Artwork.

This paper proposes a series of new approaches to improve Generative Adversarial Network (GAN) for conditional image synthesis and we name the proposed model as "ArtGAN". One of the key innovation of ArtGAN is that, the gradient of the loss function w.r.t. the label (randomly assigned to each generated image) is back-propagated from the categorical discriminator to the generator. With the feedback from the label information, the generator is able to learn more efficiently and generate image with better quality. Inspired by recent works, an autoencoder is incorporated into the categorical discriminator for additional complementary information. Last but not least, we introduce a novel strategy to improve the image quality. In the experiments, we evaluate ArtGAN on CIFAR-10 and STL-10 via ablation studies. The empirical results showed that our proposed model outperforms the state-of-the-art results on CIFAR-10 in terms of Inception score. Qualitatively, we demonstrate that ArtGAN is able to generate plausible-looking images on Oxford-102 and CUB-200, as well as able to draw realistic artworks based on style, artist, and genre. The source code and models are available at: https://github.com/cs-chan/ArtGAN.

Multi-Organ Plant Classification Based on Convolutional and Recurrent Neural Networks.

Classification of plants based on a multi-organ approach is very challenging. Although additional data provide more information that might help to disambiguate between species, the variability in shape and appearance in plant organs also raises the degree of complexity of the problem. Despite promising solutions built using deep learning enable representative features to be learned for plant images, the existing approaches focus mainly on generic features for species classification, disregarding the features representing plant organs. In fact, plants are complex living organisms sustained by a number of organ systems. In our approach, we introduce a hybrid generic-organ convolutional neural network (HGO-CNN), which takes into account both organ and generic information, combining them using a new feature fusion scheme for species classification. Next, instead of using a CNN-based method to operate on one image with a single organ, we extend our approach. We propose a new framework for plant structural learning using the recurrent neural network-based method. This novel approach supports classification based on a varying number of plant views, capturing one or more organs of a plant, by optimizing the contextual dependencies between them. We also present the qualitative results of our proposed models based on feature visualization techniques and show that the outcomes of visualizations depict our hypothesis and expectation. Finally, we show that by leveraging and combining the aforementioned techniques, our best network outperforms the state of the art on the PlantClef2015 benchmark. The source code and models are available at https://github.com/cs-chan/Deep-Plant.

GrCS: Granular Computing-Based Crowd Segmentation.

Crowd segmentation is important in serving as the basis for a wide range of crowd analysis tasks such as density estimation and behavior understanding. However, due to interocclusions, perspective distortion, clutter background, and random crowd distribution, localizing crowd segments is technically a very challenging task. This paper proposes a novel crowd segmentation framework-based on granular computing (GrCS) to enable the problem of crowd segmentation to be conceptualized at different levels of granularity, and to map problems into computationally tractable subproblems. It shows that by exploiting the correlation among pixel granules, we are able to aggregate structurally similar pixels into meaningful atomic structure granules. This is useful in outlining natural boundaries between crowd and background (i.e., noncrowd) regions. From the structure granules, we infer the crowd and background regions by granular information classification. GrCS is scene-independent and can be applied effectively to crowd scenes with a variety of physical layout and crowdedness. Extensive experiments have been conducted on hundreds of real and synthetic crowd scenes. The results demonstrate that by exploiting the correlation among granules, we can outline the natural boundaries of structurally similar crowd and background regions necessary for crowd segmentation.

Assessing agreement in measurements of orthodontic study models: Digital caliper on plaster models vs 3-dimensional software on models scanned by structured-light scanner.

INTRODUCTION: In this study we aimed to compare measurements on plaster models using a digital caliper, and on 3-dimensional (3D) digital models, produced using a structured-light scanner, using 3D software. METHODS: Fifty digital models were scanned from the same plaster models. Arch and tooth size measurements were made by 2 operators, twice. Calibration was done on 10 sets of models and checked using the Pearson correlation coefficient. Data were analyzed by error variances, repeatability coefficient, repeated-measures analysis of variance, and Bland-Altman plots. RESULTS: Error variances ranged between 0.001 and 0.044 mm for the digital caliper method, and between 0.002 and 0.054 mm for the 3D software method. Repeated-measures analysis of variance showed small but statistically significant differences (P <0.05) between the repeated measurements in the arch and buccolingual planes (0.011 and 0.008 mm, respectively). There were no statistically significant differences between methods and between operators. Bland-Altman plots showed that the mean biases were close to zero, and the 95% limits of agreement were within ±0.50 mm. Repeatability coefficients for all measurements were similar. CONCLUSIONS: Measurements made on models scanned by the 3D structured-light scanner were in good agreement with those made on conventional plaster models and were, therefore, clinically acceptable.

User acceptance of a touchless sterile system to control virtual orthodontic study models.

INTRODUCTION: In this article, we present an evaluation of user acceptance of our innovative hand-gesture-based touchless sterile system for interaction with and control of a set of 3-dimensional digitized orthodontic study models using the Kinect motion-capture sensor (Microsoft, Redmond, Wash). METHODS: The system was tested on a cohort of 201 participants. Using our validated questionnaire, the participants evaluated 7 hand-gesture-based commands that allowed the user to adjust the model in size, position, and aspect and to switch the image on the screen to view the maxillary arch, the mandibular arch, or models in occlusion. Participants' responses were assessed using Rasch analysis so that their perceptions of the usefulness of the hand gestures for the commands could be directly referenced against their acceptance of the gestures. Their perceptions of the potential value of this system for cross-infection control were also evaluated. RESULTS: Most participants endorsed these commands as accurate. Our designated hand gestures for these commands were generally accepted. We also found a positive and significant correlation between our participants' level of awareness of cross infection and their endorsement to use this system in clinical practice. CONCLUSIONS: This study supports the adoption of this promising development for a sterile touch-free patient record-management system.

Recurrent Fusion Genes in Gastric Cancer: CLDN18-ARHGAP26 Induces Loss of Epithelial Integrity.

Genome rearrangements, a hallmark of cancer, can result in gene fusions with oncogenic properties. Using DNA paired-end-tag (DNA-PET) whole-genome sequencing, we analyzed 15 gastric cancers (GCs) from Southeast Asians. Rearrangements were enriched in open chromatin and shaped by chromatin structure. We identified seven rearrangement hot spots and 136 gene fusions. In three out of 100 GC cases, we found recurrent fusions between CLDN18, a tight junction gene, and ARHGAP26, a gene encoding a RHOA inhibitor. Epithelial cell lines expressing CLDN18-ARHGAP26 displayed a dramatic loss of epithelial phenotype and long protrusions indicative of epithelial-mesenchymal transition (EMT). Fusion-positive cell lines showed impaired barrier properties, reduced cell-cell and cell-extracellular matrix adhesion, retarded wound healing, and inhibition of RHOA. Gain of invasion was seen in cancer cell lines expressing the fusion. Thus, CLDN18-ARHGAP26 mediates epithelial disintegration, possibly leading to stomach H(+) leakage, and the fusion might contribute to invasiveness once a cell is transformed.

Long span DNA paired-end-tag (DNA-PET) sequencing strategy for the interrogation of genomic structural mutations and fusion-point-guided reconstruction of amplicons.

Structural variations (SVs) contribute significantly to the variability of the human genome and extensive genomic rearrangements are a hallmark of cancer. While genomic DNA paired-end-tag (DNA-PET) sequencing is an attractive approach to identify genomic SVs, the current application of PET sequencing with short insert size DNA can be insufficient for the comprehensive mapping of SVs in low complexity and repeat-rich genomic regions. We employed a recently developed procedure to generate PET sequencing data using large DNA inserts of 10-20 kb and compared their characteristics with short insert (1 kb) libraries for their ability to identify SVs. Our results suggest that although short insert libraries bear an advantage in identifying small deletions, they do not provide significantly better breakpoint resolution. In contrast, large inserts are superior to short inserts in providing higher physical genome coverage for the same sequencing cost and achieve greater sensitivity, in practice, for the identification of several classes of SVs, such as copy number neutral and complex events. Furthermore, our results confirm that large insert libraries allow for the identification of SVs within repetitive sequences, which cannot be spanned by short inserts. This provides a key advantage in studying rearrangements in cancer, and we show how it can be used in a fusion-point-guided-concatenation algorithm to study focally amplified regions in cancer.

CTCF-mediated functional chromatin interactome in pluripotent cells.

Mammalian genomes are viewed as functional organizations that orchestrate spatial and temporal gene regulation. CTCF, the most characterized insulator-binding protein, has been implicated as a key genome organizer. However, little is known about CTCF-associated higher-order chromatin structures at a global scale. Here we applied chromatin interaction analysis by paired-end tag (ChIA-PET) sequencing to elucidate the CTCF-chromatin interactome in pluripotent cells. From this analysis, we identified 1,480 cis- and 336 trans-interacting loci with high reproducibility and precision. Associating these chromatin interaction loci with their underlying epigenetic states, promoter activities, enhancer binding and nuclear lamina occupancy, we uncovered five distinct chromatin domains that suggest potential new models of CTCF function in chromatin organization and transcriptional control. Specifically, CTCF interactions demarcate chromatin-nuclear membrane attachments and influence proper gene expression through extensive cross-talk between promoters and regulatory elements. This highly complex nuclear organization offers insights toward the unifying principles that govern genome plasticity and function.

Comprehensive long-span paired-end-tag mapping reveals characteristic patterns of structural variations in epithelial cancer genomes.

Somatic genome rearrangements are thought to play important roles in cancer development. We optimized a long-span paired-end-tag (PET) sequencing approach using 10-Kb genomic DNA inserts to study human genome structural variations (SVs). The use of a 10-Kb insert size allows the identification of breakpoints within repetitive or homology-containing regions of a few kilobases in size and results in a higher physical coverage compared with small insert libraries with the same sequencing effort. We have applied this approach to comprehensively characterize the SVs of 15 cancer and two noncancer genomes and used a filtering approach to strongly enrich for somatic SVs in the cancer genomes. Our analyses revealed that most inversions, deletions, and insertions are germ-line SVs, whereas tandem duplications, unpaired inversions, interchromosomal translocations, and complex rearrangements are over-represented among somatic rearrangements in cancer genomes. We demonstrate that the quantitative and connective nature of DNA-PET data is precise in delineating the genealogy of complex rearrangement events, we observe signatures that are compatible with breakage-fusion-bridge cycles, and we discover that large duplications are among the initial rearrangements that trigger genome instability for extensive amplification in epithelial cancers.

Deconvolution of chromatin immunoprecipitation-microarray (ChIP-chip) analysis of MBF occupancies reveals the temporal recruitment of Rep2 at the MBF target genes.

MBF (or DSC1) is known to regulate transcription of a set of G(1)/S-phase genes encoding proteins involved in regulation of DNA replication. Previous studies have shown that MBF binds not only the promoter of G(1)/S-phase genes, but also the constitutive genes; however, it was unclear if the MBF bindings at the G(1)/S-phase and constitutive genes were mechanistically distinguishable. Here, we report a chromatin immunoprecipitation-microarray (ChIP-chip) analysis of MBF binding in the Schizosaccharomyces pombe genome using high-resolution genome tiling microarrays. ChIP-chip analysis indicates that the majority of the MBF occupancies are located at the intragenic regions. Deconvolution analysis using Rpb1 ChIP-chip results distinguishes the Cdc10 bindings at the Rpb1-poor loci (promoters) from those at the Rpb1-rich loci (intragenic sequences). Importantly, Res1 binding at the Rpb1-poor loci, but not at the Rpb1-rich loci, is dependent on the Cdc10 function, suggesting a distinct binding mechanism. Most Cdc10 promoter bindings at the Rpb1-poor loci are associated with the G(1)/S-phase genes. While Res1 or Res2 is found at both the Cdc10 promoter and intragenic binding sites, Rep2 appears to be absent at the Cdc10 promoter binding sites but present at the intragenic sites. Time course ChIP-chip analysis demonstrates that Rep2 is temporally accumulated at the coding region of the MBF target genes, resembling the RNAP-II occupancies. Taken together, our results show that deconvolution analysis of Cdc10 occupancies refines the functional subset of genomic binding sites. We propose that the MBF activator Rep2 plays a role in mediating the cell cycle-specific transcription through the recruitment of RNAP-II to the MBF-bound G(1)/S-phase genes.