TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning.

Zero-shot learning (ZSL) tackles the novel class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Semantic knowledge is typically represented by attribute descriptions shared between different classes, which act as strong priors for localizing object attributes that represent discriminative region features, enabling significant and sufficient visual-semantic interaction for advancing ZSL. Existing attention-based models have struggled to learn inferior region features in a single image by solely using unidirectional attention, which ignore the transferable and discriminative attribute localization of visual features for representing the key semantic knowledge for effective knowledge transfer in ZSL. In this paper, we propose a cross attribute-guided Transformer network, termed TransZero++, to refine visual features and learn accurate attribute localization for key semantic knowledge representations in ZSL. Specifically, TransZero++ employs an attribute → visual Transformer sub-net (AVT) and a visual → attribute Transformer sub-net (VAT) to learn attribute-based visual features and visual-based attribute features, respectively. By further introducing feature-level and prediction-level semantical collaborative losses, the two attribute-guided transformers teach each other to learn semantic-augmented visual embeddings for key semantic knowledge representations via semantical collaborative learning. Finally, the semantic-augmented visual embeddings learned by AVT and VAT are fused to conduct desirable visual-semantic interaction cooperated with class semantic vectors for ZSL classification. Extensive experiments show that TransZero++ achieves the new state-of-the-art results on three golden ZSL benchmarks and on the large-scale ImageNet dataset. The project website is available at: https://shiming-chen.github.io/TransZero-pp/TransZero-pp.html.

Detachable interface toward a low-loss reflow-compatible fiber coupling for co-packaged optics (CPO).

High-density reflow-compatible fiber I/O is one of the challenges for co-packaged optics (CPO). This paper developed a detachable coupling interface based on expanded beam edge coupling, which can be applied for optical coupling between lasers, PICs, and fibers, seamlessly supporting many channels with high efficiency. It comprises a removable fiber connector and a permanent chip/device connector, in which microlens/lens arrays are used for waveguide mode expansion and MT-like connectors are used for position registration. An effective alignment scheme based on beam detection was developed and implemented in an assembly station for building the removable fiber connectors, while the permanent chip/device connector was assembled by active alignment to a pre-made fiber connector mated with a registration connector. Promising results were obtained from the proof-of-concept demonstrations of the coupling from SiP PIC and III/V lasers to fibers using the off-the-shelf lenses and modified MT registration connectors. In both cases, less than 1 dB coupling loss was achieved with an expanded beam size of 160 µm in diameter. Even with a relatively large lens offset of ∼35 µm, the detachable fiber array connectors showed good interchangeability. Such a coupling interface is expected to be solder-reflow compatible by replacing the plastic registration connectors with ceramic ones, making it a promising candidate for the solution to CPO fiber I/O.

Design of two-dimensional sampled Bragg grating for a curved waveguide.

Due to the ability of changing light propagation path direction, curved waveguide Bragg grating (CWG) plays an important role in photonic integrated circuits. In this paper, we proposed a cascaded sampled Bragg grating on tilted waveguide (CSBG-TW) structure to equivalently realize CWG. As an example, by designing two-dimensional (2D) sampled gratings, the direction of +1st sub-grating vector in CSBG-TW can be changed. Then if a curved waveguide is divided into several sections of tilted waveguide, we can keep the grating direction being always parallel to the longitudinal direction of each section of tilted waveguide, while the basic grating is uniform. Hence, the required CWG can be equivalently realized, and the light responses such as reflection Bragg wavelength shift and backward mode convert caused by the tilted grating in curved waveguide can be compensated for. The results show that the sampling structures of CSBG-TW is micro-scale and the difference between reflection intensity between the CSBG-TW with four section tilted waveguide and CWG as design target is less than 0.1 dB. Compared with CWG, the CSBG-TW allows convenient holographic exposure and the wavelength can be accurately controlled. Therefore, the CSBG-TW can be used in various photonic integrated devices that require changing propagation paths.

GNDAN: Graph Navigated Dual Attention Network for Zero-Shot Learning.

Zero-shot learning (ZSL) tackles the unseen class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Typically, to guarantee desirable knowledge transfer, a direct embedding is adopted for associating the visual and semantic domains in ZSL. However, most existing ZSL methods focus on learning the embedding from implicit global features or image regions to the semantic space. Thus, they fail to: 1) exploit the appearance relationship priors between various local regions in a single image, which corresponds to the semantic information and 2) learn cooperative global and local features jointly for discriminative feature representations. In this article, we propose the novel graph navigated dual attention network (GNDAN) for ZSL to address these drawbacks. GNDAN employs a region-guided attention network (RAN) and a region-guided graph attention network (RGAT) to jointly learn a discriminative local embedding and incorporate global context for exploiting explicit global embeddings under the guidance of a graph. Specifically, RAN uses soft spatial attention to discover discriminative regions for generating local embeddings. Meanwhile, RGAT employs an attribute-based attention to obtain attribute-based region features, where each attribute focuses on the most relevant image regions. Motivated by the graph neural network (GNN), which is beneficial for structural relationship representations, RGAT further leverages a graph attention network to exploit the relationships between the attribute-based region features for explicit global embedding representations. Based on the self-calibration mechanism, the joint visual embedding learned is matched with the semantic embedding to form the final prediction. Extensive experiments on three benchmark datasets demonstrate that the proposed GNDAN achieves superior performances to the state-of-the-art methods. Our code and trained models are available at https://github.com/shiming-chen/GNDAN.

Experimental demonstration of a photonic convolutional accelerator based on a monolithically integrated multi-wavelength distributed feedback laser.

We propose and experimentally demonstrate a simple and energy-efficient photonic convolutional accelerator based on a monolithically integrated multi-wavelength distributed feedback semiconductor laser using the superimposed sampled Bragg grating structure. The photonic convolutional accelerator operates at 44.48 GOPS for one 2 × 2 kernel with a convolutional window vertical sliding stride of 2 and generates 100 images of real-time recognition. Furthermore, a real-time recognition task on the MNIST database of handwritten digits with a prediction accuracy of 84% is achieved. This work provides a compact and low-cost way to realize photonic convolutional neural networks.

IAS-NET: Joint intraclassly adaptive GAN and segmentation network for unsupervised cross-domain in neonatal brain MRI segmentation.

PURPOSE: In neonatal brain magnetic resonance image (MRI) segmentation, the model we trained on the training set (source domain) often performs poorly in clinical practice (target domain). As the label of target-domain images is unavailable, this cross-domain segmentation needs unsupervised domain adaptation (UDA) to make the model adapt to the target domain. However, the shape and intensity distribution of neonatal brain MRI images across the domains are largely different from adults'. Current UDA methods aim to make synthesized images similar to the target domain as a whole. But it is impossible to synthesize images with intraclass similarity because of the regional misalignment caused by the cross-domain difference. This will result in generating intraclassly incorrect intensity information from target-domain images. To address this issue, we propose an IAS-NET (joint intraclassly adaptive generative adversarial network (GAN) (IA-NET) and segmentation) framework to bridge the gap between the two domains for intraclass alignment. METHODS: Our proposed IAS-NET is an elegant learning framework that transfers the appearance of images across the domains from both image and feature perspectives. It consists of the proposed IA-NET and a segmentation network (S-NET). The proposed IA-NET is a GAN-based adaptive network that contains one generator (including two encoders and one shared decoder) and four discriminators for cross-domain transfer. The two encoders are implemented to extract original image, mean, and variance features from source and target domains. The proposed local adaptive instance normalization algorithm is used to perform intraclass feature alignment to the target domain in the feature-map level. S-NET is a U-net structure network that is used to provide semantic constraint by a segmentation loss for the training of IA-NET. Meanwhile, it offers pseudo-label images for calculating intraclass features of the target domain. Source code (in Tensorflow) is available at https://github.com/lb-whu/RAS-NET/. RESULTS: Extensive experiments are carried out on two different data sets (NeoBrainS12 and dHCP), respectively. There exist great differences in the shape, size, and intensity distribution of magnetic resonance (MR) images in the two databases. Compared to baseline, we improve the average dice score of all tissues on NeoBrains12 by 6% through adaptive training with unlabeled dHCP images. Besides, we also conduct experiments on dHCP and improved the average dice score by 4%. The quantitative analysis of the mean and variance of the synthesized images shows that the synthesized image by the proposed is closer to the target domain both in the full brain or within each class than that of the compared methods. CONCLUSIONS: In this paper, the proposed IAS-NET can improve the performance of the S-NET effectively by its intraclass feature alignment in the target domain. Compared to the current UDA methods, the synthesized images by IAS-NET are more intraclassly similar to the target domain for neonatal brain MR images. Therefore, it achieves state-of-the-art results in the compared UDA models for the segmentation task.

A machine-learning-based method to predict adverse events in patients with dilated cardiomyopathy and severely reduced ejection fractions.

OBJECTIVE: Patients with dilated cardiomyopathy (DCM) and severely reduced left ventricular ejection fractions (LVEFs) are at very high risks of experiencing adverse cardiac events. A machine learning (ML) method could enable more effective risk stratification for these high-risk patients by incorporating various types of data. The aim of this study was to build an ML model to predict adverse events including all-cause deaths and heart transplantation in DCM patients with severely impaired LV systolic function. METHODS: One hundred and eighteen patients with DCM and severely reduced LVEFs (<35%) were included. The baseline clinical characteristics, laboratory data, electrocardiographic, and cardiac magnetic resonance (CMR) features were collected. Various feature selection processes and classifiers were performed to select an ML model with the best performance. The predictive performance of tested ML models was evaluated using the area under the curve (AUC) of the receiver operating characteristic curve using 10-fold cross-validation. RESULTS: Twelve patients died, and 17 patients underwent heart transplantation during the median follow-up of 508 days. The ML model included systolic blood pressure, left ventricular end-systolic and end-diastolic volume indices, and late gadolinium enhancement (LGE) extents on CMR imaging, and a support vector machine was selected as a classifier. The model showed excellent performance in predicting adverse events in DCM patients with severely reduced LVEF (the AUC and accuracy values were 0.873 and 0.763, respectively). CONCLUSIONS: This ML technique could effectively predict adverse events in DCM patients with severely reduced LVEF. ADVANCES IN KNOWLEDGE: The ML method has superior ability in risk stratification in severe DCM patients.

Prognostic Value of Late Enhanced Cardiac Magnetic Resonance Imaging Derived Texture Features in Dilated Cardiomyopathy Patients With Severely Reduced Ejection Fractions.

Background: Late enhanced cardiac magnetic resonance (CMR) images of the left ventricular myocardium contain an enormous amount of information that could provide prognostic value beyond that of late gadolinium enhancements (LGEs). With computational postprocessing and analysis, the heterogeneities and variations of myocardial signal intensities can be interpreted and measured as texture features. This study aimed to evaluate the value of texture features extracted from late enhanced CMR images of the myocardium to predict adverse outcomes in patients with dilated cardiomyopathy (DCM) and severe systolic dysfunction. Methods: This single-center study retrospectively enrolled patients with DCM with severely reduced left ventricular ejection fractions (LVEFs < 35%). Texture features were extracted from enhanced late scanning images, and the presence and extent of LGEs were also measured. Patients were followed-up for clinical endpoints composed of all-cause deaths and cardiac transplantation. Cox proportional hazard regression and Kaplan-Meier analyses were used to evaluate the prognostic value of texture features and conventional CMR parameters with event-free survival. Results: A total of 114 patients (37 women, median age 47.5 years old) with severely impaired systolic function (median LVEF, 14.0%) were followed-up for a median of 504.5 days. Twenty-nine patients experienced endpoint events, 12 died, and 17 underwent cardiac transplantations. Three texture features from a gray-level co-occurrence matrix (GLCM) (GLCM_contrast, GLCM_difference average, and GLCM_difference entropy) showed good prognostic value for adverse events when analyzed using univariable Cox hazard ratio regression (p = 0.007, p = 0.011, and p = 0.007, retrospectively). When each of the three features was analyzed using a multivariable Cox regression model that included the clinical parameter (systolic blood pressure) and LGE extent, they were found to be independently associated with adverse outcomes. Conclusion: Texture features related LGE heterogeneities and variations (GLCM_contrast, GLCM_difference average, and GLCM_difference entropy) are novel markers for risk stratification toward adverse events in DCM patients with severe systolic dysfunction.

Opportunities and Challenges for Chinese Elderly Care Industry in Smart Environment Based on Occupants' Needs and Preferences.

New developments in intelligent devices for assisting elderly people can provide elders with friendly, mutual, and personalized interactions. Since the intelligent devices should continually make an important contribution to the smart elderly care industry, smart services or policies for the elders are recently provided by a large number of government programs in China. At present, the smart elderly care industry in China has attracted numerous investors' attention, but the smart elderly care industry in China is still at the beginning stage. Though there are great opportunities in the market, many challenges and limitations still need to be solved. This study analyzes 198 news reports about opportunities and challenges in the smart elderly care industry from six major Chinese portals. The analysis is mainly based on needs assessment for elderly people, service providers, and the Chinese government. It is concluded that smart elderly care services satisfy the elders' mental wants and that needs for improving modernization services are the most frequently mentioned opportunities. Also, the frequently mentioned challenges behind opportunities are intelligent products not being able to solve the just-needed, user-consumption concept and the ability to pay, which is the most frequently mentioned challenge. The results of this study will enable stakeholders in the smart elderly care industry to clarify the opportunities and challenges faced by smart elderly care services in China's development process and provide a theoretical basis for better decision making.