Parallel Radars: From Digital Twins to Digital Intelligence for Smart Radar Systems.

Radar is widely employed in many applications, especially in autonomous driving. At present, radars are only designed as simple data collectors, and they are unable to meet new requirements for real-time and intelligent information processing as environmental complexity increases. It is inevitable that smart radar systems will need to be developed to deal with these challenges and digital twins in cyber-physical systems (CPS) have proven to be effective tools in many aspects. However, human involvement is closely related to radar technology and plays an important role in the operation and management of radars; thus, digital twins' radars in CPS are insufficient to realize smart radar systems due to the inadequate consideration of human factors. ACP-based parallel intelligence in cyber-physical-social systems (CPSS) is used to construct a novel framework for smart radars, called Parallel Radars. A Parallel Radar consists of three main parts: a Descriptive Radar for constructing artificial radar systems in cyberspace, a Predictive Radar for conducting computational experiments with artificial systems, and a Prescriptive Radar for providing prescriptive control to both physical and artificial radars to complete parallel execution. To connect silos of data and protect data privacy, federated radars are proposed. Additionally, taking mines as an example, the application of Parallel Radars in autonomous driving is discussed in detail, and various experiments have been conducted to demonstrate the effectiveness of Parallel Radars.

Pay attention to doctor-patient dialogues: Multi-modal knowledge graph attention image-text embedding for COVID-19 diagnosis.

The sudden increase in coronavirus disease 2019 (COVID-19) cases puts high pressure on healthcare services worldwide. At this stage, fast, accurate, and early clinical assessment of the disease severity is vital. In general, there are two issues to overcome: (1) Current deep learning-based works suffer from multimodal data adequacy issues; (2) In this scenario, multimodal (e.g., text, image) information should be taken into account together to make accurate inferences. To address these challenges, we propose a multi-modal knowledge graph attention embedding for COVID-19 diagnosis. Our method not only learns the relational embedding from nodes in a constituted knowledge graph but also has access to medical knowledge, aiming at improving the performance of the classifier through the mechanism of medical knowledge attention. The experimental results show that our approach significantly improves classification performance compared to other state-of-the-art techniques and possesses robustness for each modality from multi-modal data. Moreover, we construct a new COVID-19 multi-modal dataset based on text mining, consisting of 1393 doctor-patient dialogues and their 3706 images (347 X-ray + 2598 CT + 761 ultrasound) about COVID-19 patients and 607 non-COVID-19 patient dialogues and their 10754 images (9658 X-ray + 494 CT + 761 ultrasound), and the fine-grained labels of all. We hope this work can provide insights to the researchers working in this area to shift the attention from only medical images to the doctor-patient dialogue and its corresponding medical images.

A novel framework of collaborative early warning for COVID-19 based on blockchain and smart contracts.

Early warning is a vital component of emergency response systems for infectious diseases. However, most early warning systems are centralized and isolated, thus there are potential risks of single evidence bias and decision-making errors. In this paper, we tackle this issue via proposing a novel framework of collaborative early warning for COVID-19 based on blockchain and smart contracts, aiming to crowdsource early warning tasks to distributed channels including medical institutions, social organizations, and even individuals. Our framework supports two surveillance modes, namely, medical federation surveillance based on federated learning and social collaboration surveillance based on the learning markets approach, and fuses their monitoring results on emerging cases to alert. By using our framework, medical institutions are expected to obtain better federated surveillance models with privacy protection, and social participants without mutual trusts can also share verified surveillance resources such as data and models, and fuse their surveillance solutions. We implemented our proposed framework based on the Ethereum and IPFS platforms. Experimental results show that our framework has advantages of decentralized decision-making, fairness, auditability, and universality. It also has potential guidance and reference value for the early warning and prevention of unknown infectious diseases.

Learning to learn by yourself: Unsupervised meta-learning with self-knowledge distillation for COVID-19 diagnosis from pneumonia cases.

The goal of diagnosing the coronavirus disease 2019 (COVID-19) from suspected pneumonia cases, that is, recognizing COVID-19 from chest X-ray or computed tomography (CT) images, is to improve diagnostic accuracy, leading to faster intervention. The most important and challenging problem here is to design an effective and robust diagnosis model. To this end, there are three challenges to overcome: (1) The lack of training samples limits the success of existing deep-learning-based methods. (2) Many public COVID-19 data sets contain only a few images without fine-grained labels. (3) Due to the explosive growth of suspected cases, it is urgent and important to diagnose not only COVID-19 cases but also the cases of other types of pneumonia that are similar to the symptoms of COVID-19. To address these issues, we propose a novel framework called Unsupervised Meta-Learning with Self-Knowledge Distillation to address the problem of differentiating COVID-19 from pneumonia cases. During training, our model cannot use any true labels and aims to gain the ability of learning to learn by itself. In particular, we first present a deep diagnosis model based on a relation network to capture and memorize the relation among different images. Second, to enhance the performance of our model, we design a self-knowledge distillation mechanism that distills knowledge within our model itself. Our network is divided into several parts, and the knowledge in the deeper parts is squeezed into the shallow ones. The final results are derived from our model by learning to compare the features of images. Experimental results demonstrate that our approach achieves significantly higher performance than other state-of-the-art methods. Moreover, we construct a new COVID-19 pneumonia data set based on text mining, consisting of 2696 COVID-19 images (347 X-ray + 2349 CT), 10,155 images (9661 X-ray + 494 CT) about other types of pneumonia, and the fine-grained labels of all. Our data set considers not only a bacterial infection or viral infection which causes pneumonia but also a viral infection derived from the influenza virus or coronavirus.

A Novel Parallel Control Method for Optimal Consensus of Nonlinear Multiagent Systems.

This work concentrates on the initial introduction of parallel control to investigate an optimal consensus control strategy for continuous-time nonlinear multiagent systems (MASs) via adaptive dynamic programming (ADP). First, the control input is integrated into the feedback system for parallel control, facilitating an augmented system's optimal consensus control with an appropriate augmented performance index function to be established, which is identical to the original system's suboptimal control with a conventional performance index. Second, the feasibility of the proposed control scheme is evaluated based on the policy iteration algorithm, and the convergence of the algorithm is demonstrated. Then, an online learning algorithm becomes available to implement the ADP-based optimal parallel consensus control protocol without prior knowledge of the system. The Lyapunov approach is employed to indicate that the signals are convergent. Ultimately, the experimental data support the theoretical results.

Knowledge-Embedded Mutual Guidance for Visual Reasoning.

Visual reasoning between visual images and natural language is a long-standing challenge in computer vision. Most of the methods aim to look for answers to questions only on the basis of the analysis of the offered questions and images. Other approaches treat knowledge graphs as flattened tables to search for the answer. However, there are two major problems with these works: 1) the model disregards the fact that the world we surrounding us interlinks our hearing and speaking of natural language and 2) the model largely ignores the structure of the acrlong KG. To overcome these challenging deficiencies, a model should jointly consider two modalities of vision and language, as well as the rich structural and logical information embedded in knowledge graphs. To this end, we propose a general joint representation learning framework for visual reasoning, namely, knowledge-embedded mutual guidance. It realizes mutual guidance not only between visual data and natural language descriptions but also between knowledge graphs and reasoning models. In addition, it exploits the knowledge derived from the reasoning model to boost knowledge graphs when applying the visual relation detection task. The experimental results demonstrate that the proposed approach performs dramatically better than state-of-the-art methods on two benchmarks for visual reasoning.

Parallel Driving with Big Models and Foundation Intelligence in Cyber-Physical-Social Spaces.

Recent years have witnessed numerous technical breakthroughs in connected and autonomous vehicles (CAVs). On the one hand, these breakthroughs have significantly advanced the development of intelligent transportation systems (ITSs); on the other hand, these new traffic participants introduce more complex and uncertain elements to ITSs from the social space. Digital twins (DTs) provide real-time, data-driven, precise modeling for constructing the digital mapping of physical-world ITSs. Meanwhile, the metaverse integrates emerging technologies such as virtual reality/mixed reality, artificial intelligence, and DTs to model and explore how to realize improved sustainability, increased efficiency, and enhanced safety. More recently, as a leading effort toward general artificial intelligence, the concept of foundation model was proposed and has achieved significant success, showing great potential to lay the cornerstone for diverse artificial intelligence applications across different domains. In this article, we explore the big models embodied foundation intelligence for parallel driving in cyber-physical-social spaces, which integrate metaverse and DTs to construct a parallel training space for CAVs, and present a comprehensive elucidation of the crucial characteristics and operational mechanisms. Beyond providing the infrastructure and foundation intelligence of big models for parallel driving, this article also discusses future trends and potential research directions, and the "6S" goals of parallel driving.

A Novel Parallel Control Method for Continuous-Time Linear Output Regulation With Disturbances.

In this article, a novel linear parallel control method is developed for output regulation problems with disturbances. The traditional feedback regulators are passive regulation methods. In order to solve this problem, the parallel controllers are presented, where the time variation of the control is constructed instead of the control value itself, to stabilize the output of the system. The main contributions of the developed method include two aspects: 1) a novel parallel regulator structure is presented, which can provide greater flexibility comparing with traditional methods and 2) the necessary and sufficient conditions for the existence of parallel regulators are analyzed. First, the structure of the linear parallel regulators is provided. Next, considering the situations that the full system information is obtained and the information of error is obtained, respectively, the properties of the parallel regulators are analyzed, and the regulator designs for the two situations are provided. Finally, numerical examples are provided to verify the correctness of the present method.

Towards Better Generalization of Deep Neural Networks via Non-Typicality Sampling Scheme.

Improving the generalization performance of deep neural networks (DNNs) trained by minibatch stochastic gradient descent (SGD) has raised lots of concerns from deep learning practitioners. The standard simple random sampling (SRS) scheme used in minibatch SGD treats all training samples equally in gradient estimation. In this article, we study a new data selection method based on the intrinsic property of the training set to help DNNs have better generalization performance. Our theoretical analysis suggests that this new sampling scheme, called the nontypicality sampling scheme, boosts the generalization performance of DNNs through biasing the solution toward wider minima, under certain assumptions. We confirm our findings experimentally and show that more variants of minibatch SGD can also benefit from the new sampling scheme. Finally, we discuss an extension of the nontypicality sampling scheme that holds promise to enhance both generalization performance and convergence speed of minibatch SGD.

MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams.

A challenge for contemporary deep neural networks in real-world problems is learning from an imbalanced data stream, where data tends to be received chunk by chunk over time, and the prior class distribution is severely imbalanced. Although many sophisticated algorithms have been derived, most of them overlook the importance of gradient information. From this perspective, the difficulty of learning from imbalanced data streams lies in the fact that the gradient estimated on an uneven class distribution is not informative enough to reflect the critical pattern of each class. To this end, we propose to assign higher weights on the training samples whose gradients are close to the gradient of corresponding typical samples, thus highlighting the important samples in minority classes and suppressing the noisy samples in majority classes. Such an idea can be combined with Mixup, which exploits the interpolation information of data to further compensate for the information of sample space that the typical samples do not provide and expand the role of the proposed re-weighting scheme. Experiments on artificially induced long-tailed CIFAR data streams and long-tailed MiniPlaces data stream show that the resulting method, termed MixGradient, boosts the generalization performance of DNNs under different imbalance ratios and achieves up to 10% accuracy improvement.

Webly Supervised Knowledge-Embedded Model for Visual Reasoning.

Visual reasoning between visual images and natural language remains a long-standing challenge in computer vision. Conventional deep supervision methods target at finding answers to the questions relying on the datasets containing only a limited amount of images with textual ground-truth descriptions. Facing learning with limited labels, it is natural to expect to constitute a larger scale dataset consisting of several million visual data annotated with texts, but this approach is extremely time-intensive and laborious. Knowledge-based works usually treat knowledge graphs (KGs) as static flattened tables for searching the answer, but fail to take advantage of the dynamic update of KGs. To overcome these deficiencies, we propose a Webly supervised knowledge-embedded model for the task of visual reasoning. On the one hand, vitalized by the overwhelming successful Webly supervised learning, we make much use readily available images from the Web with their weakly annotated texts for an effective representation. On the other hand, we design a knowledge-embedded model, including the dynamically updated interaction mechanism between semantic representation models and KGs. Experimental results on two benchmark datasets demonstrate that our proposed model significantly achieves the most outstanding performance compared with other state-of-the-art approaches for the task of visual reasoning.

Robust Optimal Parallel Tracking Control Based on Adaptive Dynamic Programming.

This article focuses on a novel robust optimal parallel tracking control method for continuous-time (CT) nonlinear systems subject to uncertainties. First, the designed virtual controller facilitates the transformation of the original nonlinear system into an affine system with an augmented state vector, which promotes the introduction of the optimal parallel tracking control problem. Then, this article generates fresh insight into counteracting the effects of uncertainty by developing a novel parallel control system that invokes the formulated virtual control law and an auxiliary variable obtained from the relationship between the solutions of the optimal control problems for the uncertain system and the nominal one. Next, critic neural networks (NNs) approximate the Hamilton-Jacobi-Bellman (HJB) equations' solution to implement the proposed robust optimal control method via adaptive dynamic programming (ADP). Finally, simulation experiments demonstrate the proposed method's remarkable effectiveness.

Event-Triggered Near-Optimal Control for Unknown Discrete-Time Nonlinear Systems Using Parallel Control.

This article uses parallel control to investigate the problem of event-triggered near-optimal control (ETNOC) for unknown discrete-time (DT) nonlinear systems. First, to achieve parallel control, an augmented nonlinear system (ANS) with an augmented performance index (API) is proposed to introduce the control input into the feedback system. The control stability relationship between the ANS and the original system is analyzed, and it is shown that, by choosing a proper API, optimal control of the ANS with the API can be seen as near-optimal control of the original system with the original performance index (OPI). Second, based on parallel control, a novel event-triggered scheme is proposed, and then a novel ETNOC method is developed using the time-triggered optimal value function of the ANS with the API. The control stability is proved, and an upper bound, which is related to the design parameter, is provided for the actual performance index in advance. Then, to implement the developed ETNOC method for unknown DT nonlinear systems, a novel online learning algorithm is developed without reconstructing unknown systems, and neural network (NN) and adaptive dynamic programming (ADP) techniques are employed in the developed algorithm. The convergence of the signals in the closed-loop system (CLS) is shown using the Lyapunov approach, and the assumption of boundedness of input dynamics is not required. Finally, two simulations justify the theoretical conjectures.

One Size Fits All: A Unified Traffic Predictor for Capturing the Essential Spatial-Temporal Dependency.

Traffic prediction is a keystone for building smart cities in the new era and has found wide applications in traffic scheduling and management, environment policy making, public safety, and so on. Instead of creating a traffic predictor for each city, this article focuses on designing a unified network model that could be directly applied for traffic prediction in any city, by learning the essential spatial-temporal dependencies, i.e., the mutual relationship between traffic and the corresponding fine-grained road network. To achieve this goal, this article proposes a joint knowledge-and data-driven mechanism that novelly divides dependencies into three kinds of correlations, i.e., road segment, intra-intersection, and inter-intersection correlation, which capture the microcosmic, middle, and macroscopic dependencies between traffic and the road network, respectively. Specifically, we first construct traffic datasets that could cover all road segments from real-world trajectory datasets, which makes it possible to model the whole road network as a graph, with the help of fine-grained road topology. Then, we propose meta road segment learner, connection-aware spatial-temporal graph convolutional network (GCN), and multiscale residual networks for capturing the microcosmic, middle, and macroscopic dependencies, respectively. Our experiments on three real-world datasets demonstrate that our proposed method could: 1) achieve better prediction accuracy compared with several approaches and 2) capture the mutual relationship between traffic and the fine-grained road network since our model trained only using data from the source city achieves good performance when it is directly applied for traffic prediction in the target city, without any fine-tuning. The codes will be made publicly available.

GLaLT: Global-Local Attention-Augmented Light Transformer for Scene Text Recognition.

Recent years have witnessed the growing popularity of connectionist temporal classification (CTC) and attention mechanism in scene text recognition (STR). CTC-based methods consume less time with few computational burdens, while they are not as effective as attention-based methods. To retain computational efficiency and effectiveness, we propose the global-local attention-augmented light Transformer (GLaLT), which adopts a Transformer-based encoder-decoder structure to orchestrate CTC and attention mechanism. The encoder integrates the self-attention module with the convolution module to augment the attention, where the self-attention module pays more attention to capturing long-term global dependencies and the convolution module focuses on local context modeling. The decoder consists of two parallel modules: one is the Transformer-decoder-based attention module and the other is the CTC module. The first one is removed in the testing phase and can guide the second one to extract robust features in the training phase. Extensive experiments on standard benchmarks demonstrate that GLaLT achieves state-of-the-art performance for both regular and irregular STR. In terms of tradeoffs, the proposed GLaLT is at or near the frontiers for maximizing speed, accuracy, and computational efficiency at the same time.

Conservative-Progressive Collaborative Learning for Semi-Supervised Semantic Segmentation.

Pseudo supervision is regarded as the core idea in semi-supervised learning for semantic segmentation, and there is always a tradeoff between utilizing only the high-quality pseudo labels and leveraging all the pseudo labels. Addressing that, we propose a novel learning approach, called Conservative-Progressive Collaborative Learning (CPCL), among which two predictive networks are trained in parallel, and the pseudo supervision is implemented based on both the agreement and disagreement of the two predictions. One network seeks common ground via intersection supervision and is supervised by the high-quality labels to ensure a more reliable supervision, while the other network reserves differences via union supervision and is supervised by all the pseudo labels to keep exploring with curiosity. Thus, the collaboration of conservative evolution and progressive exploration can be achieved. To reduce the influences of the suspicious pseudo labels, the loss is dynamic re-weighted according to the prediction confidence. Extensive experiments demonstrate that CPCL achieves state-of-the-art performance for semi-supervised semantic segmentation.

Deep Rank-Consistent Pyramid Model for Enhanced Crowd Counting.

Most conventional crowd counting methods utilize a fully-supervised learning framework to establish a mapping between scene images and crowd density maps. They usually rely on a large quantity of costly and time-intensive pixel-level annotations for training supervision. One way to mitigate the intensive labeling effort and improve counting accuracy is to leverage large amounts of unlabeled images. This is attributed to the inherent self-structural information and rank consistency within a single image, offering additional qualitative relation supervision during training. Contrary to earlier methods that utilized the rank relations at the original image level, we explore such rank-consistency relation within the latent feature spaces. This approach enables the incorporation of numerous pyramid partial orders, strengthening the model representation capability. A notable advantage is that it can also increase the utilization ratio of unlabeled samples. Specifically, we propose a Deep Rank-consistEnt pyrAmid Model (), which makes full use of rank consistency across coarse-to-fine pyramid features in latent spaces for enhanced crowd counting with massive unlabeled images. In addition, we have collected a new unlabeled crowd counting dataset, FUDAN-UCC, comprising 4000 images for training purposes. Extensive experiments on four benchmark datasets, namely UCF-QNRF, ShanghaiTech PartA and PartB, and UCF-CC-50, show the effectiveness of our method compared with previous semi-supervised methods. The codes are available at https://github.com/bridgeqiqi/DREAM.

SASAN: Shape-Adaptive Set Abstraction Network for Point-Voxel 3D Object Detection.

Point-voxel 3D object detectors have achieved impressive performance in complex traffic scenes. However, they utilize the 3D sparse convolution (spconv) layers with fixed receptive fields, such as voxel-based detectors, and inherit the fixed sphere radius from point-based methods for generating the features of keypoints, which make them weak in adaptively modeling various geometrical deformations and sizes of real objects. To tackle this issue, we propose a shape-adaptive set abstraction network (SASAN) for point-voxel 3D object detection. First, the proposal and offset generation module is adopted to learn the coordinates and confidences of 3D proposals and shape-adaptive offsets of the certain number of offset points for each voxel. Meanwhile, an extra offset supervision task is employed to guide the learning of shifting values of offset points, aiming at motivating the predicted offsets to preferably adapt to the various shapes of objects. Then, the shape-adaptive set abstraction module is proposed to extract multiscale keypoints features by grouping the neighboring offset points' features, as well as features learned from adjacent raw points and the 2-D bird-view map. Finally, the region of interest (RoI)-grid proposal refinement module is used to aggregate the keypoints features for further proposal refinement and confidence prediction. Extensive experiments on the competitive KITTI 3D detection benchmark demonstrate that the proposed SASAN gains superior performance as compared with state-of-the-art methods.

3D printing in materials manufacturing industry: A realm of Industry 4.0.

Additive manufacturing (AM), also known as 3D printing, is a new manufacturing trend showing promising progress over time in the era of Industry 4.0. So far, various research has been done for increasing the reliability and productivity of a 3D printing process. In this regard, reviewing the existing concepts and forwarding novel research directions are important. This paper reviews and summarizes the process flow, technologies, configurations, and monitoring of AM. It started with the general AM process flow, followed by the definitions and the working principles of various AM technologies and the possible AM configurations, such as traditional and robot-assisted AM. Then, defect detection, fault diagnosis, and open-loop and closed-loop control systems in AM are discussed. It is noted that introducing robots into the assisting mechanism of AM increases the reliability and productivity of the manufacturing process. Moreover, integrating machine learning and conventional control algorithms ensures a closed-loop control in AM, a promising control strategy. Lastly, the paper addresses the challenges and future trends.

Toward parallel intelligence: An interdisciplinary solution for complex systems.

The growing complexity of real-world systems necessitates interdisciplinary solutions to confront myriad challenges in modeling, analysis, management, and control. To meet these demands, the parallel systems method rooted in the artificial systems, computational experiments, and parallel execution (ACP) approach has been developed. The method cultivates a cycle termed parallel intelligence, which iteratively creates data, acquires knowledge, and refines the actual system. Over the past two decades, the parallel systems method has continuously woven advanced knowledge and technologies from various disciplines, offering versatile interdisciplinary solutions for complex systems across diverse fields. This review explores the origins and fundamental concepts of the parallel systems method, showcasing its accomplishments as a diverse array of parallel technologies and applications while also prognosticating potential challenges. We posit that this method will considerably augment sustainable development while enhancing interdisciplinary communication and cooperation.