Influence of Perceptual and Conceptual Information on Fear Generalization: A Behavioral and Event-Related Potential Study.

Learned fear can be generalized through both perceptual and conceptual information. This study investigated how perceptual and conceptual similarities influence this generalization process. Twenty-three healthy volunteers completed a fear-generalization test as brain activity was recorded in the form of event-related potentials (ERPs). Participants were exposed to a de novo fear acquisition paradigm with four categories of conditioned stimuli (CS): two conceptual cues (animals and furniture); and two perceptual cues (blue and purple shapes). Animals (C+) and purple shapes (P+) were paired with the unconditioned stimulus (US), whereas furniture (C-) and blue shapes (P-) never were. The generalized stimuli were thus blue animals (C+P+, determined danger), blue furniture (C-P+, perceptual danger), purple animals (C+P-, conceptual danger), and purple furniture (C-P-, determined safe). We found that perceptual cues elicited larger fear responses and shorter reaction times than did conceptual cues during fear acquisition. This suggests that a perceptually related pathway might evoke greater fear than a conceptually based route. During generalization, participants were more afraid of C+ exemplars than of C- exemplars. Furthermore, C+ trials elicited greater N400 amplitudes. Thus, participants appear able to use conceptually based cues to infer the value of the current stimuli. Additionally, compared with C+ exemplars, we found an enhanced late positive potential effect in response to C- exemplars, which seems to reflect a late inhibitory process and might index safety learning. These findings may offer new insights into the pathological mechanism of anxiety disorders.

Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images.

BACKGROUND: Quantizing the Breast Imaging Reporting and Data System (BI-RADS) criteria into different categories with the single ultrasound modality has always been a challenge. To achieve this, we proposed a two-stage grading system to automatically evaluate breast tumors from ultrasound images into five categories based on convolutional neural networks (CNNs). METHODS: This new developed automatic grading system was consisted of two stages, including the tumor identification and the tumor grading. The constructed network for tumor identification, denoted as ROI-CNN, can identify the region contained the tumor from the original breast ultrasound images. The following tumor categorization network, denoted as G-CNN, can generate effective features for differentiating the identified regions of interest (ROIs) into five categories: Category "3", Category "4A", Category "4B", Category "4C", and Category "5". Particularly, to promote the predictions identified by the ROI-CNN better tailor to the tumor, refinement procedure based on Level-set was leveraged as a joint between the stage and grading stage. RESULTS: We tested the proposed two-stage grading system against 2238 cases with breast tumors in ultrasound images. With the accuracy as an indicator, our automatic computerized evaluation for grading breast tumors exhibited a performance comparable to that of subjective categories determined by physicians. Experimental results show that our two-stage framework can achieve the accuracy of 0.998 on Category "3", 0.940 on Category "4A", 0.734 on Category "4B", 0.922 on Category "4C", and 0.876 on Category "5". CONCLUSION: The proposed scheme can extract effective features from the breast ultrasound images for the final classification of breast tumors by decoupling the identification features and classification features with different CNNs. Besides, the proposed scheme can extend the diagnosing of breast tumors in ultrasound images to five sub-categories according to BI-RADS rather than merely distinguishing the breast tumor malignant from benign.

Responses of Transgenic Melatonin-Enriched Goats on LPS Stimulation and the Proteogenomic Profiles of Their PBMCs.

The anti-inflammatory activity of melatonin (MT) has been well documented; however, little is known regarding endogenously occurring MT in this respect, especially for large animals. In the current study, we created a MT-enriched animal model (goats) overexpressing the MT synthetase gene Aanat. The responses of these animals to lipopolysaccharide (LPS) stimulation were systematically studied. It was found that LPS treatment exacerbated the inflammatory response in wild-type (WT) goats and increased their temperature to 40 °C. In addition, their granulocyte counts were also significantly elevated. In contrast, these symptoms were not observed in transgenic goats with LPS treatment. The rescue study with MT injection into WT goats who were treated with LPS confirmed that the protective effects in transgenic goats against LPS were attributed to a high level of endogenously produced MT. The proteomic analysis in the peripheral blood mononuclear cells (PBMCs) isolated from the transgenic animals uncovered several potential mechanisms. MT suppressed the lysosome formation as well as its function by downregulation of the lysosome-associated genes Lysosome-associated membrane protein 2 (LAMP2), Insulin-like growth factor 2 receptor (IGF2R), and Arylsulfatase B (ARSB). A high level of MT enhanced the antioxidant capacity of these cells to reduce the cell apoptosis induced by the LPS. In addition, the results also uncovered previously unknown information that showed that MT may have protective effects on some human diseases, including tuberculosis, bladder cancer, and rheumatoid arthritis, by downregulation of these disease-associated genes. All these observations warranted further investigations.

Simultaneous analysis of polygala acid, senegenin and 3,6'-disinapoylsucrose in rat plasma by liquid chromatography-tandem mass spectrometry: application to a pharmacokinetic study after oral administration.

A rapid and specific LC-MS/MS method has been developed for the simultaneous analysis of polygala acid, senegenin and 3,6'-disinapoylsucrose (DSS) in rat plasma. The method was applied to the pharmacokinetics studies of polygala acid, senegenin and DSS. The analysis was carried out on an Agilent Eclipse plus C18 reversed-phase column (100 × 4.6 mm, 3.5 µm) by gradient elution with methanol and ammonia (0.01%, v/v). The flow rate was 0.4 mL/min. All analytes including internal standard (IS) were monitored by selected reaction monitoring with an electrospray ionization source. Linear responses were obtained for polygala acid and DSS ranging from 2.5 to 2000 ng/mL, and senegenin ranging from 5 to 2000 ng/mL. The intra- and inter-day precisions (relative standard deviation) were <11.34 and 8.99%. The extraction recovery ranged from 70.89 ± 4.60 to 88.49 ± 3.26%, and that for the IS was 77.23 ± 3.68%. Stability studies showed that polygala acid, senegenin and DSS are stable during the preparation and analytical process. The validated method was successfully used to determine the concentration-time profiles of polygala acid, senegenin and DSS.

[Study on simplification of extraction kinetics model and adaptability of total flavonoids model of Scutellariae radix].

Because of irregular shapes of Chinese herbal pieces, we simplified the previously deduced general extraction kinetic model for TCMs, and integrated particle diameters of Chinese herbs that had been hard to be determined in the final parameter "a". The reduction of the direct determination of particle diameters of Chinese herbs was conducive to increase the accuracy of the model, expand the application scope of the model, and get closer to the actual production conditions. Finally, a simplified model was established, with its corresponding experimental methods and data processing methods determined. With total flavonoids in Scutellariae Radix as the determination index, we conducted a study on the adaptability of total flavonoids extracted from Scutellariae Radix with the water decoction method in the model. The results showed a good linear correlation among the natural logarithm value of the mass concentration of total flavonoids in Scutellariae Radix, the time and the changes in the natural logarithm of solvent multiple. Through calculating and fitting, efforts were made to establish the kinetic model of extracting total flavonoids from Scutellariae Radix with the water decoction method, and verify the model, with a good degree of fitting and deviation within the range of the industrial production requirements. This indicated that the model established by the method has a good adaptability.

A Generative Shape Compositional Framework to Synthesize Populations of Virtual Chimeras.

Generating virtual organ populations that capture sufficient variability while remaining plausible is essential to conduct in silico trials (ISTs) of medical devices. However, not all anatomical shapes of interest are always available for each individual in a population. The imaging examinations and modalities used can vary between subjects depending on their individualized clinical pathways. Different imaging modalities may have various fields of view and are sensitive to signals from other tissues/organs, or both. Hence, missing/partially overlapping anatomical information is often available across individuals. We introduce a generative shape model for multipart anatomical structures, learnable from sets of unpaired datasets, i.e., where each substructure in the shape assembly comes from datasets with missing or partially overlapping substructures from disjoint subjects of the same population. The proposed generative model can synthesize complete multipart shape assemblies coined virtual chimeras (VCs). We applied this framework to build VCs from databases of whole-heart shape assemblies that each contribute samples for heart substructures. Specifically, we propose a graph neural network-based generative shape compositional framework, which comprises two components, a part-aware generative shape model that captures the variability in shape observed for each structure of interest in the training population and a spatial composition network that assembles/composes the structures synthesized by the former into multipart shape assemblies (i.e., VCs). We also propose a novel self-supervised learning scheme that enables the spatial composition network to be trained with partially overlapping data and weak labels. We trained and validated our approach using shapes of cardiac structures derived from cardiac magnetic resonance (MR) images in the UK Biobank (UKBB). When trained with complete and partially overlapping data, our approach significantly outperforms a principal component analysis (PCA)-based shape model (trained with complete data) in terms of generalizability and specificity. This demonstrates the superiority of the proposed method, as the synthesized cardiac virtual populations are more plausible and capture a greater degree of shape variability than those generated by the PCA-based shape model.

Reduced generalization of reward among individuals with subthreshold depression: Behavioral and EEG evidence.

Altered stimulus generalization has been well-documented in anxiety disorders; however, there is a paucity of research investigating this phenomenon in the context of depression. Depression is characterized by impaired reward processing and heightened attention to negative stimuli. It is hypothesized that individuals with depression exhibit reduced generalization of reward stimuli and enhanced generalization of loss stimuli. Nevertheless, no study has examined this process and its underlying neural mechanisms. In the present study, we recruited 25 participants with subthreshold depression (SD group) and 24 age-matched healthy controls (HC group). Participants completed an acquisition task, in which they learned to associate three distinct pure tones (conditioned stimuli, CSs) with a reward, a loss, or no outcome. Subsequently, a generalization session was conducted, during which similar tones (generalization stimuli, GSs) were presented, and participants were required to classify them as a reward tone, a loss tone, or neither. The results revealed that the SD group exhibited reduced generalization errors in the early phase of generalization, suggesting a diminished ability to generalize reward-related stimuli. The event-related potential (ERP) results indicated that the SD group exhibited decreased generalization of positive valence to reward-related GSs and heightened generalization of negative valence to loss-related GSs, as reflected by the N1 and P2 components. However, the late positive potential (LPP) was not modulated by depression in reward generalization or loss generalization. These findings suggested that individuals with subthreshold depression may have a blunted or reduced ability to generalize reward stimuli, shedding light on potential treatment strategies targeting this particular process.

The relationship between autistic traits and empathy in adolescents: An ERP study.

Based on the mind-blindness hypothesis, a large number of studies have shown that individuals with autism-spectrum disorder (ASD) and autistic traits have empathy deficits. However, the recent double empathy theory contradicts the mind-blindness hypothesis and suggests that individuals with ASD and autistic traits do not necessarily lack empathy. Thus, the presence of empathy deficits in individuals with ASD and autistic traits is still controversial. We recruited 56 adolescents (28 high autistic traits, 28 low autistic traits, 14-17 years old) in this study to explore the relationship between empathy and autistic traits. The study participants were required to undertake the pain empathy task, during which the electroencephalograph (EEG) activities were recorded. Our results show that empathy was negatively associated with autistic traits at the questionnaire, behavioral, and EEG levels. Our results also suggested that empathy deficits in adolescents with autistic traits may be manifested mainly in the late stages of cognitive control processing.

Impact of observational and direct learning on fear conditioning generalization in humans.

Humans gain knowledge about threats not only from their own experiences but also from observing others' behavior. A neutral stimulus is associated with a threat stimulus for several times and the neutral stimulus will evoke fear responses, which is known as fear conditioning. When encountering a new event that is similar to one previously associated with a threat, one may feel afraid and produce fear responses. This is called fear generalization. Previous studies have mostly focused on fear conditioning and generalization based on direct learning, but few have explored how observational fear learning affects fear conditioning and generalization. To the best of our knowledge, no previous study has focused on the neural correlations of fear conditioning and generalization based on observational learning. In the present study, 58 participants performed a differential conditioning paradigm in which they learned the associations between neutral cues (i.e., geometric figures) and threat stimuli (i.e., electric shock). The learning occurred on their own (i.e., direct learning) and by observing other participant's responses (i.e., observational learning); the study used a within-subjects design. After each learning condition, a fear generalization paradigm was conducted by each participant independently while their behavioral responses (i.e., expectation of a shock) and electroencephalography (EEG) recordings or responses were recorded. The shock expectancy ratings showed that observational learning, compared to direct learning, reduced the differentiation between the conditioned threatening stimuli and safety stimuli and the increased shock expectancy to the generalization stimuli. The EEG indicated that in fear learning, threatening conditioned stimuli in observational and direct learning increased early discrimination (P1) and late motivated attention (late positive potential [LPP]), compared with safety conditioned stimuli. In fear generalization, early discrimination, late motivated attention, and orienting attention (alpha-event-related desynchronization [alpha-ERD]) to generalization stimuli were reduced in the observational learning condition. These findings suggest that compared to direct learning, observational learning reduces differential fear learning and increases the generalization of fear, and this might be associated with reduced discrimination and attentional function related to generalization stimuli.

RecON: Online learning for sensorless freehand 3D ultrasound reconstruction.

Sensorless freehand 3D ultrasound (US) reconstruction based on deep networks shows promising advantages, such as large field of view, relatively high resolution, low cost, and ease of use. However, existing methods mainly consider vanilla scan strategies with limited inter-frame variations. These methods thus are degraded on complex but routine scan sequences in clinics. In this context, we propose a novel online learning framework for freehand 3D US reconstruction under complex scan strategies with diverse scanning velocities and poses. First, we devise a motion-weighted training loss in training phase to regularize the scan variation frame-by-frame and better mitigate the negative effects of uneven inter-frame velocity. Second, we effectively drive online learning with local-to-global pseudo supervisions. It mines both the frame-level contextual consistency and the path-level similarity constraint to improve the inter-frame transformation estimation. We explore a global adversarial shape before transferring the latent anatomical prior as supervision. Third, we build a feasible differentiable reconstruction approximation to enable the end-to-end optimization of our online learning. Experimental results illustrate that our freehand 3D US reconstruction framework outperformed current methods on two large, simulated datasets and one real dataset. In addition, we applied the proposed framework to clinical scan videos to further validate its effectiveness and generalizability.

Test-time bi-directional adaptation between image and model for robust segmentation.

BACKGROUND AND OBJECTIVE: Deep learning models often suffer from performance degradations when deployed in real clinical environments due to appearance shifts between training and testing images. Most extant methods use training-time adaptation, which almost require target domain samples in the training phase. However, these solutions are limited by the training process and cannot guarantee the accurate prediction of test samples with unforeseen appearance shifts. Further, it is impractical to collect target samples in advance. In this paper, we provide a general method of making existing segmentation models robust to samples with unknown appearance shifts when deployed in daily clinical practice. METHODS: Our proposed test-time bi-directional adaptation framework combines two complementary strategies. First, our image-to-model (I2M) adaptation strategy adapts appearance-agnostic test images to the learned segmentation model using a novel plug-and-play statistical alignment style transfer module during testing. Second, our model-to-image (M2I) adaptation strategy adapts the learned segmentation model to test images with unknown appearance shifts. This strategy applies an augmented self-supervised learning module to fine-tune the learned model with proxy labels that it generates. This innovative procedure can be adaptively constrained using our novel proxy consistency criterion. This complementary I2M and M2I framework demonstrably achieves robust segmentation against unknown appearance shifts using existing deep-learning models. RESULTS: Extensive experiments on 10 datasets containing fetal ultrasound, chest X-ray, and retinal fundus images demonstrate that our proposed method achieves promising robustness and efficiency in segmenting images with unknown appearance shifts. CONCLUSIONS: To address the appearance shift problem in clinically acquired medical images, we provide robust segmentation by using two complementary strategies. Our solution is general and amenable for deployment in clinical settings.

Medical Image Segmentation Using Transformer Networks.

Deep learning models represent the state of the art in medical image segmentation. Most of these models are fully-convolutional networks (FCNs), namely each layer processes the output of the preceding layer with convolution operations. The convolution operation enjoys several important properties such as sparse interactions, parameter sharing, and translation equivariance. Because of these properties, FCNs possess a strong and useful inductive bias for image modeling and analysis. However, they also have certain important shortcomings, such as performing a fixed and pre-determined operation on a test image regardless of its content and difficulty in modeling long-range interactions. In this work we show that a different deep neural network architecture, based entirely on self-attention between neighboring image patches and without any convolution operations, can achieve more accurate segmentations than FCNs. Our proposed model is based directly on the transformer network architecture. Given a 3D image block, our network divides it into non-overlapping 3D patches and computes a 1D embedding for each patch. The network predicts the segmentation map for the block based on the self-attention between these patch embeddings. Furthermore, in order to address the common problem of scarcity of labeled medical images, we propose methods for pre-training this model on large corpora of unlabeled images. Our experiments show that the proposed model can achieve segmentation accuracies that are better than several state of the art FCN architectures on two datasets. Our proposed network can be trained using only tens of labeled images. Moreover, with the proposed pre-training strategies, our network outperforms FCNs when labeled training data is small.

Attachment voices promote safety learning in humans: A critical role for P2.

Humans have evolved to seek the proximity of attachment figures during times of threat in order to obtain a sense of safety. In this context, we examined whether or not the voice of an intimate partner (termed "attachment voice") could reduce fear-learning of conditioned stimuli (CS+) and enhance learning of safety signals (CS-). Although the ability to learn safety signals is vital for human survival, few studies have explored how attachment voices affect safety learning. To test our hypothesis, we recruited thirty-five young couples and performed a classic Pavlovian conditioning experiment, recording behavioral and electroencephalographic (EEG) data. The results showed that compared with a stranger's voice, the voices of the partners reduced expectancy of the unconditioned stimulus (a shock) during fear-conditioning, as well as the magnitude of P2 event-related potentials within the EEG responses, provided the voices were safety signals. Additionally, behavioral and EEG responses to the CS+ and CS- differed more when the participants heard their partner's voice than when they heard the stranger's voice. Thus, attachment voices, even as pure vowel sounds without any semantic information, enhanced acquisition of conditioned safety (CS-). These findings may provide implications for investigating other new techniques to improve clinical treatments for fear- and anxiety-related disorders and for psychological interventions against the mental health effects of the public health emergency.

Boundary-rendering network for breast lesion segmentation in ultrasound images.

Breast Ultrasound (BUS) has proven to be an effective tool for the early detection of cancer in the breast. A lesion segmentation provides identification of the boundary, shape, and location of the target, and serves as a crucial step toward accurate diagnosis. Despite recent efforts in developing machine learning algorithms to automate this process, problems remain due to the blurry or occluded edges and highly irregular nodule shapes. Existing methods often produce over-smooth or inaccurate results, failing the need of identifying detailed boundary structures which are of clinical interest. To overcome these challenges, we propose a novel boundary-rendering framework that explicitly highlights the importance of boundary for automated nodule segmentation in BUS images. It utilizes a boundary selection module to automatically focuses on the ambiguous boundary region and a graph convolutional-based boundary rendering module to exploit global contour information. Furthermore, the proposed framework embeds nodule classification via semantic segmentation and encourages co-learning across tasks. Validation experiments were performed on different BUS datasets to verify the robustness of the proposed method. Results show that the proposed method outperforms states-of-art segmentation approaches (Dice=0.854, IOU=0.919, HD=17.8) in nodule delineation, as well as obtains a higher classification accuracy than classical classification models.

Intranasal oxytocin decreases fear generalization in males, but does not modulate discrimination threshold.

BACKGROUND: A previously acquired fear response often spreads to perceptually or conceptually close stimuli or contexts. This process, known as fear generalization, facilitates the avoidance of danger, and dysregulations in this process play an important role in anxiety disorders. Oxytocin (OT) has been shown to modulate fear learning, yet effects on fear generalization remain unknown. METHODS: We employed a randomized, placebo-controlled, double-blind, between-subject design during which healthy male participants received either intranasal OT or placebo (PLC) following fear acquisition and before fear generalization with concomitant acquisition of skin conductance responses (SCRs). Twenty-four to 72 h before the fear learning and immediately after the fear generalization task, participants additionally complete a discrimination threshold task. RESULTS: Relative to PLC, OT significantly reduced perceived risk and SCRs towards the CS+ and GS1 (the generalization stimulus that is most similar to CS+) during fear generalization, whereas the discrimination threshold was not affected. CONCLUSIONS: Together, the results suggest that OT can attenuate fear generalization in the absence of effects on discrimination threshold. This study provides the first evidence for effects of OT on fear generalization in humans and suggests that OT may have therapeutic potential in anxiety disorders characterized by dysregulated fear generalization.

Generalization gradients for fear and disgust in human associative learning.

Previous research indicates that excessive fear is a critical feature in anxiety disorders; however, recent studies suggest that disgust may also contribute to the etiology and maintenance of some anxiety disorders. It remains unclear if differences exist between these two threat-related emotions in conditioning and generalization. Evaluating different patterns of fear and disgust learning would facilitate a deeper understanding of how anxiety disorders develop. In this study, 32 college students completed threat conditioning tasks, including conditioned stimuli paired with frightening or disgusting images. Fear and disgust were divided into two randomly ordered blocks to examine differences by recording subjective US expectancy ratings and eye movements in the conditioning and generalization process. During conditioning, differing US expectancy ratings (fear vs. disgust) were found only on CS-, which may demonstrated that fear is associated with inferior discrimination learning. During the generalization test, participants exhibited greater US expectancy ratings to fear-related GS1 (generalized stimulus) and GS2 relative to disgust GS1 and GS2. Fear led to longer reaction times than disgust in both phases, and the pupil size and fixation duration for fear stimuli were larger than for disgust stimuli, suggesting that disgust generalization has a steeper gradient than fear generalization. These findings provide preliminary evidence for differences between fear- and disgust-related stimuli in conditioning and generalization, and suggest insights into treatment for anxiety and other fear- or disgust-related disorders.

Irrelevant task suppresses the N170 of automatic attention allocation to fearful faces.

Recent researches have provided evidence that stimulus-driven attentional bias for threats can be modulated by top-down goals. However, it is highlight essential to indicate whether and to what extent the top-down goals can affect the early stage of attention processing and its early neural mechanism. In this study, we collected electroencephalographic data from 28 healthy volunteers with a modified spatial cueing task. The results revealed that in the irrelevant task, there was no significant difference between the reaction time (RT) of the fearful and neutral faces. In the relevant task, we found that RT of fearful faces was faster than that of neutral faces in the valid cue condition, whereas the RT of fearful faces was slower than that of neutral faces in the invalid cue condition. The N170 component in our study showed a similar result compared with RT. Specifically, we noted that in the relevant task, fearful faces in the cue position of the target evoked a larger N170 amplitude than neutral faces, whereas this effect was suppressed in the irrelevant task. These results suggest that the irrelevant task may inhibit the early attention allocation to the fearful faces. Furthermore, the top-down goals can modulate the early attentional bias for threatening facial expressions.

Agent With Warm Start and Adaptive Dynamic Termination for Plane Localization in 3D Ultrasound.

Accurate standard plane (SP) localization is the fundamental step for prenatal ultrasound (US) diagnosis. Typically, dozens of US SPs are collected to determine the clinical diagnosis. 2D US has to perform scanning for each SP, which is time-consuming and operator-dependent. While 3D US containing multiple SPs in one shot has the inherent advantages of less user-dependency and more efficiency. Automatically locating SP in 3D US is very challenging due to the huge search space and large fetal posture variations. Our previous study proposed a deep reinforcement learning (RL) framework with an alignment module and active termination to localize SPs in 3D US automatically. However, termination of agent search in RL is important and affects the practical deployment. In this study, we enhance our previous RL framework with a newly designed adaptive dynamic termination to enable an early stop for the agent searching, saving at most 67% inference time, thus boosting the accuracy and efficiency of the RL framework at the same time. Besides, we validate the effectiveness and generalizability of our algorithm extensively on our in-house multi-organ datasets containing 433 fetal brain volumes, 519 fetal abdomen volumes, and 683 uterus volumes. Our approach achieves localization error of 2.52mm/10.26° , 2.48mm/10.39° , 2.02mm/10.48° , 2.00mm/14.57° , 2.61mm/9.71° , 3.09mm/9.58° , 1.49mm/7.54° for the transcerebellar, transventricular, transthalamic planes in fetal brain, abdominal plane in fetal abdomen, and mid-sagittal, transverse and coronal planes in uterus, respectively. Experimental results show that our method is general and has the potential to improve the efficiency and standardization of US scanning.

Searching collaborative agents for multi-plane localization in 3D ultrasound.

3D ultrasound (US) has become prevalent due to its rich spatial and diagnostic information not contained in 2D US. Moreover, 3D US can contain multiple standard planes (SPs) in one shot. Thus, automatically localizing SPs in 3D US has the potential to improve user-independence and scanning-efficiency. However, manual SP localization in 3D US is challenging because of the low image quality, huge search space and large anatomical variability. In this work, we propose a novel multi-agent reinforcement learning (MARL) framework to simultaneously localize multiple SPs in 3D US. Our contribution is four-fold. First, our proposed method is general and it can accurately localize multiple SPs in different challenging US datasets. Second, we equip the MARL system with a recurrent neural network (RNN) based collaborative module, which can strengthen the communication among agents and learn the spatial relationship among planes effectively. Third, we explore to adopt the neural architecture search (NAS) to automatically design the network architecture of both the agents and the collaborative module. Last, we believe we are the first to realize automatic SP localization in pelvic US volumes, and note that our approach can handle both normal and abnormal uterus cases. Extensively validated on two challenging datasets of the uterus and fetal brain, our proposed method achieves the average localization accuracy of 7.03∘/1.59mm and 9.75∘/1.19mm. Experimental results show that our light-weight MARL model has higher accuracy than state-of-the-art methods.

AW3M: An auto-weighting and recovery framework for breast cancer diagnosis using multi-modal ultrasound.

Recently, more clinicians have realized the diagnostic value of multi-modal ultrasound in breast cancer identification and began to incorporate Doppler imaging and Elastography in the routine examination. However, accurately recognizing patterns of malignancy in different types of sonography requires expertise. Furthermore, an accurate and robust diagnosis requires proper weights of multi-modal information as well as the ability to process missing data in practice. These two aspects are often overlooked by existing computer-aided diagnosis (CAD) approaches. To overcome these challenges, we propose a novel framework (called AW3M) that utilizes four types of sonography (i.e. B-mode, Doppler, Shear-wave Elastography, and Strain Elastography) jointly to assist breast cancer diagnosis. It can extract both modality-specific and modality-invariant features using a multi-stream CNN model equipped with self-supervised consistency loss. Instead of assigning the weights of different streams empirically, AW3M automatically learns the optimal weights using reinforcement learning techniques. Furthermore, we design a light-weight recovery block that can be inserted to a trained model to handle different modality-missing scenarios. Experimental results on a large multi-modal dataset demonstrate that our method can achieve promising performance compared with state-of-the-art methods. The AW3M framework is also tested on another independent B-mode dataset to prove its efficacy in general settings. Results show that the proposed recovery block can learn from the joint distribution of multi-modal features to further boost the classification accuracy given single modality input during the test.