Efficient Masked Autoencoders With Self-Consistency.

Inspired by the masked language modeling (MLM) in natural language processing tasks, the masked image modeling (MIM) has been recognized as a strong self-supervised pre-training method in computer vision. However, the high random mask ratio of MIM results in two serious problems: 1) the inadequate data utilization of images within each iteration brings prolonged pre-training, and 2) the high inconsistency of predictions results in unreliable generations, i.e., the prediction of the identical patch may be inconsistent in different mask rounds, leading to divergent semantics in the ultimately generated outcomes. To tackle these problems, we propose the efficient masked autoencoders with self-consistency (EMAE) to improve the pre-training efficiency and increase the consistency of MIM. In particular, we present a parallel mask strategy that divides the image into K non-overlapping parts, each of which is generated by a random mask with the same mask ratio. Then the MIM task is conducted parallelly on all parts in an iteration and the model minimizes the loss between the predictions and the masked patches. Besides, we design the self-consistency learning to further maintain the consistency of predictions of overlapping masked patches among parts. Overall, our method is able to exploit the data more efficiently and obtains reliable representations. Experiments on ImageNet show that EMAE achieves the best performance on ViT-Large with only 13% of MAE pre-training time using NVIDIA A100 GPUs. After pre-training on diverse datasets, EMAE consistently obtains state-of-the-art transfer ability on a variety of downstream tasks, such as image classification, object detection, and semantic segmentation.

Artificial intelligence for automatic surgical phase recognition of laparoscopic gastrectomy in gastric cancer.

PURPOSE: This study aimed to classify laparoscopic gastric cancer phases. We also aimed to develop a transformer-based artificial intelligence (AI) model for automatic surgical phase recognition and to evaluate the model's performance using laparoscopic gastric cancer surgical videos. METHODS: One hundred patients who underwent laparoscopic surgery for gastric cancer were included in this study. All surgical videos were labeled and classified into eight phases (P0. Preparation. P1. Separate the greater gastric curvature. P2. Separate the distal stomach. P3. Separate lesser gastric curvature. P4. Dissect the superior margin of the pancreas. P5. Separation of the proximal stomach. P6. Digestive tract reconstruction. P7. End of operation). This study proposed an AI phase recognition model consisting of a convolutional neural network-based visual feature extractor and temporal relational transformer. RESULTS: A visual and temporal relationship network was proposed to automatically perform accurate surgical phase prediction. The average time for all surgical videos in the video set was 9114 ± 2571 s. The longest phase was at P1 (3388 s). The final research accuracy, F1, recall, and precision were 90.128, 87.04, 87.04, and 87.32%, respectively. The phase with the highest recognition accuracy was P1, and that with the lowest accuracy was P2. CONCLUSION: An AI model based on neural and transformer networks was developed in this study. This model can identify the phases of laparoscopic surgery for gastric cancer accurately. AI can be used as an analytical tool for gastric cancer surgical videos.

Automatic Recognition of Concealed Fish Bones under Laryngoscopy: A Practical AI Model Based on YOLO-V5.

BACKGROUND: Fish bone impaction is one of the most common problems encountered in otolaryngology emergencies. Due to their small and transparent nature, as well as the complexity of pharyngeal anatomy, identifying fish bones efficiently under laryngoscopy requires substantial clinical experience. This study aims to create an AI model to assist clinicians in detecting pharyngeal fish bones more efficiently under laryngoscopy. METHODS: Totally 3133 laryngoscopic images related to fish bones were collected for model training and validation. The images in the training dataset were trained using the YOLO-V5 algorithm model. After training, the model was validated and its performance was evaluated using a test dataset. The model's predictions were compared to those of human experts. Seven laryngoscopic videos related to fish bone were used to validate real-time target detection by the model. RESULTS: The model trained in YOLO-V5 demonstrated good generalization and performance, with an average precision of 0.857 when the intersection over union (IOU) threshold was set to 0.5. The precision, recall rate, and F1 scores of the model are 0.909, 0.818, and 0.87, respectively. The overall accuracy of the model in the validation set was 0.821, comparable to that of ENT specialists. The model processed each image in 0.012 s, significantly faster than human processing (p < 0.001). Furthermore, the model exhibited outstanding performance in video recognition. CONCLUSION: Our AI model based on YOLO-V5 effectively identifies and localizes fish bone foreign bodies in static laryngoscopic images and dynamic videos. It shows great potential for clinical application. LEVEL OF EVIDENCE: 3 Laryngoscope, 134:2162-2169, 2024.

AAformer: Auto-Aligned Transformer for Person Re-Identification.

In person re-identification (re-ID), extracting part-level features from person images has been verified to be crucial to offer fine-grained information. Most of the existing CNN-based methods only locate the human parts coarsely, or rely on pretrained human parsing models and fail in locating the identifiable nonhuman parts (e.g., knapsack). In this article, we introduce an alignment scheme in transformer architecture for the first time and propose the auto-aligned transformer (AAformer) to automatically locate both the human parts and nonhuman ones at patch level. We introduce the "Part tokens (PARTs)", which are learnable vectors, to extract part features in the transformer. A PART only interacts with a local subset of patches in self-attention and learns to be the part representation. To adaptively group the image patches into different subsets, we design the auto-alignment. Auto-alignment employs a fast variant of optimal transport (OT) algorithm to online cluster the patch embeddings into several groups with the PARTs as their prototypes. AAformer integrates the part alignment into the self-attention and the output PARTs can be directly used as part features for retrieval. Extensive experiments validate the effectiveness of PARTs and the superiority of AAformer over various state-of-the-art methods.

Learning Semantics-Consistent Stripes With Self-Refinement for Person Re-Identification.

Aligning human parts automatically is one of the most challenging problems for person re-identification (re-ID). Recently, the stripe-based methods, which equally partition the person images into the fixed stripes for aligned representation learning, have achieved great success. However, the stripes with fixed height and position cannot well handle the misalignment problems caused by inaccurate detection and occlusion and may introduce much background noise. In this article, we aim at learning adaptive stripes with foreground refinement to achieve pixel-level part alignment by only using person identity labels for person re-ID and make two contributions. 1) A semantics-consistent stripe learning method (SCS). Given an image, SCS partitions it into adaptive horizontal stripes and each stripe is corresponding to a specific semantic part. Specifically, SCS iterates between two processes: i) clustering the rows to human parts or background to generate the pseudo-part labels of rows and ii) learning a row classifier to partition a person image, which is supervised by the latest pseudo-labels. This iterative scheme guarantees the accuracy of the learned image partition. 2) A self-refinement method (SCS+) to remove the background noise in stripes. We employ the above row classifier to generate the probabilities of pixels belonging to human parts (foreground) or background, which is called the class activation map (CAM). Only the most confident areas from the CAM are assigned with foreground/background labels to guide the human part refinement. Finally, by intersecting the semantics-consistent stripes with the foreground areas, SCS+ locates the human parts at pixel-level, obtaining a more robust part-aligned representation. Extensive experiments validate that SCS+ sets the new state-of-the-art performance on three widely used datasets including Market-1501, DukeMTMC-reID, and CUHK03-NP.

Cracking enabled unclonability in colloidal crystal patterns authenticated with computer vision.

Colloidal crystals with iridescent structural coloration have appealing applications in the fields of sensors, displays, anti-counterfeiting, etc. A serious issue accompanying the facile chemical self-assembly approach to colloidal crystals is the formation of uncontrolled and irregular cracks. In contrast to the previous efforts to avoid cracking, the unfavorable and random micro-cracks in colloidal crystals were utilized here as unclonable codes for tamper-proof anti-counterfeiting. The special structural and optical characteristics of the colloidal crystal patterns assembled with monodisperse poly(styrene-methyl methacrylate-acrylic acid) core-shell nanospheres enabled multi-anti-counterfeiting modes, including angle-dependent structural colors and polarization anisotropy, besides the physically unclonable functions (PUFs) of random micro-cracks. Moreover, by using the random cracks in the colloidal crystals as templates to guide fluorescent silica nanoparticle deposition, an fluorescent anti-counterfeiting mode with PUFs was introduced. To validate the PUFs of the fluorescent micro-cracks in the colloidal crystals, a novel edge-sensitive template matching approach based on a computer vision algorithm with an accuracy of ∼100% was developed, enabling ultimate security immune to forgery. The computer-vision verifiable physically unclonable colloidal crystals with multi-anti-counterfeiting modes are superior to conventional photonic crystal anti-counterfeiting materials that rely on angle-dependent or tunable structural colors, and the conventional PUF labels in the aspect of decorative functions, which will open a new avenue for advanced security materials with multi-functionality.

Grammar-Induced Wavelet Network for Human Parsing.

Most existing methods of human parsing still face a challenge: how to extract the accurate foreground from similar or cluttered scenes effectively. In this paper, we propose a Grammar-induced Wavelet Network (GWNet), to deal with the challenge. GWNet mainly consists of two modules, including a blended grammar-induced module and a wavelet prediction module. We design the blended grammar-induced module to exploit the relationship of different human parts and the inherent hierarchical structure of a human body by means of grammar rules in both cascaded and paralleled manner. In this way, conspicuous parts, which are easily distinguished from the background, can amend the segmentation of inconspicuous ones, improving the foreground extraction. We also design a Part-aware Convolutional Recurrent Neural Network (PCRNN) to pass messages which are generated by grammar rules. To further improve the performance, we propose a wavelet prediction module to capture the basic structure and the edge details of a person by decomposing the low-frequency and high-frequency components of features. The low-frequency component can represent the smooth structures and the high-frequency components can describe the fine details. We conduct extensive experiments to evaluate GWNet on PASCAL-Person-Part, LIP, and PPSS datasets. GWNet obtains state-of-the-art performance on these human parsing datasets.

Fine-Grained Human-Centric Tracklet Segmentation with Single Frame Supervision.

In this paper, we target at the Fine-grAined human-Centric Tracklet Segmentation (FACTS) problem, where 12 human parts, e.g., face, pants, left-leg, are segmented. To reduce the heavy and tedious labeling efforts, FACTS requires only one labeled frame per video during training. The small size of human parts and the labeling scarcity makes FACTS very challenging. Considering adjacent frames of videos are continuous and human usually do not change clothes in a short time, we explicitly consider the pixel-level and frame-level context in the proposed Temporal Context segmentation Network (TCNet). On the one hand, optical flow is on-line calculated to propagate the pixel-level segmentation results to neighboring frames. On the other hand, frame-level classification likelihood vectors are also propagated to nearby frames. By fully exploiting the pixel-level and frame-level context, TCNet indirectly uses the large amount of unlabeled frames during training and produces smooth segmentation results during inference. Experimental results on four video datasets show the superiority of TCNet over the state-of-the-arts. The newly annotated datasets can be downloaded via http://liusi-group.com/projects/FACTS for the further studies.

Enhanced Bounding Box Estimation with Distribution Calibration for Visual Tracking.

Bounding box estimation by overlap maximization has improved the state of the art of visual tracking significantly, yet the improvement in robustness and accuracy is restricted by the limited reference information, i.e., the initial target. In this paper, we present DCOM, a novel bounding box estimation method for visual tracking, based on distribution calibration and overlap maximization. We assume every dimension in the modulation vector follows a Gaussian distribution, so that the mean and the variance can borrow from those of similar targets in large-scale training datasets. As such, sufficient and reliable reference information can be obtained from the calibrated distribution, leading to a more robust and accurate target estimation. Additionally, an updating strategy for the modulation vector is proposed to adapt the variation of the target object. Our method can be built on top of off-the-shelf networks without finetuning and extra parameters. It yields state-of-the-art performance on three popular benchmarks, including GOT-10k, LaSOT, and NfS while running at around 40 FPS, confirming its effectiveness and efficiency.

Rehabilitation Evaluation of Hemiplegic Patients with Anterior Circulation Cerebral Infarction Based on Cranial Magnetic Stimulation.

Cerebral infarction is a common cerebrovascular disease in clinical medicine. Cerebral infarction in the anterior circulation accounts for about 90% of cerebral infarction. Its treatment and rehabilitation has always been a research hotspot in the medical field. Functional retraining can enhance the afferent impulses received by receptors, make the plasticity development of cerebral cortex function, and improve the loss of function. Based on the patient's individual condition, exercise therapy carries out the corresponding comprehensive functional training plan, which also includes the training of patients' daily living ability, turning over, bridge exercise, trunk rotation, etc., in order to improve the motor function of patients. The other is psychotherapy, which can cause emotional fluctuations, depression, anxiety, and other negative emotions due to the occurrence of diseases. In the rehabilitation treatment, relevant personnel can conduct psychological counseling for patients through timely and effective communication, so as to better establish patients' confidence in rehabilitation and improve the effect of rehabilitation treatment. The third is acupuncture treatment. Acupuncture is a traditional rehabilitation treatment in China. The rehabilitation effect of stroke has been proved by a large number of clinical practice. Acupuncture at Hegu, Quchi, Zusanli, and Taichong points can dredge channels and improve blood circulation. This paper mainly studies and analyzes the effect of behavior rehabilitation of hemiplegic patients with cerebral anterior circulation infarction treated by cranial magnetic stimulation. The rehabilitation treatment status of hemiplegic patients with anterior circulation cerebral infarction in a hospital was selected, and 100 cases were studied. Among them, 50 cases were treated with conventional rehabilitation therapy, and the other 50 cases were treated with cranial magnetic stimulation. The motor function, activities of daily living, and language expression ability of the two groups were compared for statistical analysis. After transcranial magnetic stimulation treatment, the abilities of the study group were better than those of the control group, P < 0.05, with statistical significance. Based on the reliable experimental data, we can draw a conclusion that the treatment of cranial magnetic stimulation has a significant effect on the rehabilitation of hemiplegic patients with cerebral anterior circulation infarction, which is higher than the conventional treatment and rehabilitation methods, and can be popularized in clinical application.

Semi-Supervised Scene Text Recognition.

Scene text recognition has been widely researched with supervised approaches. Most existing algorithms require a large amount of labeled data and some methods even require character-level or pixel-wise supervision information. However, labeled data is expensive, unlabeled data is relatively easy to collect, especially for many languages with fewer resources. In this paper, we propose a novel semi-supervised method for scene text recognition. Specifically, we design two global metrics, i.e., edit reward and embedding reward, to evaluate the quality of generated string and adopt reinforcement learning techniques to directly optimize these rewards. The edit reward measures the distance between the ground truth label and the generated string. Besides, the image feature and string feature are embedded into a common space and the embedding reward is defined by the similarity between the input image and generated string. It is natural that the generated string should be the nearest with the image it is generated from. Therefore, the embedding reward can be obtained without any ground truth information. In this way, we can effectively exploit a large number of unlabeled images to improve the recognition performance without any additional laborious annotations. Extensive experimental evaluations on the five challenging benchmarks, the Street View Text, IIIT5K, and ICDAR datasets demonstrate the effectiveness of the proposed approach, and our method significantly reduces annotation effort while maintaining competitive recognition performance.

Siamese Regression Tracking With Reinforced Template Updating.

Siamese networks are prevalent in visual tracking because of the efficient localization. The networks take both a search patch and a target template as inputs where the target template is usually from the initial frame. Meanwhile, Siamese trackers do not update network parameters online for real-time efficiency. The fixed target template and CNN parameters make Siamese trackers not effective to capture target appearance variations. In this paper, we propose a template updating method via reinforcement learning for Siamese regression trackers. We collect a series of templates and learn to maintain them based on an actor-critic framework. Among this framework, the actor network that is trained by deep reinforcement learning effectively updates the templates based on the tracking result on each frame. Besides the target template, we update the Siamese regression tracker online to adapt to target appearance variations. The experimental results on the standard benchmarks show the effectiveness of both template and network updating. The proposed tracker SiamRTU performs favorably against state-of-the-art approaches.

Antidecay LSTM for Siamese Tracking With Adversarial Learning.

Visual tracking is one of the fundamental tasks in computer vision with many challenges, and it is mainly due to the changes in the target's appearance in temporal and spatial domains. Recently, numerous trackers model the appearance of the targets in the spatial domain well by utilizing deep convolutional features. However, most of these CNN-based trackers only take the appearance variations between two consecutive frames in a video sequence into consideration. Besides, some trackers model the appearance of the targets in the long term by applying RNN, but the decay of the target's features degrades the tracking performance. In this article, we propose the antidecay long short-term memory (AD-LSTM) for the Siamese tracking. Especially, we extend the architecture of the standard LSTM in two aspects for the visual tracking task. First, we replace all of the fully connected layers with convolutional layers to extract the features with spatial structure. Second, we improve the architecture of the cell unit. In this way, the information of the target appearance can flow through the AD-LSTM without decay as long as possible in the temporal domain. Meanwhile, since there is no ground truth for the feature maps generated by the AD-LSTM, we propose an adversarial learning algorithm to optimize the AD-LSTM. With the help of adversarial learning, the Siamese network can generate the response maps more accurately, and the AD-LSTM can generate the feature maps of the target more robustly. The experimental results show that our tracker performs favorably against the state-of-the-art trackers on six challenging benchmarks: OTB-100, TC-128, VOT2016, VOT2017, GOT-10k, and TrackingNet.

Nanoparticle-Based Drug Delivery System: A Patient-Friendly Chemotherapy for Oncology.

Chemotherapy is widely used to treat cancer. The toxic effect of conventional chemotherapeutic drugs on healthy cells leads to serious toxic and side effects of conventional chemotherapy. The application of nanotechnology in tumor chemotherapy can increase the specificity of anticancer agents, increase the killing effect of tumors, and reduce toxic and side effects. Currently, a variety of formulations based on nanoparticles (NPs) for delivering chemotherapeutic drugs have been put into clinical use, and several others are in the stage of development or clinical trials. In this review, after briefly introducing current cancer chemotherapeutic methods and their limitations, we describe the clinical applications and advantages and disadvantages of several different types of NPs-based chemotherapeutic agents. We have summarized a lot of information in tables and figures related to the delivery of chemotherapeutic drugs based on NPs and the design of NPs with active targeting capabilities.

An end-to-end exemplar association for unsupervised person Re-identification.

Tracklet association methods learn the cross camera retrieval ability though associating underlying cross camera positive samples, which have proven to be successful in unsupervised person re-identification task. However, most of them use poor-efficiency association strategies which costs long training hours but gains the low performance. To solve this, we propose an effective end-to-end exemplar associations (EEA) framework in this work. EEA mainly adapts three strategies to improve efficiency: (1) end-to-end exemplar-based training, (2) exemplar association and (3) dynamic selection threshold. The first one is to accelerate the training process, while the others aim to improve the tracklet association precision. Compared with existing tracklet associating methods, EEA obviously reduces the training cost and achieves the higher performance. Extensive experiments and ablation studies on seven RE-ID datasets demonstrate the superiority of the proposed EEA over most state-of-the-art unsupervised and domain adaptation RE-ID methods.

Neuroprotective Effect of Activated Protein C on Blood-Brain Barrier Injury During Focal Cerebral Ischemia/Reperfusion.

Although the effect of activated protein C (APC) on neuronal injury and neuroinflammatory responses has been extensively studied, the detailed mechanism underlying APC-protective effect in the blood-brain barrier (BBB) injury during ischemia is still not clear. In this study, the APC effect against neuroinflammatory responses was evaluated in the model of right middle cerebral artery occlusion in male Sprague-Dawley rats with 2 hours of ischemia and 22 hours of reperfusion. The results showed that APC can significantly improve the neurological function scoring and reduce the infarct volume and BBB permeability. Moreover, the expression of protein nuclear factor-kappa B (NF-κB), both in cytoplasm and nuclei, was reduced. The downstream of NF-κB activation, including tumor necrosis factor-α and interleukin-1ß secretion, was inhibited. In all, APC exerts a neuroprotective effect in focal cerebral ischemia-reperfusion in rats by inhibiting the activation and nuclear translocation of NF-κB. It may indicate a therapeutic approach for ischemic brain injury.

Effect of 3-Aminobenzamide on the Ultrastructure of Astrocytes and Microvessels After Focal Cerebral Ischemia in Rats.

The disruption of blood-brain barrier (BBB) is a critical event in the formation of brain edema during early phases of ischemic brain injury. Poly(ADP-ribose) polymerase (PARP) activation, which contributes to BBB damage, has been reported in ischemia-reperfusion and traumatic brain injury. Here, we investigated the effect of 3-aminobenzamide (3-AB), a PARP-1 inhibitor, on the ultrastructure of BBB. Male Sprague Dawley rats were suffered from 90 minutes of middle cerebral artery occlusion, followed by 4.5 hours or 22.5 hours of reperfusion (R). The vehicle or 3-AB (10 mg/kg) was administered intraperitoneally (ip) 60 minutes after lacking of blood. Tissue Evans Blue (EB) levels, ultrastructures of astrocytes and microvessels, and areas of perivascular edema were examined in penumbra and core, at I 1.5 hours /R 4.5 hours and I 1.5 hours /R 22.5 hours, respectively. The severity of ultrastructural changes was graded with a scoring system in each group. We showed that 3-AB treatment significantly decreased tissue EB levels and ultrastructural scores, attenuated damages in astrocytes and microvessels, and reduced areas of perivascular edema. In conclusion, PARP inhibition may provide a novel therapeutic approach to ischemic brain injury.

Exosomes: A Novel Therapeutic Agent for Cartilage and Bone Tissue Regeneration.

Despite traditionally treating autologous and allogeneic transplantation and emerging tissue engineering (TE)-based therapies, which have commonly performed in clinic for skeletal diseases, as the "gold standard" for care, undesirably low efficacy and other complications remain. Therefore, exploring new strategies with better therapeutic outcomes and lower incidences of unfavorable side effect is imperative. Recently, exosomes, secreted microvesicles of endocytic origin, have caught researcher's eyes in tissue regeneration fields, especially in cartilage and bone-related regeneration. Multiple researchers have demonstrated the crucial roles of exosomes throughout every developing stage of cartilage and bone tissue regeneration, indicating that there may be a potential therapeutic application of exosomes in future clinical use. Herein, we summarize the function of exosomes derived from the primary cells functioning in skeletal diseases and their restoration processes, therapeutic exosomes used to promote cartilage and bone repairing in recent research, and applications of exosomes within the setting of the TE matrix.