Deep learning approaches for assessing pediatric sleep apnea severity through SpO2 signals.

Pediatric Sleep Apnea-Hypopnea (SAH) presents a significant health challenge, particularly in diagnostic contexts, where conventional Polysomnography (PSG) testing, although effective, can be distressing for children. Addressing this, our research proposes a less invasive method to assess pediatric SAH severity by analyzing blood oxygen saturation (SpO2) signals. We adopted two advanced deep learning architectures, namely ResNet-based and attention-augmented hybrid CNN-BiGRU models, to process SpO2 signals in a one-dimensional (1D) format for Apnea-Hypopnea Index (AHI) estimation in pediatric subjects. Employing the CHAT dataset, which includes 844 SpO2 signals, the data was partitioned into training (60%), testing (30%), and validation (10%) sets. A predefined validation subset was randomly selected to ensure the models' robustness via a threefold cross-validation approach. Comparative analysis revealed that while the ResNet model attained an average accuracy of 72.9% across four SAH severity categories with a kappa score of 0.57, the CNN-BiGRU-Attention model demonstrated superior performance, achieving an average accuracy of 75.95% and a kappa score of 0.63. This distinction underscores our method's efficacy in both estimating AHI and categorizing SAH severity levels with notable precision. Further, to evaluate diagnostic capabilities, the models were benchmarked against common AHI thresholds (1, 5, and 10 events/hour) in each test fold, affirming their effectiveness in identifying pediatric SAH. This study marks a significant advance in the field, offering a non-invasive, child-friendly alternative for pediatric SAH diagnosis. Although challenges persist in accurately estimating AHI, particularly in severe cases, our findings represent a critical stride towards improving diagnostic processes in pediatric SAH.

ResNet diagnosis of rotor faults in oil transfer pumps.

To address rotor imbalance and misalignment in oil transfer pumps, an innovative diagnostic framework using Residual Network (ResNet) is proposed. The model incorporates advanced signal processing algorithms and strategic sensor placement to enhance diagnostic efficacy. A fault simulation test rig captured vibration signals from eight key measurement points on the pump. One-dimensional and multi-dimensional signal processing techniques generated comprehensive datasets for training and validating the model. Sensor placement optimization, focusing on the bearing seat's axial direction, inlet flange's vertical direction, and outlet flange's axial direction, increased rotor fault sensitivity. Time-frequency data processed via Short-Time Fourier Transform (STFT) achieved the highest diagnostic accuracy, surpassing 98 %. This study highlights the importance of optimal signal processing and precise sensor placement in improving the accuracy of diagnosing rotor faults in oil transfer pumps, thus enhancing the operational reliability and efficiency of energy transportation systems.

Reconstruction residual network with a fused spatial-channel attention mechanism for automatically classifying diabetic foot ulcer.

Diabetic foot ulcer (DFU) is a common chronic complication of diabetes. This complication is characterized by the formation of ulcers that are difficult to heal on the skin of the foot. Ulcers can negatively affect patients' quality of life, and improperly treated lesions can result in amputation and even death. Traditionally, the severity and type of foot ulcers are determined by doctors through visual observations and on the basis of their clinical experience; however, this subjective evaluation can lead to misjudgments. In addition, quantitative methods have been developed for classifying and scoring are therefore time-consuming and labor-intensive. In this paper, we propose a reconstruction residual network with a fused spatial-channel attention mechanism (FARRNet) for automatically classifying DFUs. The use of pseudo-labeling and Data augmentation as a pre-processing technique can overcome problems caused by data imbalance and small sample size. The developed model's attention was enhanced using a spatial channel attention (SPCA) module that incorporates spatial and channel attention mechanisms. A reconstruction mechanism was incorporated into the developed residual network to improve its feature extraction ability for achieving better classification. The performance of the proposed model was compared with that of state-of-the-art models and those in the DFUC Grand Challenge. When applied to the DFUC Grand Challenge, the proposed method outperforms other state-of-the-art schemes in terms of accuracy, as evaluated using 5-fold cross-validation and the following metrics: macro-average F1-score, AUC, Recall, and Precision. FARRNet achieved the F1-score of 60.81%, AUC of 87.37%, Recall of 61.04%, and Precision of 61.56%. Therefore, the proposed model is more suitable for use in medical diagnosis environments with embedded devices and limited computing resources. The proposed model can assist patients in initial identifications of ulcer wounds, thereby helping them to obtain timely treatment.

High performance filtering and high-sensitivity concentration retrieval of methane in photoacoustic spectroscopy utilizing deep learning residual networks.

A novel method is introduced to improve the detection performance of photoacoustic spectroscopy for trace gas detection. For effectively suppressing various types of noise, this method integrates photoacoustic spectroscopy with residual networks model which encompasses a total of 40 weighted layers. Firstly, this approach was employed to accurately retrieve methane concentrations at various levels. Secondly, the analysis of the signal-to-noise ratio (SNR) of multiple sets of photoacoustic spectroscopy signals revealed significant enhancement. The SNR was improved from 21 to 805, 52-962, 98-944, 188-933, 310-941, and 587-936 across the different concentrations, respectively, as a result of the application of the residual networks. Finally, further exploration for the measurement precision and stability of photoacoustic spectroscopy system utilizing residual networks was carried out. The measurement precision of 0.0626 ppm was obtained and the minimum detectable limit was found to be 1.47 ppb. Compared to traditional photoacoustic spectroscopy method, an approximately 46-fold improvement in detection limit and 69-fold enhancement in measurement precision were achieved, respectively. This method not only advances the measurement precision and stability of trace gas detection but also highlights the potential of deep learning algorithms in spectroscopy detection.

A Perovskite Material Screening and Performance Study Based on Asymmetric Convolutional Blocks.

This study introduces an innovative method for identifying high-efficiency perovskite materials using an asymmetric convolution block (ACB). Our approach involves preprocessing extensive data on perovskite oxide materials and developing a precise predictive model. This system is designed to accurately predict key properties such as band gap and stability, thereby eliminating the reliance on traditional feature importance filtering. It exhibited outstanding performance, achieving an accuracy of 96.8% and a recall of 0.998 in classification tasks, and a coefficient of determination (R2) value of 0.993 with a mean squared error (MSE) of 0.004 in regression tasks. Notably, DyCoO3 and YVO3 were identified as promising candidates for photovoltaic applications due to their optimal band gaps. This efficient and precise method significantly advances the development of advanced materials for solar cells, providing a robust framework for rapid material screening.

A novel framework for three-dimensional electrical impedance tomography reconstruction of maize ear via feature reconfiguration and residual networks.

Electrical impedance tomography (EIT) provides an indirect measure of the physiological state and growth of the maize ear by reconstructing the distribution of electrical impedance. However, the two-dimensional (2D) EIT within the electrode plane finds it challenging to comprehensively represent the spatial distribution of conductivity of the intact maize ear, including the husk, kernels, and cob. Therefore, an effective method for 3D conductivity reconstruction is necessary. In practical applications, fluctuations in the contact impedance of the maize ear occur, particularly with the increase in the number of grids and computational workload during the reconstruction of 3D spatial conductivity. These fluctuations may accentuate the ill-conditioning and nonlinearity of the EIT. To address these challenges, we introduce RFNetEIT, a novel computational framework specifically tailored for the absolute imaging of the three-dimensional electrical impedance of maize ear. This strategy transforms the reconstruction of 3D electrical conductivity into a regression process. Initially, a feature map is extracted from measured boundary voltage via a data reconstruction module, thereby enhancing the correlation among different dimensions. Subsequently, a nonlinear mapping model of the 3D spatial distribution of the boundary voltage and conductivity is established, utilizing the residual network. The performance of the proposed framework is assessed through numerical simulation experiments, acrylic model experiments, and maize ear experiments. Our experimental results indicate that our method yields superior reconstruction performance in terms of root-mean-square error (RMSE), correlation coefficient (CC), structural similarity index (SSIM), and inverse problem-solving time (IPST). Furthermore, the reconstruction experiments on maize ears demonstrate that the method can effectively reconstruct the 3D conductivity distribution.

Fast reconstruction of EEG signal compression sensing based on deep learning.

When traditional EEG signals are collected based on the Nyquist theorem, long-time recordings of EEG signals will produce a large amount of data. At the same time, limited bandwidth, end-to-end delay, and memory space will bring great pressure on the effective transmission of data. The birth of compressed sensing alleviates this transmission pressure. However, using an iterative compressed sensing reconstruction algorithm for EEG signal reconstruction faces complex calculation problems and slow data processing speed, limiting the application of compressed sensing in EEG signal rapid monitoring systems. As such, this paper presents a non-iterative and fast algorithm for reconstructing EEG signals using compressed sensing and deep learning techniques. This algorithm uses the improved residual network model, extracts the feature information of the EEG signal by one-dimensional dilated convolution, directly learns the nonlinear mapping relationship between the measured value and the original signal, and can quickly and accurately reconstruct the EEG signal. The method proposed in this paper has been verified by simulation on the open BCI contest dataset. Overall, it is proved that the proposed method has higher reconstruction accuracy and faster reconstruction speed than the traditional CS reconstruction algorithm and the existing deep learning reconstruction algorithm. In addition, it can realize the rapid reconstruction of EEG signals.

PlaqueNet: deep learning enabled coronary artery plaque segmentation from coronary computed tomography angiography.

Cardiovascular disease, primarily caused by atherosclerotic plaque formation, is a significant health concern. The early detection of these plaques is crucial for targeted therapies and reducing the risk of cardiovascular diseases. This study presents PlaqueNet, a solution for segmenting coronary artery plaques from coronary computed tomography angiography (CCTA) images. For feature extraction, the advanced residual net module was utilized, which integrates a deepwise residual optimization module into network branches, enhances feature extraction capabilities, avoiding information loss, and addresses gradient issues during training. To improve segmentation accuracy, a depthwise atrous spatial pyramid pooling based on bicubic efficient channel attention (DASPP-BICECA) module is introduced. The BICECA component amplifies the local feature sensitivity, whereas the DASPP component expands the network's information-gathering scope, resulting in elevated segmentation accuracy. Additionally, BINet, a module for joint network loss evaluation, is proposed. It optimizes the segmentation model without affecting the segmentation results. When combined with the DASPP-BICECA module, BINet enhances overall efficiency. The CCTA segmentation algorithm proposed in this study outperformed the other three comparative algorithms, achieving an intersection over Union of 87.37%, Dice of 93.26%, accuracy of 93.12%, mean intersection over Union of 93.68%, mean Dice of 96.63%, and mean pixel accuracy value of 96.55%.

Lightweight Knowledge Distillation-Based Transfer Learning Framework for Rolling Bearing Fault Diagnosis.

Compared to fault diagnosis across operating conditions, the differences in data distribution between devices are more pronounced and better aligned with practical application needs. However, current research on transfer learning inadequately addresses fault diagnosis issues across devices. To better balance the relationship between computational resources and diagnostic accuracy, a knowledge distillation-based lightweight transfer learning framework for rolling bearing diagnosis is proposed in this study. Specifically, a deep teacher-student model based on variable-scale residual networks is constructed to learn domain-invariant features relevant to fault classification within both the source and target domain data. Subsequently, a knowledge distillation framework incorporating a temperature factor is established to transfer fault features learned by the large teacher model in the source domain to the smaller student model, thereby reducing computational and parameter overhead. Finally, a multi-kernel domain adaptation method is employed to capture the feature probability distribution distance of fault characteristics between the source and target domains in Reproducing Kernel Hilbert Space (RKHS), and domain-invariant features are learned by minimizing the distribution distance between them. The effectiveness and applicability of the proposed method in situations of incomplete data across device types were validated through two engineering cases, spanning device models and transitioning from laboratory equipment to real-world operational devices.

Rolling bearing fault diagnosis based on Gramian angular difference field and improved channel attention model.

Fault diagnosis of rolling bearings is a critical task, and in previous research, convolutional neural networks (CNN) have been used to process vibration signals and perform fault diagnosis. However, traditional CNN models have certain limitations in terms of accuracy. To improve accuracy, we propose a method that combines the Gramian angular difference field (GADF) with residual networks (ResNet) and embeds frequency channel attention module (Fca) in the ResNet to diagnose rolling bearing fault. Firstly, we used GADF to convert the signals into RGB three-channel fault images during data preprocessing. Secondly, to further enhance the performance of the model, on the foundation of the ResNet we embedded the frequency channel attention module with discrete cosine transform (DCT) to form Fca, to effectively explores the channel information of fault images and identifies the corresponding fault characteristics. Finally, the experiment validated that the accuracy of the new model reaches 99.3% and the accuracy reaches 98.6% even under an unbalanced data set, which significantly improves the accuracy of fault diagnosis and the generalization of the model.

psoResNet: An improved PSO-based residual network search algorithm.

Neural Architecture Search (NAS) methods are widely employed to address the time-consuming and costly challenges associated with manual operation and design of deep convolutional neural networks (DCNNs). Nonetheless, prevailing methods still encounter several pressing obstacles, including limited network architecture design, excessively lengthy search periods, and insufficient utilization of the search space. In light of these concerns, this study proposes an optimization strategy for residual networks that leverages an enhanced Particle swarm optimization algorithm. Primarily, low-complexity residual architecture block is employed as the foundational unit for architecture exploration, facilitating a more diverse investigation into network architectures while minimizing parameters. Additionally, we employ a depth initialization strategy to confine the search space within a reasonable range, thereby mitigating unnecessary particle exploration. Lastly, we present a novel approach for computing particle differences and updating velocity mechanisms to enhance the exploration of updated trajectories. This method significantly contributes to the improved utilization of the search space and the augmentation of particle diversity. Moreover, we constructed a crime-dataset comprising 13 classes to assess the effectiveness of the proposed algorithm. Experimental results demonstrate that our algorithm can design lightweight networks with superior classification performance on both benchmark datasets and the crime-dataset.

Residual networks models detection of atrial septal defect from chest radiographs.

OBJECT: The purpose of this study was to explore a machine learning-based residual networks (ResNets) model to detect atrial septal defect (ASD) on chest radiographs. METHODS: This retrospective study included chest radiographs consecutively collected at our hospital from June 2017 to May 2022. Qualified chest radiographs were obtained from patients who had finished echocardiography. These chest radiographs were labeled as positive or negative for ASD based on the echocardiographic reports and were divided into training, validation, and test dataset. Six ResNets models were employed to examine and compare by using the training dataset and was tuned using the validation dataset. The area under the curve, recall, precision and F1-score were taken as the evaluation metrics for classification result in the test dataset. Visualizing regions of interest for the ResNets models using heat maps. RESULTS: This study included a total of 2105 chest radiographs of children with ASD (mean age 4.14 ± 2.73 years, 54% male), patients were randomly assigned to training, validation, and test dataset with an 8:1:1 ratio. Healthy children's images were supplemented to three datasets in a 1:1 ratio with ASD patients. Following the training, ResNet-10t and ResNet-18D have a better estimation performance, with precision, recall, accuracy, F1-score, and the area under the curve being (0.92, 0.93), (0.91, 0.91), (0.90, 0.90), (0.91, 0.91) and (0.97, 0.96), respectively. Compared to ResNet-18D, ResNet-10t was more focused on the distribution of the heat map of the interest region for most chest radiographs from ASD patients. CONCLUSION: The ResNets model is feasible for identifying ASD through children's chest radiographs. ResNet-10t stands out as the preferable estimation model, providing exceptional performance and clear interpretability.

4D-CT deformable image registration using unsupervised recursive cascaded full-resolution residual networks.

A novel recursive cascaded full-resolution residual network (RCFRR-Net) for abdominal four-dimensional computed tomography (4D-CT) image registration was proposed. The entire network was end-to-end and trained in the unsupervised approach, which meant that the deformation vector field, which presented the ground truth, was not needed during training. The network was designed by cascading three full-resolution residual subnetworks with different architectures. The training loss consisted of the image similarity loss and the deformation vector field regularization loss, which were calculated based on the final warped image and the fixed image, allowing all cascades to be trained jointly and perform the progressive registration cooperatively. Extensive network testing was conducted using diverse datasets, including an internal 4D-CT dataset, a public DIRLAB 4D-CT dataset, and a 4D cone-beam CT (4D-CBCT) dataset. Compared with the iteration-based demon method and two deep learning-based methods (VoxelMorph and recursive cascaded network), the RCFRR-Net achieved consistent and significant gains, which demonstrated that the proposed method had superior performance and generalization capability in medical image registration. The proposed RCFRR-Net was a promising tool for various clinical applications.

Efficient Skip Connections-Based Residual Network (ESRNet) for Brain Tumor Classification.

Brain tumors pose a complex and urgent challenge in medical diagnostics, requiring precise and timely classification due to their diverse characteristics and potentially life-threatening consequences. While existing deep learning (DL)-based brain tumor classification (BTC) models have shown significant progress, they encounter limitations like restricted depth, vanishing gradient issues, and difficulties in capturing intricate features. To address these challenges, this paper proposes an efficient skip connections-based residual network (ESRNet). leveraging the residual network (ResNet) with skip connections. ESRNet ensures smooth gradient flow during training, mitigating the vanishing gradient problem. Additionally, the ESRNet architecture includes multiple stages with increasing numbers of residual blocks for improved feature learning and pattern recognition. ESRNet utilizes residual blocks from the ResNet architecture, featuring skip connections that enable identity mapping. Through direct addition of the input tensor to the convolutional layer output within each block, skip connections preserve the gradient flow. This mechanism prevents vanishing gradients, ensuring effective information propagation across network layers during training. Furthermore, ESRNet integrates efficient downsampling techniques and stabilizing batch normalization layers, which collectively contribute to its robust and reliable performance. Extensive experimental results reveal that ESRNet significantly outperforms other approaches in terms of accuracy, sensitivity, specificity, F-score, and Kappa statistics, with median values of 99.62%, 99.68%, 99.89%, 99.47%, and 99.42%, respectively. Moreover, the achieved minimum performance metrics, including accuracy (99.34%), sensitivity (99.47%), specificity (99.79%), F-score (99.04%), and Kappa statistics (99.21%), underscore the exceptional effectiveness of ESRNet for BTC. Therefore, the proposed ESRNet showcases exceptional performance and efficiency in BTC, holding the potential to revolutionize clinical diagnosis and treatment planning.

Integrating unsupervised language model with multi-view multiple sequence alignments for high-accuracy inter-chain contact prediction.

Accurate identification of inter-chain contacts in the protein complex is critical to determine the corresponding 3D structures and understand the biological functions. We proposed a new deep learning method, ICCPred, to deduce the inter-chain contacts from the amino acid sequences of the protein complex. This pipeline was built on the designed deep residual network architecture, integrating the pre-trained language model with three multiple sequence alignments (MSAs) from different biological views. Experimental results on 709 non-redundant benchmarking protein complexes showed that the proposed ICCPred significantly increased inter-chain contact prediction accuracy compared to the state-of-the-art approaches. Detailed data analyses showed that the significant advantage of ICCPred lies in the utilization of pre-trained transformer language models which can effectively extract the complementary co-evolution diversity from three MSAs. Meanwhile, the designed deep residual network enhances the correlation between the co-evolution diversity and the patterns of inter-chain contacts. These results demonstrated a new avenue for high-accuracy deep-learning inter-chain contact prediction that is applicable to large-scale protein-protein interaction annotations from sequence alone.

Prediction of drug-protein interaction based on dual channel neural networks with attention mechanism.

The precise identification of drug-protein inter action (DPI) can significantly speed up the drug discovery process. Bioassay methods are time-consuming and expensive to screen for each pair of drug proteins. Machine-learning-based methods cannot accurately predict a large number of DPIs. Compared with traditional computing methods, deep learning methods need less domain knowledge and have strong data learning ability. In this study, we construct a DPI prediction model based on dual channel neural networks with an efficient path attention mechanism, called DCA-DPI. The drug molecular graph and protein sequence are used as the data input of the model, and the residual graph neural network and the residual convolution network are used to learn the feature representation of the drug and protein, respectively, to obtain the feature vector of the drug and the hidden vector of protein. To get a more accurate protein feature vector, the weighted sum of the hidden vector of protein is applied using the neural attention mechanism. In the end, drug and protein vectors are concatenated and input into the full connection layer for classification. In order to evaluate the performance of DCA-DPI, three widely used public data, Human, C.elegans and DUD-E, are used in the experiment. The evaluation metrics values in the experiment are superior to other relevant methods. Experiments show that our model is efficient for DPI prediction.

R2Net: Efficient and flexible diffeomorphic image registration using Lipschitz continuous residual networks.

Classical diffeomorphic image registration methods, while being accurate, face the challenges of high computational costs. Deep learning based approaches provide a fast alternative to address these issues; however, most existing deep solutions either lose the good property of diffeomorphism or have limited flexibility to capture large deformations, under the assumption that deformations are driven by stationary velocity fields (SVFs). Also, the adopted squaring and scaling technique for integrating SVFs is time- and memory-consuming, hindering deep methods from handling large image volumes. In this paper, we present an unsupervised diffeomorphic image registration framework, which uses deep residual networks (ResNets) as numerical approximations of the underlying continuous diffeomorphic setting governed by ordinary differential equations, which is parameterized by either SVFs or time-varying (non-stationary) velocity fields. This flexible parameterization in our Residual Registration Network (R2Net) not only provides the model's ability to capture large deformation but also reduces the time and memory cost when integrating velocity fields for deformation generation. Also, we introduce a Lipschitz continuity constraint into the ResNet block to help achieve diffeomorphic deformations. To enhance the ability of our model for handling images with large volume sizes, we employ a hierarchical extension with a multi-phase learning strategy to solve the image registration task in a coarse-to-fine fashion. We demonstrate our models on four 3D image registration tasks with a wide range of anatomies, including brain MRIs, cine cardiac MRIs, and lung CT scans. Compared to classical methods SyN and diffeomorphic VoxelMorph, our models achieve comparable or better registration accuracy with much smoother deformations. Our source code is available online at https://github.com/ankitajoshi15/R2Net.

Multivariable-incorporating super-resolution residual network for transcranial focused ultrasound simulation.

BACKGROUND AND OBJECTIVE: Transcranial focused ultrasound (tFUS) has emerged as a new non-invasive brain stimulation (NIBS) modality, with its exquisite ability to reach deep brain areas at a high spatial resolution. Accurate placement of an acoustic focus to a target region of the brain is crucial during tFUS treatment; however, the distortion of acoustic wave propagation through the intact skull casts challenges. High-resolution numerical simulation allows for monitoring of the acoustic pressure field in the cranium but also demands extensive computational loads. In this study, we adopt a super-resolution residual network technique based on a deep convolution to enhance the prediction quality of the FUS acoustic pressure field in the targeted brain regions. METHODS: The training dataset was acquired by numerical simulations performed at low-(1.0 mm) and high-resolutions (0.5mm) on three ex vivo human calvariae. Five different super-resolution (SR) network models were trained by using a multivariable dataset in 3D, which incorporated information on the acoustic pressure field, wave velocity, and localized skull computed tomography (CT) images. RESULTS: The accuracy of 80.87±4.50% in predicting the focal volume with a substantial improvement of 86.91% in computational cost compared to the conventional high-resolution numerical simulation was achieved. The results suggest that the method can greatly reduce the simulation time without sacrificing accuracy and improve the accuracy further with the use of additional inputs. CONCLUSIONS: In this research, we developed multivariable-incorporating SR neural networks for transcranial focused ultrasound simulation. Our super-resolution technique may contribute to promoting the safety and efficacy of tFUS-mediated NIBS by providing on-site feedback information on the intracranial pressure field to the operator.

A Novel Image Classification Method Based on Residual Network, Inception, and Proposed Activation Function.

In deeper layers, ResNet heavily depends on skip connections and Relu. Although skip connections have demonstrated their usefulness in networks, a major issue arises when the dimensions between layers are not consistent. In such cases, it is necessary to use techniques such as zero-padding or projection to match the dimensions between layers. These adjustments increase the complexity of the network architecture, resulting in an increase in parameter number and a rise in computational costs. Another problem is the vanishing gradient caused by utilizing Relu. In our model, after making appropriate adjustments to the inception blocks, we replace the deeper layers of ResNet with modified inception blocks and Relu with our non-monotonic activation function (NMAF). To reduce parameter number, we use symmetric factorization and 1×1 convolutions. Utilizing these two techniques contributed to reducing the parameter number by around 6 M parameters, which has helped reduce the run time by 30 s/epoch. Unlike Relu, NMAF addresses the deactivation problem of the non-positive number by activating the negative values and outputting small negative numbers instead of zero in Relu, which helped in enhancing the convergence speed and increasing the accuracy by 5%, 15%, and 5% for the non-noisy datasets, and 5%, 6%, 21% for non-noisy datasets.

Improved Residual Network based on norm-preservation for visual recognition.

Residual Network (ResNet) achieves deeper and wider networks with high-performance gains, representing a powerful convolutional neural network architecture. In this paper, we propose architectural refinements to ResNet that address the information flow through several layers of the network, including the input stem, downsampling block, projection shortcut, and identity blocks. We will show that our collective refinements facilitate stable backpropagation by preserving the norm of the error gradient within the residual blocks, which can reduce the optimization difficulties of training very deep networks. Our proposed modifications enhance the learning dynamics, resulting in high accuracy and inference performance by enforcing norm-preservation throughout the network training. The effectiveness of our method is verified by extensive experimental results on five computer vision tasks, including image classification (ImageNet and CIFAR-100), video classification (Kinetics-400), multi-label image recognition (MS-COCO), object detection and semantic segmentation (PASCAL VOC). We also empirically show consistent improvements in generalization performance when applying our modifications over different networks to provide new insights and inspire new architectures. The source code is publicly available at: https://github.com/bharatmahaur/LeNo.