Severity of in vivo corticospinal tract degeneration is associated with survival in amyotrophic lateral sclerosis: a longitudinal, multicohort study.

BACKGROUND AND PURPOSE: This study sought to evaluate the relationship of progressive corticospinal tract (CST) degeneration with survival in patients with amyotrophic lateral sclerosis (ALS). METHODS: Forty-one ALS patients and 42 healthy controls were prospectively recruited from the Canadian ALS Neuroimaging Consortium. Magnetic resonance imaging scanning and clinical evaluations were performed on participants at three serial visits with 4-month intervals. Texture analysis was performed on T1-weighted magnetic resonance imaging scans and the texture feature 'autocorrelation' was quantified. Whole-brain group-level comparisons were performed between patient subgroups. Linear mixed models were used to evaluate longitudinal progression. Region-of-interest and 3D voxel-wise Cox proportional-hazards regression models were constructed for survival prediction. For all survival analyses, a second independent cohort was used for model validation. RESULTS: Autocorrelation of the bilateral CST was increased at baseline and progressively increased over time at a faster rate in ALS short survivors. Cox proportional-hazards regression analyses revealed autocorrelation of the CST as a significant predictor of survival at 5 years follow-up (hazard ratio 1.28, p = 0.005). Similarly, voxel-wise whole-brain survival analyses revealed that increased autocorrelation of the CST was associated with shorter survival. ALS patients stratified by median autocorrelation in the CST had significantly different survival times using the Kaplan-Meier curve and log-rank tests (χ2  = 7.402, p = 0.007). CONCLUSIONS: Severity of cerebral degeneration is associated with survival in ALS. CST degeneration progresses faster in subgroups of patients with shorter survival. Neuroimaging holds promise as a tool to improve patient management and facilitation of clinical trials.

Distinct patterns of progressive gray and white matter degeneration in amyotrophic lateral sclerosis.

Progressive cerebral degeneration in amyotrophic lateral sclerosis (ALS) remains poorly understood. Here, three-dimensional (3D) texture analysis was used to study longitudinal gray and white matter cerebral degeneration in ALS from routine T1-weighted magnetic resonance imaging (MRI). Participants were included from the Canadian ALS Neuroimaging Consortium (CALSNIC) who underwent up to three clinical assessments and MRI at four-month intervals, up to 8 months after baseline (T0 ). Three-dimensional maps of the texture feature autocorrelation were computed from T1-weighted images. One hundred and nineteen controls and 137 ALS patients were included, with 81 controls and 84 ALS patients returning for at least one follow-up. At baseline, texture changes in ALS patients were detected in the motor cortex, corticospinal tract, insular cortex, and bilateral frontal and temporal white matter compared to controls. Longitudinal comparison of texture maps between T0 and Tmax (last follow-up visit) within ALS patients showed progressive texture alterations in the temporal white matter, insula, and internal capsule. Additionally, when compared to controls, ALS patients had greater texture changes in the frontal and temporal structures at Tmax than at T0 . In subgroup analysis, slow progressing ALS patients had greater progressive texture change in the internal capsule than the fast progressing patients. Contrastingly, fast progressing patients had greater progressive texture changes in the precentral gyrus. These findings suggest that the characteristic longitudinal gray matter pathology in ALS is the progressive involvement of frontotemporal regions rather than a worsening pathology within the motor cortex, and that phenotypic variability is associated with distinct progressive spatial pathology.

Reliability of 3D texture analysis: A multicenter MRI study of the brain.

BACKGROUND: Texture analysis (TA) is an image-analysis technique that detects complex intervoxel statistical patterns. 3D TA has shown potential in detecting cerebral degeneration not perceptible to the human eye in many neurological disorders. The reliability of this method's application in a multicenter study is unknown. PURPOSE: To assess the intrasite and intersite reliability of a novel 3D TA method from data acquired systematically from the Canadian ALS Neuroimaging Consortium (CALSNIC). STUDY TYPE: Prospective multicenter data with harmonized MR sequence parameters acquired from five sites. POPULATION: Six healthy subjects. FIELD STRENGTH: 3T 3D-MPRAGE and 3D-SPGR T1 -weighted MRI of the brain. ASSESSMENT: Voxel-based 3D TA was performed on the whole brain to produce texture maps. STATISTICAL TESTS: Intra- and intersite reliability of texture features was assessed using a two-way mixed-effects model for intraclass correlation coefficients (ICC). ICCs were calculated in a region-of-interest (ROI) analysis of predetermined anatomically relevant areas. A voxelwise approach was used to assess the whole brain. RESULTS: In the ROI analyses, intrasite reliability was excellent (ICC > 0.75) across most regions and texture features (autocorrelation [autoc], contrast [contr], energy [energ]). Intersite reliability was excellent for most regions with autoc, ranging from fair to excellent for contr, and ICCs ranging from poor to good (<0.40-0.75) for energ. Voxelwise analyses revealed a large range in ICC across the brain for both intrasite and intersite ICCs (0.0-0.90), with higher reliability in the cortical gray matter compared with deeper subcortical structures. DATA CONCLUSION: Overall, the reliability of 3D TA was highly dependent on texture feature, region studied, and method of analysis (ROI or voxelwise). Intrasite reproducibility was good to excellent, and better than intersite. ROI-based analyses present higher reliability in comparison with voxelwise analyses. Autoc has overall excellent reliability. These factors might be considered when designing future 3D TA studies. LEVEL OF EVIDENCE: 2 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2020;51:1200-1209.

Corticospinal tract degeneration in ALS unmasked in T1-weighted images using texture analysis.

The purpose of this study was to investigate whether textures computed from T1-weighted (T1W) images of the corticospinal tract (CST) in amyotrophic lateral sclerosis (ALS) are associated with degenerative changes evaluated by diffusion tensor imaging (DTI). Nineteen patients with ALS and 14 controls were prospectively recruited and underwent T1W and diffusion-weighted magnetic resonance imaging. Three-dimensional texture maps were computed from T1W images and correlated with the DTI metrics within the CST. Significantly correlated textures were selected and compared within the CST for group differences between patients and controls using voxel-wise analysis. Textures were correlated with the patients' clinical upper motor neuron (UMN) signs and their diagnostic accuracy was evaluated. Voxel-wise analysis of textures and their diagnostic performance were then assessed in an independent cohort with 26 patients and 13 controls. Results showed that textures autocorrelation, energy, and inverse difference normalized significantly correlated with DTI metrics (p < .05) and these textures were selected for further analyses. The textures demonstrated significant voxel-wise differences between patients and controls in the centrum semiovale and the posterior limb of the internal capsule bilaterally (p < .05). Autocorrelation and energy significantly correlated with UMN burden in patients (p < .05) and classified patients and controls with 97% accuracy (100% sensitivity, 92.9% specificity). In the independent cohort, the selected textures demonstrated similar regional differences between patients and controls and classified participants with 94.9% accuracy. These results provide evidence that T1-based textures are associated with degenerative changes in the CST.

Texture Analysis to Detect Cerebral Degeneration in Amyotrophic Lateral Sclerosis.

BACKGROUND: Evidence of cerebral degeneration is not apparent on routine brain MRI in amyotrophic lateral sclerosis (ALS). Texture analysis can detect change in images based on the statistical properties of voxel intensities. Our objective was to test the utility of texture analysis in detecting cerebral degeneration in ALS. A secondary objective was to determine whether the performance of texture analysis is dependent on image resolution. METHODS: High-resolution (0.5×0.5 mm2 in-plane) coronal T2-weighted MRI of the brain were acquired from 12 patients with ALS and 19 healthy controls on a 4.7 Tesla MRI system. Image data sets at lower resolutions were created by down-sampling to 1×1, 2×2, 3×3, and 4×4 mm2. Texture features were extracted from a slice encompassing the corticospinal tract at the different resolutions and tested for their discriminatory power and correlations with clinical measures. Subjects were also classified by visual assessment by expert reviewers. RESULTS: Texture features were different between ALS patients and healthy controls at 1×1, 2×2, and 3×3 mm2 resolutions. Texture features correlated with measures of upper motor neuron function and disability. Optimal classification performance was achieved when best-performing texture features were combined with visual assessment at 2×2 mm2 resolution (0.851 area under the curve, 83% sensitivity, 79% specificity). CONCLUSIONS: Texture analysis can detect subtle abnormalities in MRI of ALS patients. The clinical yield of the method is dependent on image resolution. Texture analysis holds promise as a potential source of neuroimaging biomarkers in ALS.

Positive-unlabeled ensemble learning for kinase substrate prediction from dynamic phosphoproteomics data.

MOTIVATION: Protein phosphorylation is a post-translational modification that underlines various aspects of cellular signaling. A key step to reconstructing signaling networks involves identification of the set of all kinases and their substrates. Experimental characterization of kinase substrates is both expensive and time-consuming. To expedite the discovery of novel substrates, computational approaches based on kinase recognition sequence (motifs) from known substrates, protein structure, interaction and co-localization have been proposed. However, rarely do these methods take into account the dynamic responses of signaling cascades measured from in vivo cellular systems. Given that recent advances in mass spectrometry-based technologies make it possible to quantify phosphorylation on a proteome-wide scale, computational approaches that can integrate static features with dynamic phosphoproteome data would greatly facilitate the prediction of biologically relevant kinase-specific substrates. RESULTS: Here, we propose a positive-unlabeled ensemble learning approach that integrates dynamic phosphoproteomics data with static kinase recognition motifs to predict novel substrates for kinases of interest. We extended a positive-unlabeled learning technique for an ensemble model, which significantly improves prediction sensitivity on novel substrates of kinases while retaining high specificity. We evaluated the performance of the proposed model using simulation studies and subsequently applied it to predict novel substrates of key kinases relevant to insulin signaling. Our analyses show that static sequence motifs and dynamic phosphoproteomics data are complementary and that the proposed integrated model performs better than methods relying only on static information for accurate prediction of kinase-specific substrates. AVAILABILITY AND IMPLEMENTATION: Executable GUI tool, source code and documentation are freely available at https://github.com/PengyiYang/KSP-PUEL. CONTACT: pengyi.yang@nih.gov or jothi@mail.nih.gov SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Transcriptome-wide targets of alternative splicing by RBM4 and possible role in cancer.

This study determined transcriptome-wide targets of the splicing factor RBM4 using Affymetrix GeneChip(®) Human Exon 1.0 ST Arrays and HeLa cells treated with RBM4-specific siRNA. This revealed 238 transcripts that were targeted for alternative splicing. Cross-linking and immunoprecipitation experiments identified 945 RBM4 targets in mouse HEK293 cells, 39% of which were ascribed to "alternative splicing" by in silico pathway analysis. Mouse embryonic stem cells transfected with Rbm4 siRNA hairpins exhibited reduced colony numbers and size consistent with involvement of RBM4 in cell proliferation. RBM4 cDNA probing of a cancer cDNA array involving 18 different tumor types from 13 different tissues and matching normal tissue found overexpression of RBM4 mRNA (p<0.01) in cervical, breast, lung, colon, ovarian and rectal cancers. Many RBM4 targets we identified have been implicated in these cancers. In conclusion, our findings reveal transcriptome-wide targets of RBM4 and point to potential cancer-related targets and mechanisms that may involve RBM4.

Adaptive Local Spatiotemporal Features from RGB-D Data for One-Shot Learning Gesture Recognition.

Noise and constant empirical motion constraints affect the extraction of distinctive spatiotemporal features from one or a few samples per gesture class. To tackle these problems, an adaptive local spatiotemporal feature (ALSTF) using fused RGB-D data is proposed. First, motion regions of interest (MRoIs) are adaptively extracted using grayscale and depth velocity variance information to greatly reduce the impact of noise. Then, corners are used as keypoints if their depth, and velocities of grayscale and of depth meet several adaptive local constraints in each MRoI. With further filtering of noise, an accurate and sufficient number of keypoints is obtained within the desired moving body parts (MBPs). Finally, four kinds of multiple descriptors are calculated and combined in extended gradient and motion spaces to represent the appearance and motion features of gestures. The experimental results on the ChaLearn gesture, CAD-60 and MSRDailyActivity3D datasets demonstrate that the proposed feature achieves higher performance compared with published state-of-the-art approaches under the one-shot learning setting and comparable accuracy under the leave-one-out cross validation.

Direction pathway analysis of large-scale proteomics data reveals novel features of the insulin action pathway.

MOTIVATION: With the advancement of high-throughput techniques, large-scale profiling of biological systems with multiple experimental perturbations is becoming more prevalent. Pathway analysis incorporates prior biological knowledge to analyze genes/proteins in groups in a biological context. However, the hypotheses under investigation are often confined to a 1D space (i.e. up, down, either or mixed regulation). Here, we develop direction pathway analysis (DPA), which can be applied to test hypothesis in a high-dimensional space for identifying pathways that display distinct responses across multiple perturbations. RESULTS: Our DPA approach allows for the identification of pathways that display distinct responses across multiple perturbations. To demonstrate the utility and effectiveness, we evaluated DPA under various simulated scenarios and applied it to study insulin action in adipocytes. A major action of insulin in adipocytes is to regulate the movement of proteins from the interior to the cell surface membrane. Quantitative mass spectrometry-based proteomics was used to study this process on a large-scale. The combined dataset comprises four separate treatments. By applying DPA, we identified that several insulin responsive pathways in the plasma membrane trafficking are only partially dependent on the insulin-regulated kinase Akt. We subsequently validated our findings through targeted analysis of key proteins from these pathways using immunoblotting and live cell microscopy. Our results demonstrate that DPA can be applied to dissect pathway networks testing diverse hypotheses and integrating multiple experimental perturbations. AVAILABILITY AND IMPLEMENTATION: The R package 'directPA' is distributed from CRAN under GNU General Public License (GPL)-3 and can be downloaded from: http://cran.r-project.org/web/packages/directPA/index.html CONTACT: jean.yang@sydney.edu.au SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Learning to Recover Spectral Reflectance From RGB Images.

This paper tackles spectral reflectance recovery (SRR) from RGB images. Since capturing ground-truth spectral reflectance and camera spectral sensitivity are challenging and costly, most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images, which are suboptimal especially when the trained models are tested on real images because they never exploit the internal information of the testing images. To address this issue, we adopt a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information. To the best of our knowledge, this is the first work that successfully adapts the MAXL strategy to this problem. Instead of relying on naive end-to-end training, we also propose a novel architecture that integrates the physical relationship between the spectral reflectance and the corresponding RGB images into the network based on our mathematical analysis. Besides, since the spectral reflectance of a scene is independent to its illumination while the corresponding RGB images are not, we recover the spectral reflectance of a scene from its RGB images captured under multiple illuminations to further reduce the unknown. Qualitative and quantitative evaluations demonstrate the effectiveness of our proposed network and of the MAXL. Our code and data are available at https://github.com/Dong-Huo/SRR-MAXL.

Molecular interaction networks for the analysis of human disease: utility, limitations, and considerations.

High-throughput '-omics' data can be combined with large-scale molecular interaction networks, for example, protein-protein interaction networks, to provide a unique framework for the investigation of human molecular biology. Interest in these integrative '-omics' methods is growing rapidly because of their potential to understand complexity and association with disease; such approaches have a focus on associations between phenotype and "network-type." The potential of this research is enticing, yet there remain a series of important considerations. Here, we discuss interaction data selection, data quality, the relative merits of using data from large high-throughput studies versus a meta-database of smaller literature-curated studies, and possible issues of sociological or inspection bias in interaction data. Other work underway, especially international consortia to establish data formats, quality standards and address data redundancy, and the improvements these efforts are making to the field, is also evaluated. We present options for researchers intending to use large-scale molecular interaction networks as a functional context for protein or gene expression data, including microRNAs, especially in the context of human disease.

Estimation of data-specific constitutive exons with RNA-Seq data.

BACKGROUND: RNA-Seq has the potential to answer many diverse and interesting questions about the inner workings of cells. Estimating changes in the overall transcription of a gene is not straightforward. Changes in overall gene transcription can easily be confounded with changes in exon usage which alter the lengths of transcripts produced by a gene. Measuring the expression of constitutive exons--xons which are consistently conserved after splicing--ffers an unbiased estimation of the overall transcription of a gene. RESULTS: We propose a clustering-based method, exClust, for estimating the exons that are consistently conserved after splicing in a given data set. These are considered as the exons which are "constitutive" in this data. The method utilises information from both annotation and the dataset of interest. The method is implemented in an openly available R function package, sydSeq. CONCLUSION: When used on two real datasets exClust includes more than three times as many reads as the standard UI method, and improves concordance with qRT-PCR data. When compared to other methods, our method is shown to produce robust estimates of overall gene transcription.

Improved moderation for gene-wise variance estimation in RNA-Seq via the exploitation of external information.

BACKGROUND: The cost of RNA-Seq has been decreasing over the last few years. Despite this, experiments with four or less biological replicates are still quite common. Estimating the variances of gene expression estimates becomes both a challenging and interesting problem in these situations of low replication. However, with the wealth of microarray and other publicly available gene expression data readily accessible on public repositories, these sources of information can be leveraged to make improvements in variance estimation. RESULTS: We have proposed a novel approach called Tshrink+ for inferring differential gene expression through improved modelling of the gene-wise variances. Existing methods share information between genes of similar average expression by shrinking, or moderating, the gene-wise variances to a fitted common variance. We have been able to achieve improved estimation of the common variance by using gene-wise sample variances from external experiments, as well as gene length. CONCLUSIONS: Using biological data we show that utilising additional external information can improve the modelling of the common variance and hence the calling of differentially expressed genes. These sources of additional information include gene length and gene-wise sample variances from other RNA-Seq and microarray datasets, of both related and seemingly unrelated tissue types. The results of this are promising, with our differential expression test, Tshrink+, performing favourably when compared to existing methods such as DESeq and edgeR when considering both gene ranking and sensitivity. These improved variance models could easily be implemented in both DESeq and edgeR and highlight the need for a database that offers a profile of gene variances over a range of tissue types and organisms.

ZRANB2 localizes to supraspliceosomes and influences the alternative splicing of multiple genes in the transcriptome.

Alternative splicing is a major source of protein diversity in humans. The human splicing factor zinc finger, Ran-binding domain containing protein 2 (ZRANB2) is a splicing protein whose specific endogenous targets are unknown. Its upregulation in grade III ovarian serous papillary carcinoma could suggest a role in some cancers. To determine whether ZRANB2 is part of the supraspliceosome, nuclear supernatants from human embryonic kidney 293 cells were prepared and then fractioned on a glycerol gradient, followed by Western blotting. The same was done after treatment with a tyrosine kinase to induce phosphorylation. This showed for the first time that ZRANB2 is part of the supraspliceosome, and that phosphorylation affects its subcellular location. Studies were then performed to understand the splicing targets of ZRANB2 at the whole-transcriptome level. HeLa cells were transfected with a vector containing ZRANB2 or with a vector-only control. RNA was extracted, converted to cDNA and hybridized to Affymetrix GeneChip(®) Human Exon 1.0 ST Arrays. At the FDR ≤1.3 significance level we found that ZRANB2 influenced the alternative splicing of primary transcripts of CENTB1, WDR78, C10orf18, CABP4, SMARCC2, SPATA13, OR4C6, ZNF263, CAPN10, SALL1, ST18 and ZP2. Several of these have been implicated in tumor development. In conclusion ZRANB2 is part of the supraspliceosome and causes differential splicing of numerous primary transcripts, some of which might have a role in cancer.

Blind Image Deconvolution Using Variational Deep Image Prior.

Conventional deconvolution methods utilize hand-crafted image priors to constrain the optimization. While deep-learning-based methods have simplified the optimization by end-to-end training, they fail to generalize well to blurs unseen in the training dataset. Thus, training image-specific models is important for higher generalization. Deep image prior (DIP) provides an approach to optimize the weights of a randomly initialized network with a single degraded image by maximum a posteriori (MAP), which shows that the architecture of a network can serve as the hand-crafted image prior. Unlike conventional hand-crafted image priors, which are obtained through statistical methods, finding a suitable network architecture is challenging due to the unclear relationship between images and their corresponding architectures. As a result, the network architecture cannot provide enough constraint for the latent sharp image. This paper proposes a new variational deep image prior (VDIP) for blind image deconvolution, which exploits additive hand-crafted image priors on latent sharp images and approximates a distribution for each pixel to avoid suboptimal solutions. Our mathematical analysis shows that the proposed method can better constrain the optimization. The experimental results further demonstrate that the generated images have better quality than that of the original DIP on benchmark datasets.

Glass Segmentation with RGB-Thermal Image Pairs.

This paper proposes a new glass segmentation method utilizing paired RGB and thermal images. Due to the large difference between the transmission property of visible light and that of the thermal energy through the glass where most glass is transparent to the visible light but opaque to thermal energy, glass regions of a scene are made more distinguishable with a pair of RGB and thermal images than solely with an RGB image. To exploit such a unique property, we propose a neural network architecture that effectively combines an RGB-thermal image pair with a new multi-modal fusion module based on attention, and integrate CNN and transformer to extract local features and non-local dependencies, respectively. As well, we have collected a new dataset containing 5551 RGB-thermal image pairs with ground-truth segmentation annotations. The qualitative and quantitative evaluations demonstrate the effectiveness of the proposed approach on fusing RGB and thermal data for glass segmentation. Our code and data are available at https://github.com/Dong-Huo/RGB-T-Glass-Segmentation.

DSMRI: Domain Shift Analyzer for Multi-Center MRI Datasets.

In medical research and clinical applications, the utilization of MRI datasets from multiple centers has become increasingly prevalent. However, inherent variability between these centers presents challenges due to domain shift, which can impact the quality and reliability of the analysis. Regrettably, the absence of adequate tools for domain shift analysis hinders the development and validation of domain adaptation and harmonization techniques. To address this issue, this paper presents a novel Domain Shift analyzer for MRI (DSMRI) framework designed explicitly for domain shift analysis in multi-center MRI datasets. The proposed model assesses the degree of domain shift within an MRI dataset by leveraging various MRI-quality-related metrics derived from the spatial domain. DSMRI also incorporates features from the frequency domain to capture low- and high-frequency information about the image. It further includes the wavelet domain features by effectively measuring the sparsity and energy present in the wavelet coefficients. Furthermore, DSMRI introduces several texture features, thereby enhancing the robustness of the domain shift analysis process. The proposed framework includes visualization techniques such as t-SNE and UMAP to demonstrate that similar data are grouped closely while dissimilar data are in separate clusters. Additionally, quantitative analysis is used to measure the domain shift distance, domain classification accuracy, and the ranking of significant features. The effectiveness of the proposed approach is demonstrated using experimental evaluations on seven large-scale multi-site neuroimaging datasets.

Effects of MRI scanner manufacturers in classification tasks with deep learning models.

Deep learning has become a leading subset of machine learning and has been successfully employed in diverse areas, ranging from natural language processing to medical image analysis. In medical imaging, researchers have progressively turned towards multi-center neuroimaging studies to address complex questions in neuroscience, leveraging larger sample sizes and aiming to enhance the accuracy of deep learning models. However, variations in image pixel/voxel characteristics can arise between centers due to factors including differences in magnetic resonance imaging scanners. Such variations create challenges, particularly inconsistent performance in machine learning-based approaches, often referred to as domain shift, where the trained models fail to achieve satisfactory or improved results when confronted with dissimilar test data. This study analyzes the performance of multiple disease classification tasks using multi-center MRI data obtained from three widely used scanner manufacturers (GE, Philips, and Siemens) across several deep learning-based networks. Furthermore, we investigate the efficacy of mitigating scanner vendor effects using ComBat-based harmonization techniques when applied to multi-center datasets of 3D structural MR images. Our experimental results reveal a substantial decline in classification performance when models trained on one type of scanner manufacturer are tested with data from different manufacturers. Moreover, despite applying ComBat-based harmonization, the harmonized images do not demonstrate any noticeable performance enhancement for disease classification tasks.

Adams-based hierarchical features fusion network for image dehazing.

Recent developments in Convolutional Neural Networks (CNNs) have made them one of the most powerful image dehazing methods. In particular, the Residual Networks (ResNets), which can avoid the vanishing gradient problem effectively, are widely deployed. To understand the success of ResNets, recent mathematical analysis of ResNets reveals that a ResNet has a similar formulation as the Euler method in solving the Ordinary Differential Equations (ODE's). Hence, image dehazing which can be formulated as an optimal control problem in dynamical systems can be solved by a single-step optimal control method, such as the Euler method. This optimal control viewpoint provides a new perspective to address the problem of image restoration. Motivated by the advantages of multi-step optimal control solvers in ODE's, which include better stability and efficiency than single-step solvers, e.g. Euler, we propose the Adams-based Hierarchical Feature Fusion Network (AHFFN) for image dehazing with modules inspired by a multi-step optimal control method named the Adams-Bashforth method. Firstly, we extend a multi-step Adams-Bashforth method to the corresponding Adams block, which achieves a higher accuracy than that of single-step solvers because of its more effective use of intermediate results. Then, we stack multiple Adams blocks to mimic the discrete approximation process of an optimal control in a dynamical system. To improve the results, the hierarchical features from stacked Adams blocks are fully used by combining Hierarchical Feature Fusion (HFF) and Lightweight Spatial Attention (LSA) with Adams blocks to form a new Adams module. Finally, we not only use HFF and LSA to fuse features, but also highlight important spatial information in each Adams module for estimating the clear image. The experimental results using synthetic and real images demonstrate that the proposed AHFFN obtains better accuracy and visual results than that of state-of-the-art methods.

HIPA: Hierarchical Patch Transformer for Single Image Super Resolution.

Transformer-based architectures start to emerge in single image super resolution (SISR) and have achieved promising performance. However, most existing vision Transformer-based SISR methods still have two shortcomings: (1) they divide images into the same number of patches with a fixed size, which may not be optimal for restoring patches with different levels of texture richness; and (2) their position encodings treat all input tokens equally and hence, neglect the dependencies among them. This paper presents a HIPA, which stands for a novel Transformer architecture that progressively recovers the high resolution image using a hierarchical patch partition. Specifically, we build a cascaded model that processes an input image in multiple stages, where we start with tokens with small patch sizes and gradually merge them to form the full resolution. Such a hierarchical patch mechanism not only explicitly enables feature aggregation at multiple resolutions but also adaptively learns patch-aware features for different image regions, e.g., using a smaller patch for areas with fine details and a larger patch for textureless regions. Meanwhile, a new attention-based position encoding scheme for Transformer is proposed to let the network focus on which tokens should be paid more attention by assigning different weights to different tokens, which is the first time to our best knowledge. Furthermore, we also propose a multi-receptive field attention module to enlarge the convolution receptive field from different branches. The experimental results on several public datasets demonstrate the superior performance of the proposed HIPA over previous methods quantitatively and qualitatively. We will share our code and models when the paper is accepted.