Road Traffic Injuries and the Built Environment in Bogotá, Colombia, 2015-2019: A Cross-Sectional Analysis.

Nine in 10 road traffic deaths occur in low- and middle-income countries (LMICs). Despite this disproportionate burden, few studies have examined built environment correlates of road traffic injury in these settings, including in Latin America. We examined road traffic collisions in Bogotá, Colombia, occurring between 2015 and 2019, and assessed the association between neighborhood-level built environment features and pedestrian injury and death. We used descriptive statistics to characterize all police-reported road traffic collisions that occurred in Bogotá between 2015 and 2019. Cluster detection was used to identify spatial clustering of pedestrian collisions. Adjusted multivariate Poisson regression models were fit to examine associations between several neighborhood-built environment features and rate of pedestrian road traffic injury and death. A total of 173,443 police-reported traffic collisions occurred in Bogotá between 2015 and 2019. Pedestrians made up about 25% of road traffic injuries and 50% of road traffic deaths in Bogotá between 2015 and 2019. Pedestrian collisions were spatially clustered in the southwestern region of Bogotá. Neighborhoods with more street trees (RR, 0.90; 95% CI, 0.82-0.98), traffic signals (0.89, 0.81-0.99), and bus stops (0.89, 0.82-0.97) were associated with lower pedestrian road traffic deaths. Neighborhoods with greater density of large roads were associated with higher pedestrian injury. Our findings highlight the potential for pedestrian-friendly infrastructure to promote safer interactions between pedestrians and motorists in Bogotá and in similar urban contexts globally.

PiGLET: Pixel-Level Grounding of Language Expressions With Transformers.

This paper proposes Panoptic Narrative Grounding, a spatially fine and general formulation of the natural language visual grounding problem. We establish an experimental framework for the study of this new task, including new ground truth and metrics. We propose PiGLET, a novel multi-modal Transformer architecture to tackle the Panoptic Narrative Grounding task, and to serve as a stepping stone for future work. We exploit the intrinsic semantic richness in an image by including panoptic categories, and we approach visual grounding at a fine-grained level using segmentations. In terms of ground truth, we propose an algorithm to automatically transfer Localized Narratives annotations to specific regions in the panoptic segmentations of the MS COCO dataset. PiGLET achieves a performance of 63.2 absolute Average Recall points. By leveraging the rich language information on the Panoptic Narrative Grounding benchmark on MS COCO, PiGLET obtains an improvement of 0.4 Panoptic Quality points over its base method on the panoptic segmentation task. Finally, we demonstrate the generalizability of our method to other natural language visual grounding problems such as Referring Expression Segmentation. PiGLET is competitive with previous state-of-the-art in RefCOCO, RefCOCO+ and RefCOCOg.

3D Visualization, Skeletonization and Branching Analysis of Blood Vessels in Angiogenesis.

Angiogenesis is the process of new blood vessels growing from existing vasculature. Visualizing them as a three-dimensional (3D) model is a challenging, yet relevant, task as it would be of great help to researchers, pathologists, and medical doctors. A branching analysis on the 3D model would further facilitate research and diagnostic purposes. In this paper, a pipeline of vision algorithms is elaborated to visualize and analyze blood vessels in 3D from formalin-fixed paraffin-embedded (FFPE) granulation tissue sections with two different staining methods. First, a U-net neural network is used to segment blood vessels from the tissues. Second, image registration is used to align the consecutive images. Coarse registration using an image-intensity optimization technique, followed by finetuning using a neural network based on Spatial Transformers, results in an excellent alignment of images. Lastly, the corresponding segmented masks depicting the blood vessels are aligned and interpolated using the results of the image registration, resulting in a visualized 3D model. Additionally, a skeletonization algorithm is used to analyze the branching characteristics of the 3D vascular model. In summary, computer vision and deep learning is used to reconstruct, visualize and analyze a 3D vascular model from a set of parallel tissue samples. Our technique opens innovative perspectives in the pathophysiological understanding of vascular morphogenesis under different pathophysiological conditions and its potential diagnostic role.

The Medical Segmentation Decathlon.

International challenges have become the de facto standard for comparative assessment of image analysis algorithms. Although segmentation is the most widely investigated medical image processing task, the various challenges have been organized to focus only on specific clinical tasks. We organized the Medical Segmentation Decathlon (MSD)-a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities to investigate the hypothesis that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. MSD results confirmed this hypothesis, moreover, MSD winner continued generalizing well to a wide range of other clinical problems for the next two years. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to scientists that are not versed in AI model training.

Rational Discovery of Antimicrobial Peptides by Means of Artificial Intelligence.

Antibiotic resistance is a worldwide public health problem due to the costs and mortality rates it generates. However, the large pharmaceutical industries have stopped searching for new antibiotics because of their low profitability, given the rapid replacement rates imposed by the increasingly observed resistance acquired by microorganisms. Alternatively, antimicrobial peptides (AMPs) have emerged as potent molecules with a much lower rate of resistance generation. The discovery of these peptides is carried out through extensive in vitro screenings of either rational or non-rational libraries. These processes are tedious and expensive and generate only a few AMP candidates, most of which fail to show the required activity and physicochemical properties for practical applications. This work proposes implementing an artificial intelligence algorithm to reduce the required experimentation and increase the efficiency of high-activity AMP discovery. Our deep learning (DL) model, called AMPs-Net, outperforms the state-of-the-art method by 8.8% in average precision. Furthermore, it is highly accurate to predict the antibacterial and antiviral capacity of a large number of AMPs. Our search led to identifying two unreported antimicrobial motifs and two novel antimicrobial peptides related to them. Moreover, by coupling DL with molecular dynamics (MD) simulations, we were able to find a multifunctional peptide with promising therapeutic effects. Our work validates our previously proposed pipeline for a more efficient rational discovery of novel AMPs.

Predicting target-ligand interactions with graph convolutional networks for interpretable pharmaceutical discovery.

Drug Discovery is an active research area that demands great investments and generates low returns due to its inherent complexity and great costs. To identify potential therapeutic candidates more effectively, we propose protein-ligand with adversarial augmentations network (PLA-Net), a deep learning-based approach to predict target-ligand interactions. PLA-Net consists of a two-module deep graph convolutional network that considers ligands' and targets' most relevant chemical information, successfully combining them to find their binding capability. Moreover, we generate adversarial data augmentations that preserve relevant biological backgrounds and improve the interpretability of our model, highlighting the relevant substructures of the ligands reported to interact with the protein targets. Our experiments demonstrate that the joint ligand-target information and the adversarial augmentations significantly increase the interaction prediction performance. PLA-Net achieves 86.52% in mean average precision for 102 target proteins with perfect performance for 30 of them, in a curated version of actives as decoys dataset. Lastly, we accurately predict pharmacologically-relevant molecules when screening the ligands of ChEMBL and drug repurposing Hub datasets with the perfect-scoring targets.

Smart pooling: AI-powered COVID-19 informative group testing.

Massive molecular testing for COVID-19 has been pointed out as fundamental to moderate the spread of the pandemic. Pooling methods can enhance testing efficiency, but they are viable only at low incidences of the disease. We propose Smart Pooling, a machine learning method that uses clinical and sociodemographic data from patients to increase the efficiency of informed Dorfman testing for COVID-19 by arranging samples into all-negative pools. To do this, we ran an automated method to train numerous machine learning models on a retrospective dataset from more than 8000 patients tested for SARS-CoV-2 from April to July 2020 in Bogotá, Colombia. We estimated the efficiency gains of using the predictor to support Dorfman testing by simulating the outcome of tests. We also computed the attainable efficiency gains of non-adaptive pooling schemes mathematically. Moreover, we measured the false-negative error rates in detecting the ORF1ab and N genes of the virus in RT-qPCR dilutions. Finally, we presented the efficiency gains of using our proposed pooling scheme on proof-of-concept pooled tests. We believe Smart Pooling will be efficient for optimizing massive testing of SARS-CoV-2.

QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results.

Deep learning (DL) models have provided state-of-the-art performance in various medical imaging benchmarking challenges, including the Brain Tumor Segmentation (BraTS) challenges. However, the task of focal pathology multi-compartment segmentation (e.g., tumor and lesion sub-regions) is particularly challenging, and potential errors hinder translating DL models into clinical workflows. Quantifying the reliability of DL model predictions in the form of uncertainties could enable clinical review of the most uncertain regions, thereby building trust and paving the way toward clinical translation. Several uncertainty estimation methods have recently been introduced for DL medical image segmentation tasks. Developing scores to evaluate and compare the performance of uncertainty measures will assist the end-user in making more informed decisions. In this study, we explore and evaluate a score developed during the BraTS 2019 and BraTS 2020 task on uncertainty quantification (QU-BraTS) and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation. This score (1) rewards uncertainty estimates that produce high confidence in correct assertions and those that assign low confidence levels at incorrect assertions, and (2) penalizes uncertainty measures that lead to a higher percentage of under-confident correct assertions. We further benchmark the segmentation uncertainties generated by 14 independent participating teams of QU-BraTS 2020, all of which also participated in the main BraTS segmentation task. Overall, our findings confirm the importance and complementary value that uncertainty estimates provide to segmentation algorithms, highlighting the need for uncertainty quantification in medical image analyses. Finally, in favor of transparency and reproducibility, our evaluation code is made publicly available at https://github.com/RagMeh11/QU-BraTS.

MIcro-surgical anastomose workflow recognition challenge report.

BACKGROUND AND OBJECTIVE: Automatic surgical workflow recognition is an essential step in developing context-aware computer-assisted surgical systems. Video recordings of surgeries are becoming widely accessible, as the operational field view is captured during laparoscopic surgeries. Head and ceiling mounted cameras are also increasingly being used to record videos in open surgeries. This makes videos a common choice in surgical workflow recognition. Additional modalities, such as kinematic data captured during robot-assisted surgeries, could also improve workflow recognition. This paper presents the design and results of the MIcro-Surgical Anastomose Workflow recognition on training sessions (MISAW) challenge whose objective was to develop workflow recognition models based on kinematic data and/or videos. METHODS: The MISAW challenge provided a data set of 27 sequences of micro-surgical anastomosis on artificial blood vessels. This data set was composed of videos, kinematics, and workflow annotations. The latter described the sequences at three different granularity levels: phase, step, and activity. Four tasks were proposed to the participants: three of them were related to the recognition of surgical workflow at three different granularity levels, while the last one addressed the recognition of all granularity levels in the same model. We used the average application-dependent balanced accuracy (AD-Accuracy) as the evaluation metric. This takes unbalanced classes into account and it is more clinically relevant than a frame-by-frame score. RESULTS: Six teams participated in at least one task. All models employed deep learning models, such as convolutional neural networks (CNN), recurrent neural networks (RNN), or a combination of both. The best models achieved accuracy above 95%, 80%, 60%, and 75% respectively for recognition of phases, steps, activities, and multi-granularity. The RNN-based models outperformed the CNN-based ones as well as the dedicated modality models compared to the multi-granularity except for activity recognition. CONCLUSION: For high levels of granularity, the best models had a recognition rate that may be sufficient for applications such as prediction of remaining surgical time. However, for activities, the recognition rate was still low for applications that can be employed clinically. The MISAW data set is publicly available at http://www.synapse.org/MISAW to encourage further research in surgical workflow recognition.

Lung Nodule Malignancy Prediction in Sequential CT Scans: Summary of ISBI 2018 Challenge.

Lung cancer is by far the leading cause of cancer death in the US. Recent studies have demonstrated the effectiveness of screening using low dose CT (LDCT) in reducing lung cancer related mortality. While lung nodules are detected with a high rate of sensitivity, this exam has a low specificity rate and it is still difficult to separate benign and malignant lesions. The ISBI 2018 Lung Nodule Malignancy Prediction Challenge, developed by a team from the Quantitative Imaging Network of the National Cancer Institute, was focused on the prediction of lung nodule malignancy from two sequential LDCT screening exams using automated (non-manual) algorithms. We curated a cohort of 100 subjects who participated in the National Lung Screening Trial and had established pathological diagnoses. Data from 30 subjects were randomly selected for training and the remaining was used for testing. Participants were evaluated based on the area under the receiver operating characteristic curve (AUC) of nodule-wise malignancy scores generated by their algorithms on the test set. The challenge had 17 participants, with 11 teams submitting reports with method description, mandated by the challenge rules. Participants used quantitative methods, resulting in a reporting test AUC ranging from 0.698 to 0.913. The top five contestants used deep learning approaches, reporting an AUC between 0.87 - 0.91. The team's predictor did not achieve significant differences from each other nor from a volume change estimate (p =.05 with Bonferroni-Holm's correction).

PharmaNet: Pharmaceutical discovery with deep recurrent neural networks.

The discovery and development of novel pharmaceuticals is an area of active research mainly due to the large investments required and long payback times. As of 2016, the development of a novel drug candidate required up to $ USD 2.6 billion in investment for only 10% rate of approval by the FDA. To help decreasing the costs associated with the process, a number of in silico approaches have been developed with relatively low success due to limited predicting performance. Here, we introduced a machine learning-based algorithm as an alternative for a more accurate search of new pharmacological candidates, which takes advantage of Recurrent Neural Networks (RNN) for active molecule prediction within large databases. Our approach, termed PharmaNet was implemented here to search for ligands against specific cell receptors within 102 targets of the DUD-E database, which contains 22886 active molecules. PharmaNet comprises three main phases. First, a SMILES representation of the molecule is converted into a raw molecular image. Second, a convolutional encoder processes the data to obtain a fingerprint molecular image that is finally analyzed by a Recurrent Neural Network (RNN). This approach enables precise predictions of the molecules' target on the basis of the feature extraction, the sequence analysis and the relevant information filtered out throughout the process. Molecule Target prediction is a highly unbalanced detection problem and therefore, we propose that an adequate evaluation metric of performance is the area under the Normalized Average Precision (NAP) curve. PharmaNet largely surpasses the previous state-of-the-art method with 97.7% in the Receiver Operating Characteristic curve (ROC-AUC) and 65.5% in the NAP curve. We obtained a perfect performance for human farnesyl pyrophosphate synthase (FPPS), which is a potential target for antimicrobial and anticancer treatments. We decided to test PharmaNet for activity prediction against FPPS by searching in the CHEMBL data set. We obtained three (3) potential inhibitors that were further validated through both molecular docking and in silico toxicity prediction. Most importantly, one of this candidates, CHEMBL2007613, was predicted as a potential antiviral due to its involvement on the PCDH17 pathway, which has been reported to be related to viral infections.

Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge.

Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. While numerous methods for detecting, segmenting and tracking of medical instruments based on endoscopic video images have been proposed in the literature, key limitations remain to be addressed: Firstly, robustness, that is, the reliable performance of state-of-the-art methods when run on challenging images (e.g. in the presence of blood, smoke or motion artifacts). Secondly, generalization; algorithms trained for a specific intervention in a specific hospital should generalize to other interventions or institutions. In an effort to promote solutions for these limitations, we organized the Robust Medical Instrument Segmentation (ROBUST-MIS) challenge as an international benchmarking competition with a specific focus on the robustness and generalization capabilities of algorithms. For the first time in the field of endoscopic image processing, our challenge included a task on binary segmentation and also addressed multi-instance detection and segmentation. The challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures from three different types of surgery. The validation of the competing methods for the three tasks (binary segmentation, multi-instance detection and multi-instance segmentation) was performed in three different stages with an increasing domain gap between the training and the test data. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap. While the average detection and segmentation quality of the best-performing algorithms is high, future research should concentrate on detection and segmentation of small, crossing, moving and transparent instrument(s) (parts).

Automatic seizure detection based on imaged-EEG signals through fully convolutional networks.

Seizure detection is a routine process in epilepsy units requiring manual intervention of well-trained specialists. This process could be extensive, inefficient and time-consuming, especially for long term recordings. We proposed an automatic method to detect epileptic seizures using an imaged-EEG representation of brain signals. To accomplish this, we analyzed EEG signals from two different datasets: the CHB-MIT Scalp EEG database and the EPILEPSIAE project that includes scalp and intracranial recordings. We used fully convolutional neural networks to automatically detect seizures. For our best model, we reached average accuracy and specificity values of 99.3% and 99.6%, respectively, for the CHB-MIT dataset, and corresponding values of 98.0% and 98.3% for the EPILEPSIAE patients. For these patients, the inclusion of intracranial electrodes together with scalp ones increased the average accuracy and specificity values to 99.6% and 58.3%, respectively. Regarding the other metrics, our best model reached average precision of 62.7%, recall of 58.3%, F-measure of 59.0% and AP of 54.5% on the CHB-MIT recordings, and comparatively lowers performances for the EPILEPSIAE dataset. For both databases, the number of false alarms per hour reached values less than 0.5/h for 92% of the CHB-MIT patients and less than 1.0/h for 80% of the EPILEPSIAE patients. Compared to recent studies, our lightweight approach does not need any estimation of pre-selected features and demonstrates high performances with promising possibilities for the introduction of such automatic methods in the clinical practice.

Design, Screening, and Testing of Non-Rational Peptide Libraries with Antimicrobial Activity: In Silico and Experimental Approaches.

One of the challenges of modern biotechnology is to find new routes to mitigate the resistance to conventional antibiotics. Antimicrobial peptides (AMPs) are an alternative type of biomolecules, naturally present in a wide variety of organisms, with the capacity to overcome the current microorganism resistance threat. Here, we reviewed our recent efforts to develop a new library of non-rationally produced AMPs that relies on bacterial genome inherent diversity and compared it with rationally designed libraries. Our approach is based on a four-stage workflow process that incorporates the interplay of recent developments in four major emerging technologies: artificial intelligence, molecular dynamics, surface-display in microorganisms, and microfluidics. Implementing this framework is challenging because to obtain reliable results, the in silico algorithms to search for candidate AMPs need to overcome issues of the state-of-the-art approaches that limit the possibilities for multi-space data distribution analyses in extremely large databases. We expect to tackle this challenge by using a recently developed classification algorithm based on deep learning models that rely on convolutional layers and gated recurrent units. This will be complemented by carefully tailored molecular dynamics simulations to elucidate specific interactions with lipid bilayers. Candidate AMPs will be recombinantly-expressed on the surface of microorganisms for further screening via different droplet-based microfluidic-based strategies to identify AMPs with the desired lytic abilities. We believe that the proposed approach opens opportunities for searching and screening bioactive peptides for other applications.

An image J plugin for the high throughput image analysis of in vitro scratch wound healing assays.

In vitro scratch wound healing assay, a simple and low-cost technique that works along with other image analysis tools, is one of the most widely used 2D methods to determine the cellular migration and proliferation in processes such as regeneration and disease. There are open-source programs such as imageJ to analyze images of in vitro scratch wound healing assays, but these tools require manual tuning of various parameters, which is time-consuming and limits image throughput. For that reason, we developed an optimized plugin for imageJ to automatically recognize the wound healing size, correct the average wound width by considering its inclination, and quantify other important parameters such as: area, wound area fraction, average wound width, and width deviation of the wound images obtained from a scratch/ wound healing assay. Our plugin is easy to install and can be used with different operating systems. It can be adapted to analyze both individual images and stacks. Additionally, it allows the analysis of images obtained from bright field, phase contrast, and fluorescence microscopes. In conclusion, this new imageJ plugin is a robust tool to automatically standardize and facilitate quantification of different in vitro wound parameters with high accuracy compared with other tools and manual identification.

Automated lung cancer diagnosis using three-dimensional convolutional neural networks.

Lung cancer is the deadliest cancer worldwide. It has been shown that early detection using low-dose computer tomography (LDCT) scans can reduce deaths caused by this disease. We present a general framework for the detection of lung cancer in chest LDCT images. Our method consists of a nodule detector trained on the LIDC-IDRI dataset followed by a cancer predictor trained on the Kaggle DSB 2017 dataset and evaluated on the IEEE International Symposium on Biomedical Imaging (ISBI) 2018 Lung Nodule Malignancy Prediction test set. Our candidate extraction approach is effective to produce accurate candidates with a recall of 99.6%. In addition, our false positive reduction stage classifies successfully the candidates and increases precision by a factor of 2000. Our cancer predictor obtained a ROC AUC of 0.913 and was ranked 1st place at the ISBI 2018 Lung Nodule Malignancy Prediction challenge. Graphical abstract.

Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks.

We present Convolutional Oriented Boundaries (COB), which produces multiscale oriented contours and region hierarchies starting from generic image classification Convolutional Neural Networks (CNNs). COB is computationally efficient, because it requires a single CNN forward pass for multi-scale contour detection and it uses a novel sparse boundary representation for hierarchical segmentation; it gives a significant leap in performance over the state-of-the-art, and it generalizes very well to unseen categories and datasets. Particularly, we show that learning to estimate not only contour strength but also orientation provides more accurate results. We perform extensive experiments for low-level applications on BSDS, PASCAL Context, PASCAL Segmentation, and NYUD to evaluate boundary detection performance, showing that COB provides state-of-the-art contours and region hierarchies in all datasets. We also evaluate COB on high-level tasks when coupled with multiple pipelines for object proposals, semantic contours, semantic segmentation, and object detection on MS-COCO, SBD, and PASCAL; showing that COB also improves the results for all tasks.

Extra-articular shoulder resections: outcomes of 54 patients.

BACKGROUND: The survival of patients with tumors around the shoulder treated with extra-articular resection, the rates of reconstructions-related complications, and the function of the shoulder cannot be estimated because of limited available data from mainly small published related series and case reports. METHODS: We studied 54 patients with tumors around the shoulder treated with extra-articular shoulder resections and proximal humeral megaprosthetic reconstructions from 1985 to 2012. Mean tumor volume was 549 cm3, and the mean length of the proximal humeral resection was 110 mm. Mean follow-up was 7.8 years (range, 3-21 years). We evaluated the outcomes (survival, metastases, recurrences, and function) and the survival and complications of the reconstruction. RESULTS: Survival of patients with malignant tumors was 47%, 38%, and 35%, at 5, 10, and 20 years, respectively. Rates for metastasis and local recurrence were 60% and 18.5%, respectively. Survival was significantly higher for patients without metastases at diagnosis, tumor volume <549 cm3, and type IV resections. Survival of reconstructions was 56% at 10 years and 48% 20 years. Overall, 19 patients (35.2%) experienced 30 complications (55.5%), the most common being soft tissue failures that required subsequent surgery without, however, implant removal. The mean Musculoskeletal Tumour Society score was 25 points, without any significant difference between the types of extra-articular resections. CONCLUSION: Tumor stage and volume as well as type of resection are important predictors of survival of patients with malignant tumors around the shoulder. Survival of the reconstructions is satisfactory; nevertheless, the complication rate is high. The Musculoskeletal Tumour Society score is similar with respect to the type of resection.

Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation.

We propose a unified approach for bottom-up hierarchical image segmentation and object proposal generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object proposals by exploring efficiently their combinatorial space. We also present Single-scale Combinatorial Grouping (SCG), a faster version of MCG that produces competitive proposals in under five seconds per image. We conduct an extensive and comprehensive empirical validation on the BSDS500, SegVOC12, SBD, and COCO datasets, showing that MCG produces state-of-the-art contours, hierarchical regions, and object proposals.

Object Instance Segmentation and Fine-Grained Localization Using Hypercolumns.

Recognition algorithms based on convolutional networks (CNNs) typically use the output of the last layer as a feature representation. However, the information in this layer may be too coarse spatially to allow precise localization. On the contrary, earlier layers may be precise in localization but will not capture semantics. To get the best of both worlds, we define the hypercolumn at a pixel as the vector of activations of all CNN units above that pixel. Using hypercolumns as pixel descriptors, we show results on three fine-grained localization tasks: simultaneous detection and segmentation, where we improve state-of-the-art from 49.7 mean APr to 62.4, keypoint localization, where we get a 3.3 point boost over a strong regression baseline using CNN features, and part labeling, where we show a 6.6 point gain over a strong baseline.