Predicting Productivity Returns on Investment: Thirty Years of Peer Review, Grant Funding, and Publication of Highly Cited Papers at the National Heart, Lung, and Blood Institute.

There are conflicting data about the ability of peer review percentile rankings to predict grant productivity, as measured through publications and citations. To understand the nature of these apparent conflicting findings, we analyzed bibliometric outcomes of 6873 de novo cardiovascular R01 grants funded by the National Heart, Lung, and Blood Institute (NHLBI) between 1980 and 2011. Our outcomes focus on top-10% articles, meaning articles that were cited more often than 90% of other articles on the same topic, of the same type (eg, article, editorial), and published in the same year. The 6873 grants yielded 62 468 articles, of which 13 507 (or 22%) were top-10% articles. There was a modest association between better grant percentile ranking and number of top-10% articles. However, discrimination was poor (area under receiver operating characteristic curve [ROC], 0.52; 95% confidence interval, 0.51-0.53). Furthermore, better percentile ranking was also associated with higher annual and total inflation-adjusted grant budgets. There was no association between grant percentile ranking and grant outcome as assessed by number of top-10% articles per $million spent. Hence, the seemingly conflicting findings on peer review percentile ranking of grants and subsequent productivity largely reflect differing questions and outcomes. Taken together, these findings raise questions about how best National Institutes of Health (NIH) should use peer review assessments to make complex funding decisions.

Citation impact of NHLBI R01 grants funded through the American Recovery and Reinvestment Act as compared to R01 grants funded through a standard payline.

RATIONALE: The American Recovery and Reinvestment Act (ARRA) allowed National Heart, Lung, and Blood Institute to fund R01 grants that fared less well on peer review than those funded by meeting a payline threshold. It is not clear whether the sudden availability of additional funding enabled research of similar or lesser citation impact than already funded work. OBJECTIVE: To compare the citation impact of ARRA-funded de novo National Heart, Lung, and Blood Institute R01 grants with concurrent de novo National Heart, Lung, and Blood Institute R01 grants funded by standard payline mechanisms. METHODS AND RESULTS: We identified de novo (type 1) R01 grants funded by National Heart, Lung, and Blood Institute in fiscal year 2009: these included 458 funded by meeting Institute's published payline and 165 funded only because of ARRA funding. Compared with payline grants, ARRA grants received fewer total funds (median values, $1.03 versus $1.87 million; P<0.001) for a shorter duration (median values including no-cost extensions, 3.0 versus 4.9 years; P<0.001). Through May 2014, the payline R01 grants generated 3895 publications, whereas the ARRA R01 grants generated 996. Using the InCites database from Thomson-Reuters, we calculated a normalized citation impact for each grant by weighting each article for the number of citations it received normalizing for subject, article type, and year of publication. The ARRA R01 grants had a similar normalized citation impact per $1 million spent as the payline grants (median values [interquartile range], 2.15 [0.73-4.68] versus 2.03 [0.75-4.10]; P=0.61). The similar impact of the ARRA grants persisted even after accounting for potential confounders. CONCLUSIONS: Despite shorter durations and lower budgets, ARRA R01 grants had comparable citation outcomes per $million spent to that of contemporaneously funded payline R01 grants.

Percentile ranking and citation impact of a large cohort of National Heart, Lung, and Blood Institute-funded cardiovascular R01 grants.

RATIONALE: Funding decisions for cardiovascular R01 grant applications at the National Heart, Lung, and Blood Institute (NHLBI) largely hinge on percentile rankings. It is not known whether this approach enables the highest impact science. OBJECTIVE: Our aim was to conduct an observational analysis of percentile rankings and bibliometric outcomes for a contemporary set of funded NHLBI cardiovascular R01 grants. METHODS AND RESULTS: We identified 1492 investigator-initiated de novo R01 grant applications that were funded between 2001 and 2008 and followed their progress for linked publications and citations to those publications. Our coprimary end points were citations received per million dollars of funding, citations obtained <2 years of publication, and 2-year citations for each grant's maximally cited paper. In 7654 grant-years of funding that generated $3004 million of total National Institutes of Health awards, the portfolio yielded 16 793 publications that appeared between 2001 and 2012 (median per grant, 8; 25th and 75th percentiles, 4 and 14; range, 0-123), which received 2 224 255 citations (median per grant, 1048; 25th and 75th percentiles, 492 and 1932; range, 0-16 295). We found no association between percentile rankings and citation metrics; the absence of association persisted even after accounting for calendar time, grant duration, number of grants acknowledged per paper, number of authors per paper, early investigator status, human versus nonhuman focus, and institutional funding. An exploratory machine learning analysis suggested that grants with the best percentile rankings did yield more maximally cited papers. CONCLUSIONS: In a large cohort of NHLBI-funded cardiovascular R01 grants, we were unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics.

Prior publication productivity, grant percentile ranking, and topic-normalized citation impact of NHLBI cardiovascular R01 grants.

RATIONALE: We previously demonstrated absence of association between peer-review-derived percentile ranking and raw citation impact in a large cohort of National Heart, Lung, and Blood Institute cardiovascular R01 grants, but we did not consider pregrant investigator publication productivity. We also did not normalize citation counts for scientific field, type of article, and year of publication. OBJECTIVE: To determine whether measures of investigator prior productivity predict a grant's subsequent scientific impact as measured by normalized citation metrics. METHODS AND RESULTS: We identified 1492 investigator-initiated de novo National Heart, Lung, and Blood Institute R01 grant applications funded between 2001 and 2008 and linked the publications from these grants to their InCites (Thompson Reuters) citation record. InCites provides a normalized citation count for each publication stratifying by year of publication, type of publication, and field of science. The coprimary end points for this analysis were the normalized citation impact per million dollars allocated and the number of publications per grant that has normalized citation rate in the top decile per million dollars allocated (top 10% articles). Prior productivity measures included the number of National Heart, Lung, and Blood Institute-supported publications each principal investigator published in the 5 years before grant review and the corresponding prior normalized citation impact score. After accounting for potential confounders, there was no association between peer-review percentile ranking and bibliometric end points (all adjusted P>0.5). However, prior productivity was predictive (P<0.0001). CONCLUSIONS: Even after normalizing citation counts, we confirmed a lack of association between peer-review grant percentile ranking and grant citation impact. However, prior investigator publication productivity was predictive of grant-specific citation impact.

Standardization and clinical applications of retinal imaging biomarkers for cardiovascular disease: a Roadmap from an NHLBI workshop.

The accessibility of the retina with the use of non-invasive and relatively low-cost ophthalmic imaging techniques and analytics provides a unique opportunity to improve the detection, diagnosis and monitoring of systemic diseases. The National Heart, Lung, and Blood Institute conducted a workshop in October 2022 to examine this concept. On the basis of the discussions at that workshop, this Roadmap describes current knowledge gaps and new research opportunities to evaluate the relationships between the eye (in particular, retinal biomarkers) and the risk of cardiovascular diseases, including coronary artery disease, heart failure, stroke, hypertension and vascular dementia. Identified gaps include the need to simplify and standardize the capture of high-quality images of the eye by non-ophthalmic health workers and to conduct longitudinal studies using multidisciplinary networks of diverse at-risk populations with improved implementation and methods to protect participant and dataset privacy. Other gaps include improving the measurement of structural and functional retinal biomarkers, determining the relationship between microvascular and macrovascular risk factors, improving multimodal imaging 'pipelines', and integrating advanced imaging with 'omics', lifestyle factors, primary care data and radiological reports, by using artificial intelligence technology to improve the identification of individual-level risk. Future research on retinal microvascular disease and retinal biomarkers might additionally provide insights into the temporal development of microvascular disease across other systemic vascular beds.

Meeting the Challenges of Myocarditis: New Opportunities for Prevention, Detection, and Intervention-A Report from the 2021 National Heart, Lung, and Blood Institute Workshop.

The National Heart, Lung, and Blood Institute (NHLBI) convened a workshop of international experts to discuss new research opportunities for the prevention, detection, and intervention of myocarditis in May 2021. These experts reviewed the current state of science and identified key gaps and opportunities in basic, diagnostic, translational, and therapeutic frontiers to guide future research in myocarditis. In addition to addressing community-acquired myocarditis, the workshop also focused on emerging causes of myocarditis including immune checkpoint inhibitors and SARS-CoV-2 related myocardial injuries and considered the use of systems biology and artificial intelligence methodologies to define workflows to identify novel mechanisms of disease and new therapeutic targets. A new priority is the investigation of the relationship between social determinants of health (SDoH), including race and economic status, and inflammatory response and outcomes in myocarditis. The result is a proposal for the reclassification of myocarditis that integrates the latest knowledge of immunological pathogenesis to refine estimates of prognosis and target pathway-specific treatments.

Portfolio Analysis of Research Grants in Data Science Funded by the National Heart, Lung, and Blood Institute.

Leveraging emerging opportunities in data science to open new frontiers in heart, lung, blood, and sleep research is one of the major strategic objectives of the National Heart, Lung, and Blood Institute (NHLBI), one of the 27 Institutes/Centers within the National Institutes of Health (NIH). To assess NHLBI's recent funding of research grants in data science and to identify its relative areas of focus within data science, a portfolio analysis from fiscal year 2008 to fiscal year 2017 was performed. In this portfolio analysis, an efficient and reliable methodology was used to identify data science research grants by utilizing several NIH databases and search technologies (iSearch, Query View Reporting system, and IN-SPIRE [Pacific Northwest National Laboratory, Richland, WA]). Six hundred thirty data science-focused extramural research grants supported by NHLBI were identified using keyword searches based primarily on NIH's working definitions of bioinformatics and computational biology. Further analysis characterized the distribution of these grants among the heart, lung, blood, and sleep disease areas as well as the subtypes of data science projects funded by NHLBI. Information was also collected for data science research grants funded by other NIH institutes/centers using the same search and analysis methodology. The funding comparison among different NIH institutes/centers highlighted relative data science areas of emphasis and further identified opportunities for potential data science areas in which NHLBI could foster research advances.

First-In-Human Study Demonstrating Tumor-Angiogenesis by PET/CT Imaging with (68)Ga-NODAGA-THERANOST, a High-Affinity Peptidomimetic for αvß3 Integrin Receptor Targeting.

UNLABELLED: (68)Ga-NODAGA-THERANOST™ is an αvß3 integrin antagonist and the first radiolabeled peptidomimetic to reach clinical development for targeting integrin receptors. In this first-in-human study, the feasibility of integrin receptor peptidomimetic positron emission tomography/computed tomography (PET/CT) imaging was confirmed in patients with non-small-cell lung cancer and breast cancer. METHODS: Patients underwent PET/CT imaging with (68)Ga NODAGA-THERANOST. PET images were analyzed qualitatively and quantitatively and compared to 2-deoxy-2-((18)F) fluoro-d-glucose ((18)F-FDG) findings. Images were obtained 60 minutes postinjection of 300-500 MBq of (68)Ga-NODAGA-THERANOST. RESULTS: (68)Ga-NODAGA-THERANOST revealed high tumor-to-background ratios (SUVmax=4.8) and uptake at neoangiogenesis sites. Reconstructed fused images distinguished cancers with high malignancy potential and enabled enhanced bone metastasis detection. (18)F-FDG-positive lung and lymph node metastases did not show uptake, indicating the absence of neovascularization. CONCLUSIONS: (68)Ga-NODAGA-THERANOST was found to be safe and effective, exhibiting in this study rapid blood clearance, stability, rapid renal excretion, favorable biodistribution and PK/PD, low irradiation burden (µSv/MBq/µg), and convenient radiolabeling. This radioligand might enable theranostics, that is, a combination of diagnostics followed by the appropriate therapeutics, namely antiangiogenic therapy, image-guided presurgical assessment, treatment response evaluation, prediction of pathologic response, neoadjuvant-peptidomimetic-radiochemotherapy, and personalized medicine strategies. Further clinical trials evaluating (68)Ga-NODAGA-THERANOST are warranted.

High-affinity alphavbeta3 integrin targeted optical probe as a new imaging biomarker for early atherosclerosis: initial studies in Watanabe rabbits.

PURPOSE: A newly developed synthetic alpha v beta 3 integrin targeted optical probe (ITOP) has been demonstrated to target cancer cells, in vivo. Compared to the commercially available cyclic peptide c[RGDfv], this optical probe has at least 20 times better binding affinity for the alpha v beta 3 receptor. The present in vitro study was designed to investigate the possibility of detecting early atherosclerotic plaque by using this ITOP. PROCEDURES: Experiments were performed on five Watanabe heritable hyperlipidemic rabbits and two New Zealand White rabbits for control. Our ITOP was used for detecting the presence of alpha v beta 3 receptors in vitro. RESULTS: Segments of plaque accumulation from two distinct regions of ascending and descending aortas were labeled in Watanabe rabbits. The signal was found principally in the adventitia and proximal intima of the aortic vessel, corresponding directly to the expression of integrin alpha v beta 3 as determined by antibody assay. Moreover, there was a close association between the level of labeling with the alpha v beta 3 targeted probe and the thickness of the adventitia. CONCLUSIONS: This high-affinity ITOP identifies the site and extent of alpha v beta 3 expression and correlates with adventitial thickness. Recent evidence associates alpha v beta 3 expression with the inflammatory process in early vulnerable plaque, making this compound a promising potential biomarker for early atherosclerotic disease.

Tumor angiogenic endothelial cell targeting by a novel integrin-targeted nanoparticle.

Angiogenesis is an important process in cancer growth and metastasis. During the tumor angiogenic process, endothelial cells express various cell surface receptors which can be utilized for molecular imaging and targeted drug delivery. One such protein receptor of interest is the integrin alphav beta3. Our group is involved in the development of molecular imaging probes and drug delivery systems targeting alphav beta3. Based on extensive lead optimization study with the integrin antagonist compounds, we have developed a new generation of integrin alphav beta3 compound (IA) which has superior binding affinity to alphav beta3. Utilizing this IA as a targeting agent, we have developed a novel integrin-targeted nanoparticle (ITNP) system for targ alphav beta3 was observed. These ITNPs also were rapidly taken up by cells that express alphav beta3. The ITNPs accumulated in the angiogenic vessels, after systemic administration in a murine squamous cell carcinoma model. This novel intergrin targeted ITNP platform will likely have an application in targeted delivery of drugs and genes in vivo and can also be used for molecular imaging.