Reproducible Reporting of the Collection and Evaluation of Annotations for Artificial Intelligence Models.

This work puts forth and demonstrates the utility of a reporting framework for collecting and evaluating annotations of medical images used for training and testing artificial intelligence (AI) models in assisting detection and diagnosis. AI has unique reporting requirements, as shown by the AI extensions to the Consolidated Standards of Reporting Trials (CONSORT) and Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklists and the proposed AI extensions to the Standards for Reporting Diagnostic Accuracy (STARD) and Transparent Reporting of a Multivariable Prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklists. AI for detection and/or diagnostic image analysis requires complete, reproducible, and transparent reporting of the annotations and metadata used in training and testing data sets. In an earlier work by other researchers, an annotation workflow and quality checklist for computational pathology annotations were proposed. In this manuscript, we operationalize this workflow into an evaluable quality checklist that applies to any reader-interpreted medical images, and we demonstrate its use for an annotation effort in digital pathology. We refer to this quality framework as the Collection and Evaluation of Annotations for Reproducible Reporting of Artificial Intelligence (CLEARR-AI).

Prediction of atrial fibrillation from at-home single-lead ECG signals without arrhythmias.

Early identification of atrial fibrillation (AF) can reduce the risk of stroke, heart failure, and other serious cardiovascular outcomes. However, paroxysmal AF may not be detected even after a two-week continuous monitoring period. We developed a model to quantify the risk of near-term AF in a two-week period, based on AF-free ECG intervals of up to 24 h from 459,889 patch-based ambulatory single-lead ECG (modified lead II) recordings of up to 14 days. A deep learning model was used to integrate ECG morphology data with demographic and heart rhythm features toward AF prediction. Observing a 1-day AF-free ECG recording, the model with deep learning features produced the most accurate prediction of near-term AF with an area under the curve AUC = 0.80 (95% confidence interval, CI = 0.79-0.81), significantly improving discrimination compared to demographic metrics alone (AUC 0.67; CI = 0.66-0.68). Our model was able to predict incident AF over a two-week time frame with high discrimination, based on AF-free single-lead ECG recordings of various lengths. Application of the model may enable a digital strategy for improving diagnostic capture of AF by risk stratifying individuals with AF-negative ambulatory monitoring for prolonged or recurrent monitoring, potentially leading to more rapid initiation of treatment.

Initial interactions with the FDA on developing a validation dataset as a medical device development tool.

Quantifying tumor-infiltrating lymphocytes (TILs) in breast cancer tumors is a challenging task for pathologists. With the advent of whole slide imaging that digitizes glass slides, it is possible to apply computational models to quantify TILs for pathologists. Development of computational models requires significant time, expertise, consensus, and investment. To reduce this burden, we are preparing a dataset for developers to validate their models and a proposal to the Medical Device Development Tool (MDDT) program in the Center for Devices and Radiological Health of the U.S. Food and Drug Administration (FDA). If the FDA qualifies the dataset for its submitted context of use, model developers can use it in a regulatory submission within the qualified context of use without additional documentation. Our dataset aims at reducing the regulatory burden placed on developers of models that estimate the density of TILs and will allow head-to-head comparison of multiple computational models on the same data. In this paper, we discuss the MDDT preparation and submission process, including the feedback we received from our initial interactions with the FDA and propose how a qualified MDDT validation dataset could be a mechanism for open, fair, and consistent measures of computational model performance. Our experiences will help the community understand what the FDA considers relevant and appropriate (from the perspective of the submitter), at the early stages of the MDDT submission process, for validating stromal TIL density estimation models and other potential computational models. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland. This article has been contributed to by U.S. Government employees and their work is in the public domain in the USA.

Cost-Effectiveness of AF Screening With 2-Week Patch Monitors: The mSToPS Study.

BACKGROUND: The mSToPS study (mHealth Screening to Prevent Strokes) reported screening older Americans at risk for atrial fibrillation (AF) and stroke using 2-week patch monitors was associated with increased rates of AF diagnosis and anticoagulant prescription within 1 year and improved clinical outcomes at 3 years relative to matched controls. Cost-effectiveness of this AF screening approach has not been explored. METHODS: We conducted a US-based health economic analysis of AF screening using patient-level data from mSToPS. Clinical outcomes, resource use, and costs were obtained through 3 years using claims data. Individual costs, survival, and quality-adjusted life years (QALYs) were projected over a lifetime horizon using regression modeling, US life tables, and external data where needed. Adjustment between groups was performed using propensity score bin bootstrapping. RESULTS: Screening participants (mean age, 74 years, 41% female, median CHA2DS2-VASC score 3) wore on average 1.7 two-week monitors at a mean cost of $614/person. Over 3 years, outpatient visits were more frequent for monitored than unmonitored individuals (difference 190 per 100 patient-years [95% CI, 82-298]), but emergency department visits (-8.3 [95% CI, -12.6 to -4.1]) and hospitalizations (-15.2 [CI, -22 to -8.6]) were less frequent. Total adjusted 3-year costs were slightly higher (mean difference, $1551 [95% CI, -$1047 to $4038]) in the monitoring group. In patient-level projections, the monitoring group had slightly greater quality-adjusted survival (8.81 versus 8.71 QALYs, difference, 0.09 [95% CI, -0.05 to 0.24]) and slightly higher lifetime costs, resulting in an incremental cost-effectiveness ratio of $36 100/QALY gained. With bootstrap resampling, the incremental cost-effectiveness ratio for monitoring was <$50 000/QALY in 64% of study replicates, and <$150 000/QALY in 91%. CONCLUSIONS: Using lifetime projections derived from the mSToPS study, we found that AF screening using 2-week patch monitors in older Americans was associated with high economic value. Confirmation of these uncertain findings in a randomized trial is warranted. REGISTRATION: URL: https://www.clinicaltrials.gov; Unique identifier: NCT02506244.

Pilot study to evaluate tools to collect pathologist annotations for validating machine learning algorithms.

Purpose: Validation of artificial intelligence (AI) algorithms in digital pathology with a reference standard is necessary before widespread clinical use, but few examples focus on creating a reference standard based on pathologist annotations. This work assesses the results of a pilot study that collects density estimates of stromal tumor-infiltrating lymphocytes (sTILs) in breast cancer biopsy specimens. This work will inform the creation of a validation dataset for the evaluation of AI algorithms fit for a regulatory purpose. Approach: Collaborators and crowdsourced pathologists contributed glass slides, digital images, and annotations. Here, "annotations" refer to any marks, segmentations, measurements, or labels a pathologist adds to a report, image, region of interest (ROI), or biological feature. Pathologists estimated sTILs density in 640 ROIs from hematoxylin and eosin stained slides of 64 patients via two modalities: an optical light microscope and two digital image viewing platforms. Results: The pilot study generated 7373 sTILs density estimates from 29 pathologists. Analysis of annotations found the variability of density estimates per ROI increases with the mean; the root mean square differences were 4.46, 14.25, and 26.25 as the mean density ranged from 0% to 10%, 11% to 40%, and 41% to 100%, respectively. The pilot study informs three areas of improvement for future work: technical workflows, annotation platforms, and agreement analysis methods. Upgrades to the workflows and platforms will improve operability and increase annotation speed and consistency. Conclusions: Exploratory data analysis demonstrates the need to develop new statistical approaches for agreement. The pilot study dataset and analysis methods are publicly available to allow community feedback. The development and results of the validation dataset will be publicly available to serve as an instructive tool that can be replicated by developers and researchers.

A Pathologist-Annotated Dataset for Validating Artificial Intelligence: A Project Description and Pilot Study.

PURPOSE: Validating artificial intelligence algorithms for clinical use in medical images is a challenging endeavor due to a lack of standard reference data (ground truth). This topic typically occupies a small portion of the discussion in research papers since most of the efforts are focused on developing novel algorithms. In this work, we present a collaboration to create a validation dataset of pathologist annotations for algorithms that process whole slide images. We focus on data collection and evaluation of algorithm performance in the context of estimating the density of stromal tumor-infiltrating lymphocytes (sTILs) in breast cancer. METHODS: We digitized 64 glass slides of hematoxylin- and eosin-stained invasive ductal carcinoma core biopsies prepared at a single clinical site. A collaborating pathologist selected 10 regions of interest (ROIs) per slide for evaluation. We created training materials and workflows to crowdsource pathologist image annotations on two modes: an optical microscope and two digital platforms. The microscope platform allows the same ROIs to be evaluated in both modes. The workflows collect the ROI type, a decision on whether the ROI is appropriate for estimating the density of sTILs, and if appropriate, the sTIL density value for that ROI. RESULTS: In total, 19 pathologists made 1645 ROI evaluations during a data collection event and the following 2 weeks. The pilot study yielded an abundant number of cases with nominal sTIL infiltration. Furthermore, we found that the sTIL densities are correlated within a case, and there is notable pathologist variability. Consequently, we outline plans to improve our ROI and case sampling methods. We also outline statistical methods to account for ROI correlations within a case and pathologist variability when validating an algorithm. CONCLUSION: We have built workflows for efficient data collection and tested them in a pilot study. As we prepare for pivotal studies, we will investigate methods to use the dataset as an external validation tool for algorithms. We will also consider what it will take for the dataset to be fit for a regulatory purpose: study size, patient population, and pathologist training and qualifications. To this end, we will elicit feedback from the Food and Drug Administration via the Medical Device Development Tool program and from the broader digital pathology and AI community. Ultimately, we intend to share the dataset, statistical methods, and lessons learned.

Clinical and Analytical Validation of a Novel Urine-Based Test for the Detection of Allograft Rejection in Renal Transplant Patients.

In this clinical validation study, we developed and validated a urinary Q-Score generated from the quantitative test QSant, formerly known as QiSant, for the detection of biopsy-confirmed acute rejection in kidney transplants. Using a cohort of 223 distinct urine samples collected from three independent sites and from both adult and pediatric renal transplant patients, we examined the diagnostic utility of the urinary Q-Score for detection of acute rejection in renal allografts. Statistical models based upon the measurements of the six QSant biomarkers (cell-free DNA, methylated-cell-free DNA, clusterin, CXCL10, creatinine, and total protein) generated a renal transplant Q-Score that reliably differentiated stable allografts from acute rejections in both adult and pediatric renal transplant patients. The composite Q-Score was able to detect both T cell-mediated rejection and antibody-mediated rejection patients and differentiate them from stable non-rejecting patients with a receiver-operator characteristic curve area under the curve of 99.8% and an accuracy of 98.2%. Q-Scores < 32 indicated the absence of active rejection and Q-Scores ≥ 32 indicated an increased risk of active rejection. At the Q-Score cutoff of 32, the overall sensitivity was 95.8% and specificity was 99.3%. At a prevalence of 25%, positive and negative predictive values for active rejection were 98.0% and 98.6%, respectively. The Q-Score also detected subclinical rejection in patients without an elevated serum creatinine level but identified by a protocol biopsy. This study confirms that QSant is an accurate and quantitative measurement suitable for routine monitoring of renal allograft status.

Analytical Validation of Relative Average Telomere Length Measurement in a Clinical Laboratory Environment.

BACKGROUND: Average telomere length in whole blood has become a biomarker of aging, disease, and mortality risk across a broad range of clinical conditions. The most common method of telomere length measurement for large patient sample sets is based on quantitative PCR (qPCR). For laboratory-developed tests to be performed on clinical samples, they must undergo a rigorous analytical validation, currently regulated under CLIA. METHODS: Whole blood samples from 40 donors were used in the analytical validation of methods for relative average telomere length (rATL) measurement. Three technical replicate DNA samples were extracted from each whole blood sample and placed in three independent wells on a sample plate. Each of these sample plates was assayed 12 times during the validation process. The study was conducted over a 20-day period, once in the morning and once in the evening, using 3 different operators. RESULTS: Our process of rATL measurement beginning with DNA extraction followed by qPCR-based assay resulted in repeatability and reproducibility CV of <5% and amplification efficiencies near 100%. The validated assay was used to establish a reference interval derived from 2 cohorts of individuals: (a) San Francisco Bay area (n = 504) and (b) a US cross-sectional, demographic population (n = 357). CONCLUSIONS: We present advances in the establishment of a highly reproducible analytically validated process for determining rATLs in a CLIA laboratory environment.

Cost effectiveness analysis of a next generation risk assessment score for cardiovascular disease.

OBJECTIVES: The goal of this study is to determine the cost-effectiveness of MIRISK VP, a next generation coronary heart disease risk assessment score, in correctly reclassifying and appropriately treating asymptomatic, intermediate risk patients. STUDY DESIGN: A Markov model was employed with simulated subjects based on the Multi-Ethnic Study of Atherosclerosis (MESA). This study evaluated three treatment strategies: (i) practice at MESA enrollment, (ii) current guidelines, and (iii) MIRISK VP in MESA. METHODS: The model assessed patient healthcare costs and outcomes, expressed in terms of life years and quality-adjusted life years (QALYs), over the lifetime of the cohort from the provider and payer perspective. A total of 50,000 hypothetical individuals were used in the model. A sensitivity analysis was conducted (based on the various input parameters) for the entire cohort and also for individuals aged 65 and older. RESULTS: Guiding treatment with MIRISK VP leads to the highest net monetary benefits when compared to the 'Practice at MESA Enrollment' or to the 'Current Guidelines' strategies. MIRISK VP resulted in a lower mortality rate from any CHD event and a modest increase in QALY of 0.12-0.17 years compared to the other two approaches. LIMITATIONS: This study has limitations of not comparing performance against strategies other than the FRS, the results are simulated as with all models, the model does not incorporate indirect healthcare costs, and the impact of patient or physician behaviors on outcomes were not taken into account. CONCLUSIONS: MIRISK VP has the potential to improve patient outcomes compared to the alternative strategies. It is marginally more costly than both the 'Practice at MESA Enrollment' and the 'Current Guidelines' strategies, but it provides increased effectiveness, which leads to positive net monetary benefits over either strategy.

Clinical utility of a novel coronary heart disease risk-assessment test to further classify intermediate-risk patients.

BACKGROUND: Current coronary heart disease (CHD) risk assessments inadequately assess intermediate-risk patients, leaving many undertreated and vulnerable to heart attacks. A novel CHD risk-assessment (CHDRA) tool was developed for intermediate-risk stratification using biomarkers and established risk factors to significantly improve CHD risk discrimination. HYPOTHESIS: Physicians will change their treatment plan in response to more information about a patient's CHD risk level provided by the CHDRA test. METHODS: A Web-based survey of cardiology, internal medicine, family practice, and obstetrics/gynecology physicians (n = 206) was conducted to assess the CHDRA clinical impact. Each physician was shown 3 clinical vignettes representing community-based cohort participants randomly selected from 8 total vignettes. For each, the physicians assessed the individual's CHD risk and selected preferred therapies based on the individual's comorbidities, physical examination, and laboratory results. The individual's CHDRA score was then provided and the physicians were queried for changes to their initial treatment plans. RESULTS: After obtaining the CHDRA result, 70% of the physician responses indicated a change to the patient's treatment plan. The revised lipid-management plans agreed more often (74.6% of the time) with the current Adult Treatment Panel III guidelines than did the original plans (57.6% of the time). Most physicians (71.3%) agreed with the statement that the CHDRA result provided information that would impact their current treatment decisions. CONCLUSIONS: The CHDRA test provided additional information to which physicians responded by more often applying appropriate therapy and actions aligned with guidelines, thus demonstrating the clinical utility of the test.

Analytical performance validation of a coronary heart disease risk assessment multi-analyte proteomic test.

BACKGROUND: Coronary heart disease (CHD) remains prevalent despite efforts to improve CHD risk assessment. The authors developed a multi-analyte immunoassay-based CHD risk assessment (CHDRA) algorithm, clinically validated in a multicenter study, to improve CHDRA in intermediate risk individuals. OBJECTIVE: Clinical laboratory validation of the CHDRA biomarker assays' analytical performance. METHODS: Multiplexed immunoassay panels developed for the seven CHDRA assays were evaluated with donor sera in a clinical laboratory. Specificity, sensitivity, interfering substances and reproducibility of the CHDRA assays, along with the effects of pre-analytical specimen processing, were evaluated. RESULTS: Analytical measurements of the CHDRA panel proteins (CTACK, Eotaxin, Fas Ligand, HGF, IL-16, MCP-3 and sFas) exhibited acceptable accuracy (80 - 120%), cross-reactivity (< 1%), interference (< 30% at high concentrations of bilirubin, lipids, hemoglobin and HAMA), sensitivity and reproducibility (< 20% CV across multiple runs, operators and instruments). Recoveries from donor sera subjected to typical clinical laboratory pre-analytical conditions were within 80 - 120%. The pre-analytical variables did not substantively impact the CHDRA scores. CONCLUSIONS: The CHDRA panel analytical validation in a clinical laboratory meets or exceeds the specifications established during the clinical utility studies. Risk score reproducibility across multiple test scenarios suggests the assays are not susceptible to clinical laboratory pre-analytical and analytical variation.

Circulating angiopoietins-1 and -2, angiopoietin receptor Tie-2 and vascular endothelial growth factor-A as biomarkers of acute myocardial infarction: a prospective nested case-control study.

BACKGROUND: Angiogenesis is up-regulated in myocardial ischemia. However, limited data exist assessing the value of circulating angiogenic biomarkers in predicting future incidence of acute myocardial infarction (AMI). Our aim was to examine the association between circulating levels of markers of angiogenesis with risk of incident acute myocardial infarction (AMI) in men and women. METHODS: We performed a case-control study (nested within a large cohort of persons receiving care within Kaiser Permanente of Northern California) including 695 AMI cases and 690 controls individually matched on age, gender and race/ethnicity. RESULTS: Median [inter-quartile range] serum concentrations of vascular endothelial growth factor-A (VEGF-A; 260 [252] vs. 235 [224] pg/mL; p = 0.01) and angiopoietin-2 (Ang-2; 1.18 [0.66] vs. 1.05 [0.58] ng/mL; p < 0.0001) were significantly higher in AMI cases than in controls. By contrast, endothelium-specific receptor tyrosine kinase (Tie-2; 14.2 [3.7] vs. 14.0 [3.1] ng/mL; p = 0.07) and angiopoietin-1 levels (Ang-1; 33.1 [13.6] vs. 32.5 [12.7] ng/mL; p = 0.52) did not differ significantly by case-control status. After adjustment for educational attainment, hypertension, diabetes, smoking, alcohol consumption, body mass index, LDL-C, HDL-C, triglycerides and C-reactive protein, each increment of 1 unit of Ang-2 as a Z score was associated with 1.17-fold (95 percent confidence interval, 1.02 to 1.35) increased odds of AMI, and the upper quartile of Ang-2, relative to the lowest quartile, was associated with 1.63-fold (95 percent confidence interval, 1.09 to 2.45) increased odds of AMI. CONCLUSIONS: Our data support a role of Ang-2 as a biomarker of incident AMI independent of traditional risk factors.

Circulating chemokines accurately identify individuals with clinically significant atherosclerotic heart disease.

Serum inflammatory markers correlate with outcome and response to therapy in subjects with cardiovascular disease. However, current individual markers lack specificity for the diagnosis of coronary artery disease (CAD). We hypothesize that a multimarker proteomic approach measuring serum levels of vascular derived inflammatory biomarkers could reveal a "signature of disease" that can serve as a highly accurate method to assess for the presence of coronary atherosclerosis. We simultaneously measured serum levels of seven chemokines [CXCL10 (IP-10), CCL11 (eotaxin), CCL3 (MIP1 alpha), CCL2 (MCP1), CCL8 (MCP2), CCL7 (MCP3), and CCL13 (MCP4)] in 48 subjects with clinically significant CAD ("cases") and 44 controls from the ADVANCE Study. We applied three classification algorithms to identify the combination of variables that would best predict case-control status and assessed the diagnostic performance of these models with receiver operating characteristic (ROC) curves. The serum levels of six chemokines were significantly higher in cases compared with controls (P < 0.05). All three classification algorithms entered three chemokines in their final model, and only logistic regression selected clinical variables. Logistic regression produced the highest ROC of the three algorithms (AUC = 0.95; SE = 0.03), which was markedly better than the AUC for the logistic regression model of traditional risk factors of CAD without (AUC = 0.67; SE = 0.06) or with CRP (AUC = 0.68; SE = 0.06). A combination of serum levels of multiple chemokines identifies subjects with clinically significant atherosclerotic heart disease with a very high degree of accuracy. These results need to be replicated in larger cross-sectional studies and their prognostic value explored.

Characterization of compound mechanisms and secondary activities by BioMAP analysis.

INTRODUCTION: Unexpected drug activities account for many of the failures of new chemical entities in clinical trials. These activities can be target-dependent, resulting from feedback mechanisms downstream of the primary target, or they can occur as a result of unanticipated secondary target(s). Methods that would provide rapid and efficient characterization of compounds with respect to a broad range of biological pathways and mechanisms relevant to human disease have the potential to improve preclinical and clinical success rates. METHODS: BioMAP assays containing primary human cells (endothelial cells and co-cultures with peripheral blood leukocytes) were stimulated in complex formats (specific combinations of inflammatory mediators) for 24 h in the presence or absence of test agents (drugs, experimental compounds, etc.). The levels of selected protein readouts (adhesion receptors, cytokines, enzymes, etc.) were measured and activity profiles (normalized data sets comprising BioMAP profiles) were generated for each test agent. The resulting profiles were compared by statistical methods to identify similarities and mechanistic insights. RESULTS: Compounds with known mechanisms including inhibitors of histamine H1 receptor, angiotensin converting enzyme, IkappaB kinase-2, beta2 adrenergic receptor and others were shown to generate reproducible and distinguishable BioMAP activity profiles. Similarities were observed between compounds targeting components within the same signal transduction pathway (e.g. NFkappaB), and also between compounds that share secondary targets (e.g. ibuprofen and FMOC-L-leucine, a PPARgamma agonist). DISCUSSION: Complex primary cell-based assays can be applied for detecting and distinguishing unexpected activities that may be of relevance to drug action in vivo. The ability to rapidly test compounds prior to animal or clinical studies may reduce the number of compounds that unexpectedly fail in preclinical or clinical studies.

Approaches to the analysis of cell signaling networks and their application in drug discovery.

The ability to predict the safety and efficacy of novel drugs prior to clinical testing is a key goal in pharmaceutical drug discovery. Gaining a mechanistic understanding of the complex cell signaling networks (CSNs) underlying disease processes promises to help reduce the number of clinical failures by identifying points of intervention as well as redundancies and feedback mechanisms that contribute to toxicities, lack of efficacy and unexpected biological activities. Experimental and computational approaches to analyzing and modeling CSNs are currently being validated using simple organisms and cell lines. In vitro cell systems of sufficient complexity to resemble human disease physiology, but which are also amenable to chemical and genetic perturbations on a large scale, are now required for deciphering the signaling networks operating in human disease. In this review, experimental and computational methods for modeling complex CSNs and the applications of these approaches to pharmaceutical drug discovery are discussed.

Rapid structure-activity and selectivity analysis of kinase inhibitors by BioMAP analysis in complex human primary cell-based models.

Rapid, quantitative methods for characterizing the biological activities of kinase inhibitors in complex human cell systems could allow the biological consequences of differential target selectivity to be monitored early in development, improving the selection of drug candidates. We have previously shown that Biologically Multiplexed Activity Profiling (BioMAP) permits rapid characterization of drug function based on statistical analysis of protein expression data sets from complex primary human cellbased models of disease biology. Here, using four such model systems containing primary human endothelial cells and peripheral blood mononuclear cells in which multiple signaling pathways relevant to inflammation and immune responses are simultaneously activated, we demonstrate that BioMAP analysis can detect and distinguish a wide range of inhibitors directed against different kinase targets. Using a panel of p38 mitogen-activated protein kinase antagonists as a test set, we show further that related compounds can be distinguished by unique features of the biological responses they induce in complex systems, and can be classified according to their induction of shared (on-target) and secondary activities. Statistical comparisons of quantitative BioMAP profiles and analysis of profile features allow correlation of induced biological effects with chemical structure and mapping of biological responses to chemical series or substituents on a common scaffold. Integration of automated BioMAP analysis for prioritization of hits and for structure-activity relationship studies may improve and accelerate the design and selection of optimal therapeutic candidates.

An integrative biology approach for analysis of drug action in models of human vascular inflammation.

Unexpected drug activities discovered during clinical testing establish the need for better characterization of compounds in human disease-relevant conditions early in the discovery process. Here, we describe an approach to characterize drug function based on statistical analysis of protein expression datasets from multiple primary human cell-based models of inflammatory disease. This approach, termed Biologically Multiplexed Activity Profiling (BioMAP), provides rapid characterization of drug function, including mechanism of action, secondary or off-target activities, and insights into clinical phenomena. Using three model systems containing primary human endothelial cells and peripheral blood mononuclear cells in different environments relevant to vascular inflammation and immune activation, we show that BioMAP profiles detect and discriminate multiple functional drug classes, including glucocorticoids; TNF-alpha antagonists; and inhibitors of HMG-CoA reductase, calcineurin, IMPDH, PDE4, PI-3 kinase, hsp90, and p38 MAPK, among others. The ability of cholesterol lowering HMG-CoA reductase inhibitors (statins) to improve outcomes in rheumatic disease patients correlates with the activities of these compounds in our BioMAP assays. In addition, the activity profiles identified for the immunosuppressants mycophenolic acid, cyclosporin A, and FK-506 provide a potential explanation for a reduced incidence of posttransplant cardiovascular disease in patients receiving mycophenolic acid. BioMAP profiling can allow integration of meaningful human biology into drug development programs.

Method for analyzing signaling networks in complex cellular systems.

Now that the human genome has been sequenced, the challenge of assigning function to human genes has become acute. Existing approaches using microarrays or proteomics frequently generate very large volumes of data not directly related to biological function, making interpretation difficult. Here, we describe a technique for integrative systems biology in which: (i) primary cells are cultured under biologically meaningful conditions; (ii) a limited number of biologically meaningful readouts are measured; and (iii) the results obtained under several different conditions are combined for analysis. Studies of human endothelial cells overexpressing different signaling molecules under multiple inflammatory conditions show that this system can capture a remarkable range of functions by a relatively small number of simple measurements. In particular, measurement of seven different protein levels by ELISA under four different conditions is capable of reconstructing pathway associations of 25 different proteins representing four known signaling pathways, implicating additional participants in the NF-kappaBorRAS/mitogen-activated protein kinase pathways and defining additional interactions between these pathways.