Flagging unusual clusters based on linear mixed models using weighted and self-calibrated predictors.

Statistical models incorporating cluster-specific intercepts are commonly used in hierarchical settings, for example, observations clustered within patients or patients clustered within hospitals. Predicted values of these intercepts are often used to identify or "flag" extreme or outlying clusters, such as poorly performing hospitals or patients with rapid declines in their health. We consider a variety of flagging rules, assessing different predictors, and using different accuracy measures. Using theoretical calculations and comprehensive numerical evaluation, we show that previously proposed rules based on the 2 most commonly used predictors, the usual best linear unbiased predictor and fixed effects predictor, perform extremely poorly: the incorrect flagging rates are either unacceptably high (approaching 0.5 in the limit) or overly conservative (eg, much <0.05 for reasonable parameter values, leading to very low correct flagging rates). We develop novel methods for flagging extreme clusters that can control the incorrect flagging rates, including very simple-to-use versions that we call "self-calibrated." The new methods have substantially higher correct flagging rates than previously proposed methods for flagging extreme values, while controlling the incorrect flagging rates. We illustrate their application using data on length of stay in pediatric hospitals for children admitted for asthma diagnoses.

Analysis of longitudinal data from outcome-dependent visit processes: Failure of proposed methods in realistic settings and potential improvements.

The timing and frequency of the measurement of longitudinal outcomes in databases may be associated with the value of the outcome. Such visit processes are termed outcome dependent, and previous work showed that conducting standard analyses that ignore outcome-dependent visit times can produce highly biased estimates of the associations of covariates with outcomes. The literature contains several classes of approaches to analyze longitudinal data subject to outcome-dependent visit times, and all of these are based on simplifying assumptions about the visit process. Based on extensive discussions with subject matter investigators, we identified common characteristics of outcome-dependent visit processes that allowed us to evaluate the performance of existing methods in settings with more realistic visit processes than have been previously investigated. This paper uses the analysis of data from a study of kidney function, theory, and simulation studies to examine a range of settings that vary from those where all visits have a low degree of missingness and outcome dependence (which we call "regular" visits) to those where all visits have a high degree of missingness and outcome dependence (which we call "irregular" visits). Our results show that while all the approaches we studied can yield biased estimates of some covariate effects, other covariate effects can be estimated with little bias. In particular, mixed effects models fit by maximum likelihood yielded little bias in estimates of the effects of covariates not associated with the random effects and small bias in estimates of the effects of covariates associated with the random effects. Other approaches produced estimates with greater bias. Our results also show that the presence of some regular visits in the data set protects mixed model analyses from bias but not other methods.

Stress and coping with racism and their role in sexual risk for HIV among African American, Asian/Pacific Islander, and Latino men who have sex with men.

The deleterious effects of racism on a wide range of health outcomes, including HIV risk, are well documented among racial/ethnic minority groups in the United States. However, little is known about how men of color who have sex with men (MSM) cope with stress from racism and whether the coping strategies they employ buffer against the impact of racism on sexual risk for HIV transmission. We examined associations of stress and coping with racism with unprotected anal intercourse (UAI) in a sample of African American (N = 403), Asian/Pacific Islander (N = 393), and Latino (N = 400) MSM recruited in Los Angeles County, CA during 2008-2009. Almost two-thirds (65 %) of the sample reported being stressed as a consequence of racism experienced within the gay community. Overall, 51 % of the sample reported having UAI in the prior 6 months. After controlling for race/ethnicity, age, nativity, marital status, sexual orientation, education, HIV serostatus, and lifetime history of incarceration, the multivariate analysis found statistically significant main effects of stress from racism and avoidance coping on UAI; no statistically significant main effects of dismissal, education/confrontation, and social-support seeking were observed. None of the interactions of stress with the four coping measures were statistically significant. Although stress from racism within the gay community increased the likelihood of engaging in UAI among MSM of color, we found little evidence that coping responses to racism buffered stress from racism. Instead, avoidance coping appears to suggest an increase in UAI.

Likelihood-based analysis of longitudinal data from outcome-related sampling designs.

Investigators commonly gather longitudinal data to assess changes in responses over time and to relate these changes to within-subject changes in predictors. With rare or expensive outcomes such as uncommon diseases and costly radiologic measurements, outcome-dependent, and more generally outcome-related, sampling plans can improve estimation efficiency and reduce cost. Longitudinal follow up of subjects gathered in an initial outcome-related sample can then be used to study the trajectories of responses over time and to assess the association of changes in predictors within subjects with change in response. In this article, we develop two likelihood-based approaches for fitting generalized linear mixed models (GLMMs) to longitudinal data from a wide variety of outcome-related sampling designs. The first is an extension of the semi-parametric maximum likelihood approach developed in Neuhaus, Scott and Wild (2002, Biometrika 89, 23-37) and Neuhaus, Scott and Wild (2006, Biometrics 62, 488-494) and applies quite generally. The second approach is an adaptation of standard conditional likelihood methods and is limited to random intercept models with a canonical link. Data from a study of attention deficit hyperactivity disorder in children motivates the work and illustrates the findings.

Substance use and experienced stigmatization among ethnic minority men who have sex with men in the United States.

Research has documented deleterious effects of racism among ethnic minorities and of homophobia among men who have sex with men (MSM). Less is known about the impact of multiple forms of stigmatization on ethnic minority MSM. This study examined substance use by African American, Asian/Pacific Islander and Latino MSM, and the associations of experienced racism and homophobia from various sources with polydrug use and stimulant drug use. Experienced racism within the general community was associated with higher levels of use; other forms of discrimination were either not associated with polydrug or stimulant use or had more complex relationships with use. Implications for further research and interventions are discussed.

Projected Impact of Nonpharmacologic Management of Stage 1 Hypertension Among Lower-Risk US Adults.

BACKGROUND: The 2017 American College of Cardiology/American Heart Association blood pressure guideline classified 31 million US adults as having stage 1 hypertension and recommended clinicians provide counseling on behavioral change to the low-risk portion of this group. However, nationwide reductions in cardiovascular disease (CVD) and associated health care expenditures achievable by nonpharmacologic therapy remain unquantified. METHODS: We simulated interventions on a target population of US adults aged 35 to 64 years, identified from the 2015-2018 National Health and Nutrition Examination Survey, with low-risk stage 1 systolic hypertension: that is, untreated systolic blood pressure 130 to 139 mm Hg with diastolic BP <90 mm Hg; no history of CVD, diabetes, or chronic kidney disease; and a low 10-year risk of CVD. We used meta-analyses and trials to estimate the effects of population-level behavior modification on systolic blood pressure. We assessed the extent to which restricting intervention to those in regular contact with clinicians might prevent the delivery of nonpharmacologic therapy. RESULTS: Controlling systolic blood pressure to <130 mm Hg among the 8.8 million low-risk US adults with stage 1 hypertension could prevent 26 100 CVD events, avoid 2900 deaths, and save $1.7 billion in total direct health care costs over 10 years. Adoption of the Dietary Approaches to Stop Hypertension diet could prevent 28 000 CVD events. Other nonpharmacologic interventions could avert between 3800 and 19 500 CVD events. However, only 51% of men and 75% of women regularly interacted with clinicians for counseling opportunities. CONCLUSIONS: Among low-risk adults with stage 1 hypertension, substantial benefits to cardiovascular health could be achieved through public policy that promotes the adoption of nonpharmacologic therapy.

Experiences of discrimination and their impact on the mental health among African American, Asian and Pacific Islander, and Latino men who have sex with men.

OBJECTIVES: We examined the associations between specific types and sources of discrimination and mental health outcomes among US racial/ethnic minority men who have sex with men (MSM) and how these associations varied by race/ethnicity. METHODS: A chain-referral sample of 403 African American, 393 Asian and Pacific Islander (API), and 400 Latino MSM recruited in Los Angeles County, California completed a standardized questionnaire. Data were obtained from the Ethnic Minority Men's Health Study from May 2008 to October 2009. RESULTS: Past-year experiences of racism within the general community and perceived homophobia among heterosexual friends were positively associated with depression and anxiety. Past-year homophobia experienced within the general community was also positively associated with anxiety. These statistically significant associations did not vary across racial/ethnic groups. The positive association of perceived racism within the gay community with anxiety differed by race/ethnicity, and was statistically significant only for APIs. Perceived homophobia within the family was not associated with either depression or anxiety. CONCLUSIONS: Higher levels of experiences of discrimination were associated with psychological distress among MSM of color. However, specific types and sources of discrimination were differentially linked to negative mental health outcomes among African American, API, and Latino MSM.

Estimation of covariate effects in generalized linear mixed models with a misspecified distribution of random intercepts and slopes.

Generalized linear mixed models with random intercepts and slopes provide useful analyses of clustered and longitudinal data and typically require the specification of the distribution of the random effects. Previous work for models with only random intercepts has shown that misspecifying the shape of this distribution may bias estimates of the intercept, but typically leads to little bias in estimates of covariate effects. Very few papers have examined the effects of misspecifying the joint distribution of random intercepts and slopes. However, simulation results in a recent paper suggest that misspecifying the shape of the random slope distribution can yield severely biased estimates of all model parameters. Using analytic results, simulation studies and fits to example data, this paper examines the bias in parameter estimates due to misspecification of the shape of the joint distribution of random intercepts and slopes. Consistent with results for models with only random intercepts, and contrary to the claims of severe bias in a recent paper, we show that misspecification of the joint distribution typically yields little bias in estimates of covariate effects and is restricted to covariates associated with the misspecified random effects distributions. We also show that misspecification of the distribution of random effects has little effect on confidence interval performance. Coverage rates based on the model-based standard errors from fitted likelihoods were generally quite close to nominal.

Projected Impact of Nonpharmacologic Management of Stage 1 Hypertension Among Lower-Risk US Adults.

Background: The 2017 American College of Cardiology (ACC)/American Heart Association (AHA) guidelines newly classified 31 million US adults as having stage 1 hypertension. The ACC/AHA guidelines recommend behavioral change without pharmacology for the low-risk portion of this group. However, the nationwide reduction in cardiovascular disease (CVD) and associated healthcare expenditures achievable by evidence-based dietary improvements, sustained weight loss, adequate physical activity, and alcohol moderation remain unquantified. We estimated the effect of systolic BP (SBP) control and behavioral changes on 10-year CVD outcomes and costs. Methods: We used the CVD Policy Model to simulate CVD events, mortality, and healthcare costs among US adults aged 35-64. We simulated interventions on a target population, identified from the 2015-2018 National Health and Nutrition Examination Survey, with low-risk stage 1 systolic hypertension: defined as untreated SBP 130-139 mmHg and diastolic BP <90 mmHg; no history of CVD, diabetes, or chronic kidney disease; and low 10-year risk of CVD. We used published meta-analyses and trials to estimate the effects of behavior modification on SBP. We assessed the extent to which intermittent healthcare utilization or partial uptake of nonpharmacologic therapy would decrease CVD events prevented. Results: Controlling SBP to <130 mmHg among the estimated 8.8 million U.S. adults (51% women) in the target population could prevent 26,100 CVD events, avoid 2,900 deaths, and save $1.6 billion in healthcare costs over 10 years. The Dietary Approaches to Stop Hypertension (DASH) diet could prevent 16,000 CVD events among men and 12,000 among women over a decade. Other nonpharmacologic interventions could avert between 3,700 and 19,500 CVD events. However, only 5.5 million (61%) of the target population regularly utilized healthcare where recommended clinician counseling could occur. Conclusions: As only two-thirds of U.S. adults with Stage 1 hypertension regularly receive medical care, substantial benefits to cardiovascular health and associated costs may only stem from policies that promote widespread adoption and sustained adherence of nonpharmacologic therapy. Future work should quantify the population-level costs, benefits, and efficacy of improving the food system and local infrastructure on health behavior change.

A note on type II error under random effects misspecification in generalized linear mixed models.

Litière, Alonso, and Molenberghs (2007, Biometrics, 63, 1038-1044) presented the results of simulation studies that they claimed showed that misspecification of the shape of the random effects distribution can produce marked increases in Type II error (decreases in power) of tests based on fits of generalized linear mixed models. However, the article contains a logical fallacy that invalidates this claim. We present logically correct simulation studies that demonstrate little increase in Type II error, consistent with the earlier work that shows little effect due to misspecification.

Divergent Six Month Functional Recovery Trajectories and Predictors after Traumatic Brain Injury: Novel Insights from the Citicoline Brain Injury Treatment Trial Study.

Cross-sectional approaches to outcome assessment may not adequately capture heterogeneity in recovery after traumatic brain injury (TBI). Using latent class mixed models (LCMM), a data-driven analytic that identifies groups of patients with similar trajectories, we identified distinct 6 month functional recovery trajectories in a large cohort (n = 1046) of adults 18-70 years of age with complicated mild to severe TBI who participated in the Citicoline Brain Injury Treatment Trial (COBRIT). We used multinomial logistic fixed effect models and backward elimination, forward selection, and forward stepwise selection with several stopping rules to explore baseline predictors of functional recovery trajectory. Based on statistical and clinical considerations, the seven-class model was deemed superior. Visualization of these seven functional recovery trajectories revealed that each trajectory class started at one of three recovery levels at 1 month, which, for ease of reference we labeled groups A-C: Group A, good recovery (two classes; A1 and A2); Group B, moderate disability (two classes; B1 and B2); and Group C, severe disability (three classes; C1, C2, and C3). By 6 months, these three groups experienced dramatically divergent trajectories. Group A experienced stable good recovery (A1, n = 115) or dramatic decline (A2, n = 4); Group B experienced rapid complete recovery (B1, n = 71) or gradual recovery (B2, n = 742); Group C experienced dramatic rapid recovery (C1, n = 12), no recovery (C2, n = 91), or death (C3, n = 11). Trajectory class membership was not predicted by citicoline treatment (p = 0.57). The models identified demographic, pre-injury, and injury-related predictors of functional recovery trajectory, including: age, race, education, pre-injury employment, pre-injury diabetes, pre-injury psychiatric disorder, site, Glasgow Coma Scale (GCS) score, post-traumatic amnesia, TBI mechanism, major extracranial injury, hemoglobin, and acute computed tomographic (CT) findings. GCS was the most consistently selected predictor across all models. All models also selected at least one demographic or pre-injury medical predictor. LCMM successfully identified dramatically divergent, clinically meaningful 6 month recovery trajectories with utility to inform clinical trial design.

Genetic and clinical predictors of CD4 lymphocyte recovery during suppressive antiretroviral therapy: Whole exome sequencing and antiretroviral therapy response phenotypes.

Increase of peripheral blood CD4 lymphocyte counts is a key goal of combined antiretroviral therapy (cART); most, but not all, recipients respond adequately and promptly. A small number of studies have examined specific genetic factors associated with the extent of CD4 recovery. We report a genome-wide examination of factors that predict CD4 recovery in HIV-infected women. We identified women in in a cohort study who were on cART with viral load below 400 copies, and drew racially and ethnically matched samples of those with good CD4 response over 2 years or poor response. We analyzed the exomes of those women employing next generation sequencing for genes associated with CD4 recovery after controlling for non-genetic factors identified through forward stepwise selection as important. We studied 48 women with good CD4 recovery and 42 with poor CD4 recovery during virologically-suppressive cART. Stepwise logistic regression selected only age as a statistically significant (p<0.05) non-genetic predictor of response type (each additional year of age reduced the odds of good recovery by 11% (OR = 0.89, CI = 0.84-0.96, p = 0.0009). After adjustment for age and genomic estimates of race and ethnicity, 41 genes harbored variations associated with CD4 recovery group (p≤0.001); 5 of these have been previously reported to be associated with HIV infection, 4 genes would likely influence CD4 homeostasis, and 13 genes either had known functions or were members of product families that had functions for which interactions with HIV or effects on lymphocyte homeostasis were biologically plausible. Greater age was the strongest acquired factor that predicted poor CD4 cell recovery. Sequence variations spanning 41 genes were independently predictive of CD4 recovery. Many of these genes have functions that impact the cell cycle, apoptosis, lymphocyte migration, or have known interactions with HIV. These findings may help inform new hypotheses related to responses to HIV therapy and CD4 lymphocyte homeostasis.

Fibrosis progression in African Americans and Caucasian Americans with chronic hepatitis C.

BACKGROUND & AIMS: Prior studies suggest the rate of liver fibrosis progression is slower in African Americans (AAs) than Caucasian Americans (CAs) with chronic HCV infection. METHODS: With a multi-state Markov model, fibrosis progression was evaluated in a well-characterized cohort of 143 AA and 157 CA adults with untreated chronic HCV genotype 1 infection. In subjects with a history of injection drug use, duration of infection was imputed from a fitted risk model rather than assumed to be the reported first year of use. RESULTS: The distribution of Ishak fibrosis stages was 0 (8.7%), 1/2 (55.7%), 3/4 (29.3%), and 5/6 (6.3%) and was similar in AAs and CAs (P = .22). After adjusting for biopsy adequacy, AAs had a 10% lower rate of fibrosis progression than did CAs, but the difference was not statistically significant (hazard ratio, 0.90; 95% confidence interval, 0.72-1.12). The overall 20-year estimates of probabilities of progression from stage 0 to stages 1/2, 3/4, and 5/6 were 59.3%, 28.8%, and 4.7%, respectively. The estimated median time from no fibrosis to cirrhosis was 79 years for the entire cohort and 74 and 83 years for CAs and AAs, respectively. In 3-variable models including race and biopsy adequacy, the factors significantly associated with fibrosis progression were age when infected, steatosis, ALT level, and necroinflammatory score. CONCLUSIONS: The rates of fibrosis progression were slow and did not appear to differ substantially between AAs and CAs.

Enhanced liver fibrosis marker as a noninvasive predictor of mortality in HIV/hepatitis C virus-coinfected women from a multicenter study of women with or at risk for HIV.

OBJECTIVE: Coinfection with hepatitis C virus (HCV) is a major cause of morbidity and mortality among individuals with HIV. Our objective was to assess the prognostic performance of noninvasive measures of liver fibrosis in predicting all-cause mortality in women with HIV/HCV coinfection. DESIGN: We studied HCV/HIV coinfected women enrolled in the prospective, multicenter Women's Interagency HIV Study. Aspartate aminotransferase to platelet ratio and FIB-4 were used to identify women without fibrosis at all visits and women who progressed to severe fibrosis. METHODS: Enhanced liver fibrosis (ELF), which utilizes direct measures of fibrosis, hyaluronic acid, procollagen III aminoterminal peptide and tissue inhibitor of matrix metalloproteinase was performed. RESULTS: Included were 381 women with 2296 ELF measurements, with mean follow-up 8.3 ±â€Š3.3 years. There were 134 deaths (60% with severe liver fibrosis). Receiver operator characteristic curves at fixed time windows prior to death or at end of follow-up showed that ELF was best at predicting mortality when tested within a year of death (area under the curve for ELF 0.85 vs. APRI 0.69, P < 0.0001 and vs. FIB-4 0.75, P = 0.0036); and 1-3 years prior (ELF 0.71 vs. APRI 0.61, P = 0.005 and vs. FIB-4 0.65, P = 0.06). Use of all three measures did not improve on ELF alone. In multivariate logistic regression models controlling for CD4 cell count, HIV viral load, antiretroviral use and age, ELF continued to perform better than APRI and FIB-4. CONCLUSION: ELF predicted all-cause mortality and was superior to APRI and FIB-4 in HIV/HCV coinfected women.

Social network characteristics and HIV risk among African American, Asian/Pacific Islander, and Latino men who have sex with men.

OBJECTIVES: To examine how social networks influence HIV risk among US racial/ethnic minority men who have sex with men (MSM) and whether the associations of social network characteristics with risk vary by race/ethnicity. METHODS: A chain-referral sample of 403 African American, 393 Asian/Pacific Islander, and 400 Latino MSM recruited in Los Angeles County, California, completed a questionnaire, which asked about their egocentric social networks, safer sex peer norms, and male anal intercourse partners. HIV-nonconcordant partnerships were those reported by respondents as serodisconcordant or where self and/or partner serostatus was unknown. RESULTS: Overall, 26% of the sample reported HIV-nonconcordant unprotected anal intercourse (UAI) with a nonprimary male partner in the previous 6 months. In a generalized estimating equation (GEE) logistic model that controlled for race/ethnicity, age, nativity, incarceration history, and HIV status, being in a more dense network was associated with less HIV-nonconcordant UAI [adjusted odds ratio (AOR) = 0.92, 95% confidence interval (CI): 0.86 to 0.99, P = 0.0467]. In addition, the effect of safer sex peer norms on HIV-nonconcordant UAI was moderated by ego-alter closeness (P = 0.0021). Safer sex peer norms were protective among those reporting "medium" or "high" ego-alter closeness (AOR = 0.70, 95% CI: 0.52 to 0.95, P = 0.0213 and AOR = 0.48, 95% CI: 0.35 to 0.66, P < 0.0001, respectively), but not among those reporting "low" ego-alter closeness (AOR = 0.96, 95% CI: 0.63 to 1.46, P = 0.8333). The effects of density, closeness, and norms on HIV-nonconcordant UAI did not differ by race/ethnicity. CONCLUSIONS: The significant association of social network characteristics with UAI point to network-level factors as important loci for both ongoing research and HIV prevention interventions among US MSM of color.

Progression of biopsy-measured liver fibrosis in untreated patients with hepatitis C infection: non-Markov multistate model analysis.

BACKGROUND: Fibrosis stages from liver biopsies reflect liver damage from hepatitis C infection, but analysis is challenging due to their ordered but non-numeric nature, infrequent measurement, misclassification, and unknown infection times. METHODS: We used a non-Markov multistate model, accounting for misclassification, with multiple imputation of unknown infection times, applied to 1062 participants of whom 159 had multiple biopsies. Odds ratios (OR) quantified the estimated effects of covariates on progression risk at any given time. RESULTS: Models estimated that progression risk decreased the more time participants had already spent in the current stage, African American race was protective (OR 0.75, 95% confidence interval 0.60 to 0.95, p = 0.018), and older current age increased risk (OR 1.33 per decade, 95% confidence interval 1.15 to 1.54, p = 0.0002). When controlled for current age, older age at infection did not appear to increase risk (OR 0.92 per decade, 95% confidence interval 0.47 to 1.79, p = 0.80). There was a suggestion that co-infection with human immunodeficiency virus increased risk of progression in the era of highly active antiretroviral treatment beginning in 1996 (OR 2.1, 95% confidence interval 0.97 to 4.4, p = 0.059). Other examined risk factors may influence progression risk, but evidence for or against this was weak due to wide confidence intervals. The main results were essentially unchanged using different assumed misclassification rates or imputation of age of infection. DISCUSSION: The analysis avoided problems inherent in simpler methods, supported the previously suspected protective effect of African American race, and suggested that current age rather than age of infection increases risk. Decreasing risk of progression with longer time already spent in a stage was also previously found for post-transplant progression. This could reflect varying disease activity, with recent progression indicating active disease and high risk, while longer time already spent in a stage indicates quiescent disease and low risk.

Non-Markov multistate modeling using time-varying covariates, with application to progression of liver fibrosis due to hepatitis C following liver transplant.

Multistate modeling methods are well-suited for analysis of some chronic diseases that move through distinct stages. The memoryless or Markov assumptions typically made, however, may be suspect for some diseases, such as hepatitis C, where there is interest in whether prognosis depends on history. This paper describes methods for multistate modeling where transition risk can depend on any property of past progression history, including time spent in the current stage and the time taken to reach the current stage. Analysis of 901 measurements of fibrosis in 401 patients following liver transplantation found decreasing risk of progression as time in the current stage increased, even when controlled for several fixed covariates. Longer time to reach the current stage did not appear associated with lower progression risk. Analysis of simulation scenarios based on the transplant study showed that greater misclassification of fibrosis produced more technical difficulties in fitting the models and poorer estimation of covariate effects than did less misclassification or error-free fibrosis measurement. The higher risk of progression when less time has been spent in the current stage could be due to varying disease activity over time, with recent progression indicating an "active" period and consequent higher risk of further progression.

Estimating complex multi-state misclassification rates for biopsy-measured liver fibrosis in patients with hepatitis C.

For both clinical and research purposes, biopsies are used to classify liver damage known as fibrosis on an ordinal multi-state scale ranging from no damage to cirrhosis. Misclassification can arise from reading error (misreading of a specimen) or sampling error (the specimen does not accurately represent the liver). Studies of biopsy accuracy have not attempted to synthesize these two sources of error or to estimate actual misclassification rates from either source. Using data from two studies of reading error and two of sampling error, we find surprisingly large possible misclassification rates, including a greater than 50% chance of misclassification for one intermediate stage of fibrosis. We find that some readers tend to misclassify consistently low or consistently high, and some specimens tend to be misclassified low while others tend to be misclassified high. Non-invasive measures of liver fibrosis have generally been evaluated by comparison to simultaneous biopsy results, but biopsy appears to be too unreliable to be considered a gold standard. Non-invasive measures may therefore be more useful than such comparisons suggest. Both stochastic uncertainty and uncertainty about our model assumptions appear to be substantial. Improved studies of biopsy accuracy would include large numbers of both readers and specimens, greater effort to reduce or eliminate reading error in studies of sampling error, and careful estimation of misclassification rates rather than less useful quantities such as kappa statistics.