Evaluating surrogacy metrics and investigating approval decisions of progression-free survival (PFS) in metastatic renal cell cancer: a systematic review.

BACKGROUND: In metastatic renal cell cancer (mRCC) trials, progression-free survival (PFS) is increasingly used instead of overall survival (OS) as the approval end point. Unlike other solid tumors, there is no published demonstration of what PFS is needed across and by treatment class in mRCC. We determine this and evaluate drug approval decisions in mRCC targeted therapy. METHODS: We identified all randomized, controlled trials reporting PFS and OS in mRCC. Surrogacy metrics were the coefficient of determination and surrogate threshold effect (STE)-the PFS difference needed to predict, with 95% confidence, an OS difference. Data from regulatory commentaries, briefing documents and transcripts were extracted. RESULTS: No exclusively chemotherapy trial met criteria. Of 30 qualifying trials, 11 trials (13 comparisons) used targeted therapy. The all-trials and immunotherapy-only trials analysis failed to demonstrate a STE. The targeted trials, using the more conservative regression analysis demonstrated an STE of 3.9 months and an R(2) of 0.44. Crossover upon progression, control to active treatment, was common. Regulatory approval, accelerated or regular, labeling, interim analyses, and adjudication were context specific. CONCLUSIONS: A new targeted therapy trial showing a PFS difference of 3.9 months can claim an OS benefit in mRCC. PFS surrogacy for OS in metastatic renal cell is not generalizable across all drug classes.

How many life years are lost in patients with rheumatoid arthritis? Secular cause-specific and all-cause mortality in rheumatoid arthritis, and their predictors in a long-term Australian cohort study.

BACKGROUND: There is an excess of mortality in patients with rheumatoid arthritis (RA) but no long-term Australian cohort data. AIMS: To determine median life years lost, all-cause standardised mortality ratio (SMR) and cause-specific SMR, their predictors and secular change in Australian patients with RA. METHODS: Study population was all patients seen by a rheumatologist between 1990 and 1994. Record linkage with Australian National Death Index was performed to determine fact and cause of death up to 2004. All-cause and cause-specific SMR, and median life years lost were determined. RESULTS: There were 35 (31%) deaths in the early 1990s cohort (n = 113), SMR 1.31 (95% 0.93, 1.80). There were 216 (44%) deaths in the pre-1990s established cohort (n = 495), SMR 1.73 (1.49, 1.95). Median life years lost in the early cohort was 6 years for males and 7 years for females compared with 8 and 10 years, respectively, in the established cohort. Patients with low disease activity score at baseline (DAS < 3.2), SMR was 0.8 (0.3, 2.2) and 1.5 (1.1, 2.2) for the early and established cohorts, and if DAS ≥3.2, SMR was 1.4 (1.02, 1.98) and 1.8 (1.5, 2.1) respectively. Primary cause of death was cardiovascular disease (SMR 1.43 (1.17, 1.74). Patients at most risk were those age 45-54 years. RA was listed as a comorbid condition on the death certificate in only 16% of patients. CONCLUSIONS: Within a period of 14 years, median life expectancy of patients with RA with disease onset in the early 1990s is reduced by 6-7 years. However, our results also suggest a secular reduction in excess mortality.

Arthroscopic estimation of the extent of chondropathy.

INTRODUCTION: Arthroscopy has been used to evaluate articular cartilage (AC) pathology in osteoarthritis (OA) for outcome measurement and validation of non-invasive imaging. However, many fundamental aspects of arthroscopic assessment remain un-validated. OBJECTIVES: This study evaluated arthroscopic estimates of extent of chondropathy. METHODS: Serial arthroscopic assessments were performed in a group of 15 sheep before and after bilateral stifle medial meniscectomy (MMx). Post-mortem assessments were performed in un-MMx sheep and 4 and 16 weeks post-MMx. Arthroscopic assessments of the extent of each grade of chondropathy were compared with a non-arthroscopic hybrid assessment that incorporated biomechanical, thickness and macroscopic assessments. RESULTS: Arthroscopy evaluated only 36% of AC and missed significant pathological changes, softening and chondro-osteophyte, occurring in peripheral regions. The patterns of change in arthroscopic assessments were similar to those of the non-arthroscopic assessment but there was a very strong tendency to over-estimate the extent of softened AC after MMx. In spite of these limitations arthroscopic assessments were responsive to change. Estimates of the extent of normal and softened AC were most responsive to change over time followed by estimates of superficial and deep fibrillation. Arthroscopy was as an excellent discriminator between normal and OA. Assessments of chondro-osteophyte and exposed bone were not responsive to change. CONCLUSIONS: Arthroscopic estimates of extent of chondropathy are prone to substantial error. While experience and training may reduce these errors other approaches may more effectively improve performance.

A users guide to measurement in medicine.

Measurement is fundamental to science. In medicine measurement underpins most clinical decisions. Outcome measures for rheumatoid arthritis clinical trials (OMERACT) is an informal collaborative group of professionals dedicated to improving outcome measurement in the rheumatic disease. The methodologic hallmark of the OMERACT process is captured in the OMERACT filter--truth, discrimination, and feasibility. Using the key elements of the OMERACT filter a comprehensive checklist for evaluating reported measures is provided. The checklist guides the potential user through a series of questions. The checklist is also an important resource for researchers working in the field of measurement.

Arthroscopy -- a potential "gold standard" for the diagnosis of the chondropathy of early osteoarthritis.

OBJECTIVES: The aims of this study were to: 1. Evaluate the performance of arthroscopy for the diagnosis of chondropathy and to compare it to that of direct non-arthroscopic assessments; 2. Determine intra-observer reliability of arthroscopic assessments; 3. Evaluate the effects of the arthroscopic video quality and probing upon diagnostic performance. DESIGN: The ovine medial meniscectomy (MMx) model of early osteoarthritis (OA) was used assuming that pre-MMx articular cartilage (AC) was "normal" and post-MMx AC "chondropathic". Video recordings of arthroscopic assessments of each stifle compartment were evaluated. Scores were given for the quality of the video and the amount of probing. The diagnostic performances of dynamic shear modulus (G), light microscopic assessment and superficial zone collagen birefringence assessments were evaluated and compared to that of arthroscopy. Intra-observer reliability of arthroscopic assessments was also evaluated. RESULTS: Arthroscopic assessments had high sensitivity (91-100%), specificity (62-88%) and accuracy (75-93%) for the diagnosis of chondropathy 16 weeks after MMx. Arthroscopy compared favourably with the direct non-arthroscopic assessments in the lateral compartment and was found to have extremely high intra-observer reliability (kappa 0.78-1.00). The quality of arthroscopic video recordings and the amount of probing did not significantly influence accuracy or reliability. CONCLUSIONS: Arthroscopy performs as well as direct non-arthroscopic assessments of AC for diagnosis of early OA. These results suggest that arthroscopy can be used as a "gold standard" for the validation of non-invasive assessments like magnetic resonance imaging and that arthroscopic diagnosis can be based on small amounts of video footage without AC probing.

Biomechanical, histologic and macroscopic assessment of articular cartilage in a sheep model of osteoarthritis.

OBJECTIVES: Our primary objective was to explore the full potential of the ovine medial meniscectomy (MMx) model of early osteoarthritis (OA) for studies to validate non-destructive articular cartilage (AC) assessments and therapeutic interventions. Our secondary objective was to re-evaluate the relationships between the different types of AC assessment after MMx in sheep. METHODS: Macroscopic assessments, dynamic shear modulus (G*), phase lag and AC thickness measurements were performed at a total of 5437 reference points on all six articular surfaces in four normal joints and 16 MMx ovine stifle (knee) joints. Comparisons with histologic assessments of gross structural damage, collagen organisation (birefringence) and proteoglycan content were possible at 702 of these points. RESULTS: Histologic gross structural damage and proteoglycan loss were seen throughout the joint with greatest severity (fibrillation) in closest proximity to the MMx site. Increases in AC (30-50%) thickness, reductions in G* (30-40%) and collagen birefringence intensity (15-30%) occurred more evenly throughout the joint. Macroscopic softening was evident only when G* declined by 80%. G* correlated with AC thickness (rho=-0.47), collagen organisation rho=0.44), gross structural damage (rho=-0.44) and proteoglycan content (rho=0.42). Multivariate analysis showed that collagen organisation contributed twice as much to dynamic shear modulus (t=6.66 as proteoglycan content (t=3.21). Collagen organisation (rho=0.11) and proteoglycan content (rho=0.09) correlated only weakly to phase lag. CONCLUSIONS: Macroscopic assessments were insensitive to AC softening suggesting that arthroscopic assessments of AC status might also perform poorly. Collagen integrity was more important for the maintenance of AC stiffness (G*) than proteoglycan content. The development of major AC softening and thickening throughout the joint following MMx suggested involvement of non-mechanical (e.g., protein and biochemical) chemical and cytokine mediated processes in addition to the disturbance in biomechanical loading. The ovine MMx model provides a setting in which the spectrum of AC changes associated with the initiation and progression of OA may be evaluated.

Poor accuracy and interobserver reliability of knee arthroscopy measurements are improved by the use of variable angle elongated probes.

OBJECTIVES: (a)To determine the accuracy and reliability of arthroscopic measurements of cartilage lesion diameter in an artificial right knee model; (b) to determine whether the use of a set of variable angle elongated probes improves performance; and (c) to identify other sources of variability. METHODS: Ovoid "lesions" were drawn on the five cartilage surfaces of four plastic knees models. Two observers assessed these 20 lesions arthroscopically, measuring two diameters in orientations parallel and orthogonal to the probe. Observer 1 (orthopaedic surgeon) and observer 2 (arthroscopic rheumatologist) made two sets of measurements, firstly with the conventional probe and five months later with the variable angle elongated (VAE) probes. The knees were disarticulated to determine true lesion diameter. RESULTS: Observer 1 had negligible bias and good accuracy regardless of orientation or probe type. Observer 2 demonstrated both bias and poor accuracy using the conventional probe. Both improved using VAE probes. Poor interobserver reliability with conventional probes also improved using VAE probes. Major sources of variability could be traced to the probe type, the characteristics of the operator, and the orientation of the lesion in relation to the probe; the lesion location itself did not cause variability. CONCLUSIONS: Variation in accuracy and poor interobserver reliability of measurements with conventional methods of cartilage lesion diameter measurement improved when specially designed measurement probes were used. Arthroscopic measurements performed as well as most clinical and radiographic measures. These findings have important implications for the use of arthroscopy as an outcome in multicentre trials where arthroscopists have different levels of experience.

Foundations of the minimal clinically important difference for imaging.

This article develops a generic conceptual framework for defining and validating the concept of minimal clinically important difference. We propose 3 approaches. The first uses statistical descriptions of the population ("distribution based"), the second relies on experts ("opinion based"), and a third is based on sequential hypothesis formation and testing ("predictive/data driven based"). The first 2 approaches serve as proxies for the third, which is an experimentally driven approach, asking such questions as "What carries the least penalty?" or "What imparts the greatest gain?" As an experimental approach, it has the expected drawbacks, including the need for greater resources, and the need to tolerate trial and error en route, compared to the other 2 models.

Robustness and generalizability of smallest detectable difference in radiological progression.

The smallest detectable difference (SDD) reflects that component of a measure statistically attributable to error from the measurement process itself. As such it is an irreducible component of the inherent variability in measurements in clinical trials and will affect their design, whether randomized or observational. Even though the application of the SDD concept to assaying radiographs in rheumatoid arthritis is relatively new and not well understood, systematic work on the influences of radiographic SDD can be done. This report describes the effects of a number of clinical aspects of the disease and operational aspects of trials on the values of the SDD of radiographic progression data. We show that if conditions affecting SDD are known and kept constant across datasets, the SDD of radiological progression from one study may be generalizable to other studies. However, if any one condition varies, the SDD is distinctly unrobust and cannot be generalized to other studies.

Reliability of measures of disease activity and disease damage in rheumatoid arthritis: implications for smallest detectable difference, minimal clinically important difference, and analysis of treatment effects in randomized controlled trials.

We evaluate measurement properties of common rheumatoid arthritis (RA) assessments. Included are a comprehensive literature review and new data on the reliability and smallest detectable difference (SDD) for different classes of these measures. We found that certain common measures such as joint counts, pain, and patient global all had poor reliability and showed large SDD compared to multi-item measures of physical/psychological function or compared to radiographic measures. We discuss the implications of these findings on the use of composite endpoints such as the ACR20 or the EULAR responder index in RA clinical trials, particularly the introduction of misclassification bias that arises from differential measurement error. Finally, we consider generically how the concept of the SDD might or might not relate to the concept of the minimal clinically important difference.

Measurements of rheumatoid arthritis disease activity and damage using magnetic resonance imaging. Truth and discrimination: does MRI make the grade?

Magnetic resonance imaging (MRI) is a tool with unprecedented capabilities. Rheumatoid arthritis (RA) abnormalities that can be measured with MRI include erosions, articular cartilage thickness, synovial membrane volume, and pannus. However, as access to MRI increases, there is a risk that its use will not be evaluated using rigorous scientific measurement principles. We reviewed published MRI measurement methods for RA and investigated whether the methods were systematically evaluated for reliability, validity, and responsiveness to change--components of the OMERACT filter. Medline and Embase databases were searched from 1966 to 1999. Titles and abstracts were scanned to identify publications on MRI methods used to assess either disease activity or damage in RA. A data extraction template was developed and 68 peer reviewed publications from 40 research groups were appraised; 40 addressed RA disease activity, 4 RA damage, and 24 both activity and damage. Joints most frequently assessed were knee (32 publications) and wrist (31 publications). Ninety-one percent of publications evaluated either reliability or validity or responsiveness to change. Thirteen percent evaluated all 3 and only 9% evaluated none of these measurement properties. Validity was evaluated in 85%, responsiveness to change in 37%, and reliability in 35% of publications. Only 12% of publications evaluated both intra and inter-reliability. Few publications of MRI measures of disease activity or damage in RA met the OMERACT filter for all measurement properties. It would be regrettable if MRI measures are developed ad hoc, with little regard to considerations of scaling, reliability, validity, and responsiveness to change, because this will severely limit their ability to confidently assess treatment efficacy and prognostic indicators.

Driving and arthritis.

In a 3-year period, 94 patients with driving difficulties due to a variety of musculoskeletal disorders were assessed by the occupational therapy department of a rheumatology unit. The individual's ability to carry out each part of the driving process was recorded and the patients were classified into six broad categories. While some parts of the driving process often proved difficult, patterns of disability were found with different musculoskeletal disorders. Almost all of these difficulties could be overcome by simple modifications to vehicle or driving technique. Only one severely disabled individual required referral to a specialized mobility unit, while two individuals were found to be unsafe. Thus, almost all arthritic individuals are able to continue driving with the help of simple modifications. By providing an unsophisticated driving assessment service, a rheumatology unit can enable patients to continue driving and so maintain independence.

Recurrent calcific periarthritis, erosive osteoarthritis and hypophosphatasia: a family study.

We describe a mother and 2 daughters with familial recurrent calcific periarthritis in a family with an inherited tendency to develop generalized osteoarthritis (OA). Low levels of serum alkaline phosphatase were found in 1 of the daughters while the mother developed erosive OA in later life. HLA typing was noncontributory. However, the 3 individuals with periarthritis possessed blood group A+, while the 6 unaffected family members were O+. The experience of this family adds weight to the case that recurrent calcific periarthritis may be due to an inherited abnormality of alkaline phosphatase production and suggest that this may also be responsible for the recently observed association of calcific periarthritis and erosive OA.