CCR4 agonists CCL22 and CCL17 are elevated in pediatric OMS sera: rapid and selective down-regulation of CCL22 by ACTH or corticosteroids.

PURPOSE: To study the role of Th2-attracting chemokines in opsoclonus-myoclonus syndrome (OMS), a serious neurological paraneoplastic disorder in need of better immunological understanding and therapy. METHODS: The CCR4 agonists CCL22 and CCL17 were measured in serum by ELISA in children with OMS (238 and 260, respectively), pediatric controls (115 and 143), and other inflammatory neurological disorders (33 and 24). RESULTS: Both CCL22 (+55 %) and CCL17 (+121 %) were significantly elevated in untreated OMS compared to controls and inter-correlated (p < 0.0001). Their concentrations in untreated OMS also were higher than in OIND (21 %, 41 %). The concentration of CCL22 in ACTH and steroids groups (not IVIg) was 51 % lower than in controls, but only a smaller effect of ACTH on CCL17 was found. Prospective longitudinal studies revealed a precipitous 81 % drop in CCL22 even by the first week of high-dose ACTH therapy, staying below control mean for at least 12 weeks, and a 34 % reduction after 8 months of combined treatment. Response to ACTH was dose-related (r = -0.50, p < 0.0001). Luminex detection confirmed the ELISA results for CCL22, which were about 200 % higher. CONCLUSIONS: These data reveal an elevated serum concentration of Th2-attracting chemokines CCL22 and CCL17 in OMS. Marked and rapid reduction in CCL22, not CCL17, with either ACTH or steroid therapy suggests differential regulation and cellular sources of CCR4 ligands, and CCL22 as a potential candidate biomarker for ACTH or corticosteroid effect.

BAFF/APRIL system in pediatric OMS: relation to severity, neuroinflammation, and immunotherapy.

BACKGROUND: B-cell dysregulation has been implicated but not fully characterized in pediatric opsoclonus-myoclonus syndrome (OMS), a neuroblastoma-associated neuroinflammatory disorder. OBJECTIVE: To assess the role of B-cell activating factor (BAFF) and a proliferation-inducing ligand (APRIL), two critical B cell-modulating cytokines, as potential biomarkers of disease activity and treatment biomarkers in OMS. METHODS: Soluble BAFF and APRIL were measured in cerebrospinal fluid (CSF) and serum by ELISA in 433 children (296 OMS, 109 controls, 28 other inflammatory neurological disorders (OIND)). BAFF-R receptors on circulating CD19+ B cells were measured by flow cytometry. A blinded scorer rated motor severity on the OMS Evaluation Scale. Immunotherapies were evaluated cross-sectionally and longitudinally. RESULTS: The mean CSF BAFF concentration, which was elevated in untreated OMS and OIND, correlated with OMS severity category (P = 0.006), and reduction by adrenocorticotropic hormone or corticotropin (ACTH) (-61%) or corticosteroids (-38%) was seen at each level of severity. In contrast, CSF APRIL was normal in OMS and OIND and unaffected by immunotherapy. When the entire OMS dataset was dichotomized into 'high' versus 'normal' CSF BAFF concentration, the phenotype of the high group included greater motor severity and number of CSF oligoclonal bands, and a higher concentration of inflammatory chemokines CXCL13 and CXCL10 in CSF and CXCL9 and CCL21 in serum. Serum APRIL was 6.7-fold higher in the intravenous immunoglobulins (IVIg) group, whereas serum BAFF was 2.6-fold higher in the rituximab group. The frequency of B cell BAFF-R expression was similar in untreated and treated OMS. Longitudinal studies of CSF BAFF revealed a significant decline in ACTH-treated patients (with or without rituximab) (P < 0.0001). Longitudinal studies of serum APRIL showed a 2.9-fold increase after 1 to 2 g/kg IVIg monotherapy (P = 0.0003). CONCLUSIONS: Striking distinctions in BAFF/APRIL signaling were found. OMS displayed heterogeneity in CSF BAFF expression, which met many but not all criteria as a potential biomarker of disease activity. We speculate that CSF BAFF may have more utility in a biomarker panel than as a stand-alone biomarker, and that the selective upregulation of both serum APRIL by IVIg and BAFF by rituximab, as well as downregulation of CSF BAFF by ACTH/steroids, may have utility as treatment biomarkers.

Theory development and application in medical education.

The role and status of theory is by no means a new topic in medical education. Yet summarizing where we have been and where we are going with respect to theory development and application is difficult because our community has not yet fully elucidated what constitutes medical education theory. In this article, we explore the idea of conceptualizing theory as an effect on scholarly dialogue among medical educators. We describe theory-enabled conversation as argumentation, which frames inquiry, permits the evaluation of evidence, and enables the acquisition of community understanding that has utility beyond investigators' local circumstances. We present ideas for assessing argumentation quality and suggest approaches to increasing the frequency and quality of argumentation in the exchange among diverse medical education scholars.

From test validity to construct validity and back?

CONTEXT: Major changes in thinking about validity have occurred during the past century, shifting the focus in thinking from the validity of the test to the validity of test score interpretations. These changes have resulted from the 'new' thinking about validity in which construct validity has emerged as the central or unifying idea of validity today. Construct validity was introduced by Cronbach and Meehl in the mid-1950s in an attempt to address the validity of those many psychological concepts that have no clear referent in reality. To do this, construct validity theory required a nomological network--an elaborate theoretical network of constructs and observations connected by scientific laws--to validate the constructs. However, nomological networks are hard to come by and none that would do the job required by construct validity has been forthcoming to date. Thus, the current construct validity approach has retreated to one of simply 'interpretation and argument', but this seems to be too general to tie down the constructs in the way a nomological network would do to give credibility to the validity of the construct. As a result, the concept of validity seems to have been watered down and the credibility of validity claims weakened. OBJECTIVES: The purpose of this paper is to encourage a discussion of the use of construct validity in medical education, and to suggest that test developers and users reconsider the use of abstract theoretical constructs that have no referent apart from theory. METHODS: We present a critical review of these concerns about construct validity and provide for contrast a brief overview of a recently proposed view of measurement based on scientific realism and causality analysis.

Systems-based practice assessed with a performance-based examination simulated and scored by standardized participants in the health care system: feasibility and psychometric properties.

BACKGROUND: Systems-based practice is one of the six general competencies proposed by the Accreditation Council for Graduate Medical Education in their Outcome Project. However, little has been published on its assessment--possibly because the systems-based practice competency has been viewed as difficult to define and measure. PURPOSE: The purpose of this study was to determine whether a full performance-based examination of systems-based practice cases simulated and scored by standardized participants in the health care system could feasibly be constructed and implemented that would provide reliable and valid measurements. METHODS: In the 1st year of the project (2008), four systems-based practice cases were developed and pilot tested with 13 residents. Videotapes of residents were studied to develop an instrument for subsequent assessment of performance by standardized participants. In the 2nd year (2009), the examination was expanded to a full 12 cases, which were completed by 11 second-year residents, and psychometric analyses were performed on the scores. RESULTS: The generalizability coefficient for the full 12-case examination based on scoring by standardized participants was .71, which is nearly equal to that based on scoring by faculty physician observers, which was .78. The correlation between total scores obtained with standardized participants and physician observers was .78. CONCLUSIONS: A performance-based examination can provide a feasible and reliable assessment of systems-based practice. However, attempts to evaluate convergent validity and discriminant validity-by correlating systems-based practice performance assessments with mean global ratings of residents on the 6 competencies by faculty throughout training-were unsuccessful, due to a lack of independence between the rated dimensions.

Long-term cerebrospinal fluid and blood lymphocyte dynamics after rituximab for pediatric opsoclonus-myoclonus.

INTRODUCTION: Opsoclonus-myoclonus syndrome (OMS) is an autoimmune paraneoplastic disorder characterized by B and T cell abnormalities in cerebrospinal fluid (CSF) and propensity for relapse. The study aim was to assess whether rituximab-induced B cell ablation in CSF outlasts repopulation in blood and if there are changes in other lymphocyte subsets. MATERIALS AND METHODS: In 25 children with OMS, the expression of CSF and blood lymphocyte surface antigens was evaluated by flow cytometry before and at intervals after rituximab therapy. RESULTS: The reduction in CSF CD27+ memory, CD38+ activated, CD5+, and other B cell subsets was profound (p < 0.0001), comparable across groups (-94%), and sustained over 12-18 months despite repopulation in blood. The observed lag in memory B cell pool recovery in the CSF compared to peripheral blood may be clinically relevant. T cell phenotypic changes involved frequency, not absolute counts, and were transient. Co-treatment with IVIg or ACTH did not significantly alter B cell depletion or repletion. DISCUSSION: These data indicate that rituximab affords long-term protection against CSF B cell expansion in OMS (ClinicalTrials.gov NCT00244361).

B cell depletion therapy for new-onset opsoclonus-myoclonus.

Twelve immunotherapy-naïve children with opsoclonus-myoclonus syndrome and CSF B cell expansion received rituximab, adrenocorticotropic hormone (ACTH), and IVIg. Motor severity lessened 73% by 6 mo and 81% at 1 yr (P < 0.0001). Opsoclonus and action myoclonus disappeared rapidly, whereas gait ataxia and some other motor components improved more slowly. ACTH dose was tapered by 87%. Reduction in total CSF B cells was profound at 6 mo (-93%). By study end, peripheral B cells returned to 53% of baseline and serum IgM levels to 63%. Overall clinical response trailed peripheral B cell and IgM depletion, but improvement continued after their levels recovered. All but one non-ambulatory subject became ambulatory without additional chemotherapy; two relapsed and remitted; four had rituximab-related or possibly related adverse events; and two had low-titer human anti-chimeric antibody. Combination of rituximab with conventional agents as initial therapy was effective and safe. A controlled trial with long-term safety monitoring is indicated.

Therapeutic down-regulation of central and peripheral B-cell-activating factor (BAFF) production in pediatric opsoclonus-myoclonus syndrome.

Opsoclonus-myoclonus syndrome (OMS) is an autoimmune, paraneoplastic, central nervous system disorder, characterized by cerebrospinal fluid (CSF) B-cell expansion and various putative autoantibodies. To investigate the role of B-cell activating factor (BAFF) in OMS and the effect of disease-modifying immunotherapies used to treat it, BAFF was measured by enzyme-linked immunoadsorbent assay in the CSF and serum of 161 children with OMS and 116 pediatric controls. The mean concentration of CSF BAFF and the CSF/serum BAFF ratio were significantly higher in untreated OMS compared to neurological controls. CSF and serum BAFF levels were significantly lower in children treated with ACTH or corticosteroids, as was the CSF/serum BAFF ratio. There was a strong, negative correlation between CSF or serum BAFF levels and ACTH dose. Monthly IVIg infusions had no net impact on BAFF levels, and the combination of IVIg with ACTH or steroids did not reduce or enhance their anti-BAFF effects. These data indicate that BAFF production is increased centrally, not peripherally, in OMS, implying astrocytic over production. The novel dose-related central and peripheral anti-BAFF properties of ACTH, especially, have implications for other BAFF-related autoimmune disorders, infectious diseases, and cancers.

Meta-analysis of quasi-experimental research: are systematic narrative reviews indicated?

CONTEXT: Meta-analyses are commonly performed on quasi-experimental studies in medical education and other applied field settings, with little or no apparent concern for biases and confounds present in the studies synthesised. The implicit assumption is that the biases and confounds are randomly distributed across the studies and are averaged or cancelled out by the synthesis. OBJECTIVES: We set out to consider the possibility that the results and conclusions of meta-analyses in medical education are subject to biases and confounds and to illustrate this possibility with a re-examination of the studies synthesised in an important, recently published meta-analysis of problem-based learning. METHODS: We carefully re-examined the studies in the meta-analysis. Our aims were to identify obvious biases and confounds that provided plausible alternative explanations of each study's results and to determine whether these threats to validity were considered and convincingly ruled out as plausible rival hypotheses. RESULTS: Ten of the 11 studies in the meta-analysis used quasi-experimental designs; all 10 were subject to constant biases and confounds that favoured the intervention condition. Threats to validity were not ruled out in the individual studies, nor in the meta-analysis itself. CONCLUSIONS: Our re-examination of the results and conclusions of the meta-analysis illustrates our concerns about the validity of meta-analyses based primarily on quasi-experimental studies. Our tentative conclusion is that the field of medical education might be better served in most instances by systematic narrative reviews that describe and critically evaluate individual studies and their results in light of threats to their validity.

Assuring the reliability of resident performance appraisals: more items or more observations?

BACKGROUND: The tendency to add items to resident performance rating forms has accelerated due to new ACGME competency requirements. This study addresses the relative merits of adding items versus increasing number of observations. The specific questions addressed are (1) what is the reliability of single items used to assess resident performance, (2) what effect does adding items have on reliability, and (3) how many observations are required to obtain reliable resident performance ratings. METHODS: Surgeon ratings of resident performance were collected for 3 years. The rating instrument had 3 single items representing clinical performance, professional behavior, and comparisons to other house staff. Reliability analyses were performed separately for each year, and variance components were pooled across years to compute overall reliability coefficients. RESULTS: Single-item resident performance rating scales were equivalent to multiple-item scales using conventional reliability standards. Increasing the number of rating items had little effect on reliability. Increasing the number of observations had a much larger effect. CONCLUSIONS: Program directors should focus on increasing the number of observations per resident to improve performance sampling and reliability of assessment. Increasing the number of rating items had little effect on reliability and is unlikely to assess new ACGME competencies adequately.

Educational theory and medical education practice: a cautionary note for medical school faculty.

Educational theory is routinely cited as justification for practice in medical education, even though the justification for the theory itself is unclear. Problem-based learning (PBL), for example, is said to be based on powerful educational principles that should result in strong effects on learning and performance. But research over the past 20 years has produced little convincing evidence for the educational effectiveness of PBL, which naturally raises doubts about the underlying theory. This essay reflects on educational theory, in particular cognitive theory, and concludes that the theory is little more than metaphor, not rigorous, tested, confirmed scientific theory. This metaphor/theory may lead to ideas for basic and applied research, which in turn may facilitate the development of theory. In the meantime, however, the theory cannot be trusted to determine practice in medical education. Despite the intuitive appeal of educational theory, medical educators have a responsibility to set aside their enthusiasm and make it clear to medical school faculty and administrators that educational innovations and practice claims are, at best, founded on conjecture, not on evidence-based science.

Hot versus cold tonsillectomy: a systematic review of the literature.

OBJECTIVE: We systematically reviewed the literature comparing monopolar electrocautery versus cold knife dissection tonsillectomy. METHODS: The MEDLINE database was searched using the key words "tonsillectomy," "hot," "cold," "sharp," "bleeding," and "cautery." Selection criteria included prospective trials comparing electrodissection versus cold knife dissection on posttonsillectomy pain and hemorrhage. Six of the 815 articles met the selection criteria. RESULTS: Pooled data for the paired studies showed significantly more patients with pain worse on the electrodissection side (148 of 293 = 51%) than the cold knife side (33 of 293 [11%]; P = 0.001) on postoperative days 4 to 10. There were significantly more analgesic doses after surgery with electrodissection (means 26.7 versus 19.2; P = 0.028) and higher pain scores for adults undergoing electrodissection (means, 2.6 versus 0.8; significance could not be determined). There were no differences in hemorrhage rates. CONCLUSIONS: Electrodissection increases pain in comparison to sharp dissection for tonsillectomy. Postoperative hemorrhage rates are not significantly different when comparing the 2 methods.

Variations in senior medical student diagnostic justification ability.

PURPOSE: To determine the diagnostic justification proficiency of senior medical students across a broad spectrum of cases with common chief complaints and diagnoses. METHOD: The authors gathered diagnostic justification exercise data from the Senior Clinical Comprehensive Examination taken by Southern Illinois University School of Medicine's students from the classes of 2011 (n = 67), 2012 (n = 66), and 2013 (n = 79). After interviewing and examining standardized patients, students listed their key findings and diagnostic possibilities considered, and provided a written explanation of how they used key findings to move from their initial differential diagnoses to their final diagnosis. Two physician judges blindly rated responses. RESULTS: Student diagnostic justification performance was highly variable from case to case and often rated below expectations. Of the students in the classes of 2011, 2012, and 2013, 57% (38/67), 23% (15/66), and 33% (26/79) were judged borderline or poor on diagnostic justification performance for more than 50% of the cases on the examination. CONCLUSIONS: Student diagnostic justification performance was inconsistent across the range of cases, common chief complaints, and underlying diagnoses used in this study. More than 20% of students exhibited borderline or poor diagnostic justification performance on more than 50% of the cases. If these results are confirmed in other medical schools, attention needs to be directed to investigating new curricular methods that ensure deliberate practice of these competencies across the spectrum of common chief complaints and diagnoses and do not depend on the available mix of patients.

Pediatric reference ranges for proinflammatory and anti-inflammatory cytokines in cerebrospinal fluid and serum by multiplexed immunoassay.

To define cytokine concentrations and detectability in children with noninflammatory neurological disorders (NIND). The multiplex bead assay technology was used for simultaneous measurement of 34 soluble cytokines/chemokines in cerebrospinal fluid (CSF) from 73 NIND. Sera from 36 healthy children and 37 NIND also were analyzed. In CSF, CXCL10 had the highest concentration; CCL2, CXCL10, and interleukin (IL)-6 were detectable in all samples, and CXCL8, CCL22, CXCL1, IL-16, and IL-1 receptor antagonist were found in ≥50% of the samples. In serum, CXCL1 had the highest concentration; sIL-2Ra, CXCL1, CXCL10, and CCL22 were detectable in all samples, and CCL2, IL-12, CCL5, and granulocyte monocyte colony-stimulating factor (GM-CSF) were found in ≥50% of the samples. The mean CSF:serum ratio for CCL2 was several-fold higher than the rest, with the CXCL10 and CXCL8 ratios also >1. Intercorrelations between CSF cytokines included CCL2 versus CXCL8 and IL-6, and CXCL1 versus CCL22, reflecting both T-helper-1 (Th1)/Th1 and Th1/Th2 relations. Serum correlations included CCL11 versus CCL2, GM-CSF, and IL-4. For serum cytokines, the agreement between healthy children and NIND was good, with the exception of higher CCL4 in NIND. Cytokines in children varied greatly in concentration and detectability, with chemokines predominating in the CSF. These data allow investigators to select their own kit cytokines, instead of manufacturer-selected cytokines, for greater cost-effectiveness and interpretability.

Cytokines, cytokine antagonists, and soluble adhesion molecules in pediatric OMS and other neuroinflammatory disorders.

OBJECTIVE: To test for hypothesized disease- and treatment-induced changes in cytokines and adhesion molecules in children with opsoclonus-myoclonus syndrome (OMS). METHODS: Multiplex bead assay technology was used for simultaneous measurement of 34 soluble cytokines in cerebrospinal fluid (CSF) and serum. Soluble intercellular adhesion molecule-1 (sICAM-1) and vascular cell adhesion molecule-1 (sVCAM-1) were measured by ELISA. In total, there were 388 children (239 OMS, 114 controls, and 35 other inflammatory neurological disorders (OIND)). RESULTS: In untreated OMS, mean CSF IL-6 was elevated 2.3-fold, but 67-fold in OIND, without significant differences in other CSF cytokines. Mean serum concentrations of sIL-2Ra (+50%) and CXCL1 (+70%) (p<0.0001) were also raised. CSF CCL5 was more often detected in untreated OMS than controls (p=0.005), as was serum CCL11 and IL-13 in treated OMS. Mean CSF CCL4 and IL-1Ra were selectively higher in IVIg-treated OMS (p≤0.0001). CSF sICAM-1 was elevated only in OIND (3.3-fold); serum sICAM-1 was higher in untreated OMS (+21%); and sVCAM-1 was not affected. No correlations with OMS severity or duration were identified. CONCLUSIONS: Novel cytokine, cytokine antagonist, and soluble adhesion molecule abnormalities due to OMS or treatment were found. However, the normality of much of the data strengthens previous findings implicating B cell mechanisms.

Key role of CXCL13/CXCR5 axis for cerebrospinal fluid B cell recruitment in pediatric OMS.

To study aberrant B cell trafficking into the CSF in opsoclonus-myoclonus syndrome (OMS), chemoattractants CXCL13 and CXCL12, and B cell frequency and CXCR5 expression, were evaluated. CSF CXCL13 concentration and the CSF/serum ratio were higher in untreated OMS than controls, related directly to OMS severity and inversely to OMS duration, and correlated with CSF B cell frequency and oligoclonal bands. CXCL12 showed the opposite pattern. Selective accumulation of CXCR5+ memory B cells in CSF was found. In ACTH-treated OMS, CXCL13, but not CXCL12, was lower. These data implicate the chemokine/chemoreceptor pair CXCL13/CXR5 in B cell recruitment to the CNS in OMS. CXCL13 and CXCL12 may serve as reciprocal biomarkers of disease activity, but CXCL13 also had utility as a treatment biomarker.

A template for reliable assessment of resident operative performance: assessment intervals, numbers of cases and raters.

BACKGROUND: Operative performance rating (OPR) instruments have been developed to assess operative performance (OP). To guide program implementation, this study determined: 1) Appropriate intervals for OP progress decisions, 2) Number of OPRs and raters required per interval to achieve reproducible results. METHODS: 21 surgeons rated 897 OPs (3 procedures) by 36 residents. Six-month PGY intervals were compared to determine length of stable operative performance intervals. Variance component analyses established rating factor importance. Generalizability analyses and decision studies determined number of OPRs required for reproducible OP decisions (reliabilities = 0.80). RESULTS: Resident OPRs are stable across single PGY years. 2.3 OPRs/resident/month provided a dependable basis for annual or semi-annual resident OP decisions. Results were similar for all procedures and training years. Rater idiosyncrasies accounted for most score variation (63% when interaction effects involving rater idiosyncrasies were included). Resident ability was the next most important source of variation (12%). Procedure was a less important source (5%). CONCLUSION: Annual resident OP decisions are supported. 2.3 OPRs per month provide a dependable basis for judging resident OP. These numbers are sufficient regardless of training year or procedure mix though efforts should be made to balance procedure mix. Multiple raters should rate each resident to control for rater idiosyncrasies.