What was visualized? A method for describing content of performance summary displays in feedback interventions.

BACKGROUND: Visual displays such as charts and tables may significantly moderate the effects of audit and feedback interventions, but the systematic study of these intervention components will likely remain limited without a method for isolating the information content of a visual display from its form elements. The objective of this study is to introduce such a method based on an application of visualization frameworks to enable a systematic approach to answer the question, "What was visualized?" in studies of audit and feedback. METHODS: The proposed method uses 3 steps to systematically identify and describe the content of visual displays in feedback interventions: 1) identify displays, 2) classify content, and 3) identify elements. The use of a visualization framework led us to identify information content types as representations of measures (metrics or indicators), ascribees (feedback recipients and comparators), performance levels, and time intervals. We illustrate the proposed method in a series of 3 content analyses, one for each step, to identify visual displays and their information content in published example performance summaries. RESULTS: We analyzed a convenience sample of 44 published studies of audit and feedback. Through each step, two coders had good agreement. We identified 42 visual displays of performance, containing 6 unique combinations of content types. What was visualized most commonly in the sample was performance levels across a recipient and comparators (i.e. ascribees) for a single measure and single time interval (n = 16). Content types varied in their inclusion of measures, ascribees, and time intervals. CONCLUSIONS: The proposed method appears to be feasible to use as a systematic approach to describing visual displays of performance. The key implication of the method is that it offers more granular and consistent description for empirical, theoretical, and design studies about the information content of feedback interventions.

Exploring Anesthesia Provider Preferences for Precision Feedback: Preference Elicitation Study.

Background: Health care professionals must learn continuously as a core part of their work. As the rate of knowledge production in biomedicine increases, better support for health care professionals' continuous learning is needed. In health systems, feedback is pervasive and is widely considered to be essential for learning that drives improvement. Clinical quality dashboards are one widely deployed approach to delivering feedback, but engagement with these systems is commonly low, reflecting a limited understanding of how to improve the effectiveness of feedback about health care. When coaches and facilitators deliver feedback for improving performance, they aim to be responsive to the recipient's motivations, information needs, and preferences. However, such functionality is largely missing from dashboards and feedback reports. Precision feedback is the delivery of high-value, motivating performance information that is prioritized based on its motivational potential for a specific recipient, including their needs and preferences. Anesthesia care offers a clinical domain with high-quality performance data and an abundance of evidence-based quality metrics. Objective: The objective of this study is to explore anesthesia provider preferences for precision feedback. Methods: We developed a test set of precision feedback messages with balanced characteristics across 4 performance scenarios. We created an experimental design to expose participants to contrasting message versions. We recruited anesthesia providers and elicited their preferences through analysis of the content of preferred messages. Participants additionally rated their perceived benefit of preferred messages to clinical practice on a 5-point Likert scale. Results: We elicited preferences and feedback message benefit ratings from 35 participants. Preferences were diverse across participants but largely consistent within participants. Participants' preferences were consistent for message temporality (α=.85) and display format (α=.80). Ratings of participants' perceived benefit to clinical practice of preferred messages were high (mean rating 4.27, SD 0.77). Conclusions: Health care professionals exhibited diverse yet internally consistent preferences for precision feedback across a set of performance scenarios, while also giving messages high ratings of perceived benefit. A "one-size-fits-most approach" to performance feedback delivery would not appear to satisfy these preferences. Precision feedback systems may hold potential to improve support for health care professionals' continuous learning by accommodating feedback preferences.

Comparability of Pulmonary Nodule Size Measurements among Different Scanners and Protocols: Should Diameter Be Favorized over Volume?

BACKGROUND: To assess the impact of the lung cancer screening protocol recommended by the European Society of Thoracic Imaging (ESTI) on nodule diameter, volume, and density throughout different computed tomography (CT) scanners. METHODS: An anthropomorphic chest phantom containing fourteen different-sized (range 3-12 mm) and CT-attenuated (100 HU, -630 HU and -800 HU, termed as solid, GG1 and GG2) pulmonary nodules was imaged on five CT scanners with institute-specific standard protocols (PS) and the lung cancer screening protocol recommended by ESTI (ESTI protocol, PE). Images were reconstructed with filtered back projection (FBP) and iterative reconstruction (REC). Image noise, nodule density and size (diameter/volume) were measured. Absolute percentage errors (APEs) of measurements were calculated. RESULTS: Using PE, dosage variance between different scanners tended to decrease compared to PS, and the mean differences were statistically insignificant (p = 0.48). PS and PE(REC) showed significantly less image noise than PE(FBP) (p < 0.001). The smallest size measurement errors were noted with volumetric measurements in PE(REC) and highest with diametric measurements in PE(FBP). Volume performed better than diameter measurements in solid and GG1 nodules (p < 0.001). However, in GG2 nodules, this could not be observed (p = 0.20). Regarding nodule density, REC values were more consistent throughout different scanners and protocols. CONCLUSION: Considering radiation dose, image noise, nodule size, and density measurements, we fully endorse the ESTI screening protocol including the use of REC. For size measurements, volume should be preferred over diameter.

CMEIAS color segmentation: an improved computing technology to process color images for quantitative microbial ecology studies at single-cell resolution.

Quantitative microscopy and digital image analysis are underutilized in microbial ecology largely because of the laborious task to segment foreground object pixels from background, especially in complex color micrographs of environmental samples. In this paper, we describe an improved computing technology developed to alleviate this limitation. The system's uniqueness is its ability to edit digital images accurately when presented with the difficult yet commonplace challenge of removing background pixels whose three-dimensional color space overlaps the range that defines foreground objects. Image segmentation is accomplished by utilizing algorithms that address color and spatial relationships of user-selected foreground object pixels. Performance of the color segmentation algorithm evaluated on 26 complex micrographs at single pixel resolution had an overall pixel classification accuracy of 99+%. Several applications illustrate how this improved computing technology can successfully resolve numerous challenges of complex color segmentation in order to produce images from which quantitative information can be accurately extracted, thereby gain new perspectives on the in situ ecology of microorganisms. Examples include improvements in the quantitative analysis of (1) microbial abundance and phylotype diversity of single cells classified by their discriminating color within heterogeneous communities, (2) cell viability, (3) spatial relationships and intensity of bacterial gene expression involved in cellular communication between individual cells within rhizoplane biofilms, and (4) biofilm ecophysiology based on ribotype-differentiated radioactive substrate utilization. The stand-alone executable file plus user manual and tutorial images for this color segmentation computing application are freely available at http://cme.msu.edu/cmeias/ . This improved computing technology opens new opportunities of imaging applications where discriminating colors really matter most, thereby strengthening quantitative microscopy-based approaches to advance microbial ecology in situ at individual single-cell resolution.

Architecture and Initial Development of a Knowledge-as-a-Service Activator for Computable Knowledge Objects for Health.

The Knowledge Grid (KGrid) is a research and development program toward infrastructure capable of greatly decreasing latency between the publication of new biomedical knowledge and its widespread uptake into practice. KGrid comprises digital knowledge objects, an online Library to store them, and an Activator that uses them to provide Knowledge-as-a-Service (KaaS). KGrid's Activator enables computable biomedical knowledge, held in knowledge objects, to be rapidly deployed at Internet-scale in cloud computing environments for improved health. Here we present the Activator, its system architecture and primary functions.

Efficient identification of mutated cancer antigens recognized by T cells associated with durable tumor regressions.

PURPOSE: Cancer immunotherapy with adoptive transfer of tumor-infiltrating lymphocytes (TIL) represents an effective treatment for patients with metastatic melanoma, with the objective regressions in up to 72% of patients in three clinical trials. However, the antigen targets recognized by these effective TILs remain largely unclear. EXPERIMENTAL DESIGN: Melanoma patients 2359 and 2591 both experienced durable complete regressions of metastases ongoing beyond five years following adoptive TIL transfer. Two conventional screening approaches were carried out to identify the antigens recognized by these clinically effective TILs. In addition, a novel approach was developed in this study to identify mutated T-cell antigens by screening a tandem minigene library, which comprised nonsynonymous mutation sequences identified by whole-exome sequencing of autologous tumors. RESULTS: Screening of an autologous melanoma cDNA library using a conventional approach led to the identification of previously undescribed nonmutated targets recognized by TIL 2359 or TIL 2591. In contrast, screening of tandem minigene libraries encoding tumor-specific mutations resulted in the identification of mutated kinesin family member 2C (KIF2C) antigen as a target of TIL 2359, and mutated DNA polymerase alpha subunit B (POLA2) antigen as a target of TIL 2591. Both KIF2C and POLA2 have been found to play important roles in cell proliferation. CONCLUSIONS: These findings suggest that the minigene screening approach can facilitate the antigen repertoire analysis of tumor reactive T cells, and lead to the development of new adoptive cell therapies with purified T cells that recognize candidate-mutated antigens derived from genes essential for the carcinogenesis.

Mining exomic sequencing data to identify mutated antigens recognized by adoptively transferred tumor-reactive T cells.

Substantial regressions of metastatic lesions have been observed in up to 70% of patients with melanoma who received adoptively transferred autologous tumor-infiltrating lymphocytes (TILs) in phase 2 clinical trials. In addition, 40% of patients treated in a recent trial experienced complete regressions of all measurable lesions for at least 5 years following TIL treatment. To evaluate the potential association between the ability of TILs to mediate durable regressions and their ability to recognize potent antigens that presumably include mutated gene products, we developed a new screening approach involving mining whole-exome sequence data to identify mutated proteins expressed in patient tumors. We then synthesized and evaluated candidate mutated T cell epitopes that were identified using a major histocompatibility complex-binding algorithm for recognition by TILs. Using this approach, we identified mutated antigens expressed on autologous tumor cells that were recognized by three bulk TIL lines from three individuals with melanoma that were associated with objective tumor regressions following adoptive transfer. This simplified approach for identifying mutated antigens recognized by T cells avoids the need to generate and laboriously screen cDNA libraries from tumors and may represent a generally applicable method for identifying mutated antigens expressed in a variety of tumor types.

Randomized selection design trial evaluating CD8+-enriched versus unselected tumor-infiltrating lymphocytes for adoptive cell therapy for patients with melanoma.

PURPOSE: Adoptive cell therapy (ACT) with autologous tumor-infiltrating lymphocytes (TILs) and high-dose interleukin-2 (IL-2) administered to lymphodepleted patients with melanoma can cause durable tumor regressions. The optimal TIL product for ACT is unknown. PATIENTS AND METHODS: Patients with metastatic melanoma were prospectively assigned to receive unselected young TILs versus CD8(+)-enriched TILs. All patients received lymphodepleting chemotherapy and high-dose IL-2 therapy and were assessed for response, toxicity, survival, and immunologic end points. RESULTS: Thirty-four patients received unselected young TILs with a median of 8.0% CD4(+) lymphocytes, and 35 patients received CD8(+)-enriched TILs with a median of 0.3% CD4(+) lymphocytes. One month after TIL infusion, patients who received CD8(+)-enriched TILs had significantly fewer CD4(+) peripheral blood lymphocytes (P = .01). Twelve patients responded to therapy with unselected young TILs (according to Response Evaluation Criteria in Solid Tumors [RECIST]), and seven patients responded to CD8(+)-enriched TILs (35% v 20%; not significant). Retrospective studies showed a significant association between response to treatment and interferon gamma secretion by the infused TILs in response to autologous tumor (P = .04), and in the subgroup of patients who received TILs from subcutaneous tumors, eight of 15 patients receiving unselected young TILs responded but none of eight patients receiving CD8(+)-enriched TILs responded. CONCLUSION: A randomized selection design trial was feasible for improving individualized TIL therapy. Since the evidence indicates that CD8(+)-enriched TILs are not more potent therapeutically and they are more laborious to prepare, future studies should focus on unselected young TILs.

Tumor-specific CD4+ melanoma tumor-infiltrating lymphocytes.

Adoptive cell therapy using tumor-infiltrating lymphocytes (TIL) can mediate objective and durable tumor regressions in patients with metastatic melanoma. CD8+ tumor-reactive TIL are well studied in humans and animals, yet the function of tumor-infiltrating CD4+ T lymphocytes in patient treatments remains controversial. We recently demonstrated that CD4+ TILs are not necessary for objective responses in patients. Coinfusion with tumor-specific CD4 TIL may enhance or increase the durability of tumor regressions, but the number of patients with tumor-reactive CD4 TIL is unknown. We screened 44 CD8+-depleted TIL for in vitro reactivity against autologous tumor. Nine (20%) showed specific reactivity by interferon-γ release assay, of which 8 were specifically blocked by an anti-HLA-DR antibody. Flow-cytometric analysis of these reactive TIL confirmed a high CD4+ composition (median 89%). Highlighting the contribution of CD4+ TIL to tumor regression, a patient with widespread metastatic disease was administered TIL containing HLA class II-restricted tumor activity with high-dose interleukin-2 therapy after lymphodepletion that mediated regression of extensive metastatic disease in the liver and spleen. These results demonstrate that at least 20% of metastatic melanomas contain CD4+ lymphocytes with specific tumor recognition and suggest a possible role for CD4+ cells in the effectiveness of adoptive cell therapy.

CD8+ enriched "young" tumor infiltrating lymphocytes can mediate regression of metastatic melanoma.

PURPOSE: Tumor-infiltrating lymphocytes (TIL) and interleukin (IL)-2 administered following lymphodepletion can cause the durable complete regression of bulky metastatic melanoma in patients refractory to approved treatments. However, the generation of a unique tumor-reactive TIL culture for each patient may be prohibitively difficult. We therefore investigated the clinical and immunologic impact of unscreened, CD8+ enriched "young" TIL. EXPERIMENTAL DESIGN: Methods were developed for generating TIL that minimized the time in culture and eliminated the individualized tumor-reactivity screening step. Thirty-three patients were treated with these CD8+ enriched young TIL and IL-2 following nonmyeloablative lymphodepletion (NMA). Twenty-three additional patients were treated with CD8+ enriched young TIL and IL-2 after lymphodepletion with NMA and 6 Gy of total body irradiation. RESULTS: Young TIL cultures for therapy were successfully established from 83% of 122 consecutive melanoma patients. Nineteen of 33 patients (58%) treated with CD8+ enriched young TIL and NMA had an objective response (Response Evaluation Criteria in Solid Tumors) including 3 complete responders. Eleven of 23 patients (48%) treated with TIL and 6 Gy total body irradiation had an objective response including 2 complete responders. At 1 month after TIL infusion the absolute CD8+ cell numbers in the periphery were highly correlated with response. CONCLUSIONS: This study shows that a rapid and simplified method can be used to reliably generate CD8+ enriched young TIL for administration as an individualized therapy for advanced melanoma, and may allow this potentially effective treatment to be applied at other institutions and to reach additional patients.