Practical Benefits, Challenges, and Recommendations on Social Media Recruitment: Multi-Stakeholder Interview Study.

BACKGROUND: The increasing use of social media opens new opportunities for recruiting patients for research studies. However, systematic evaluations indicate that the success of social media recruitment in terms of cost-effectiveness and representativeness depends on the type of study and its purpose. OBJECTIVE: This study aims to explore the practical benefits and challenges of recruiting study participants with social media in the context of clinical and nonclinical studies and provide a summary of expert advice on how to conduct social media-based recruitment. METHODS: We conducted semistructured interviews with 6 patients with hepatitis B who use social media and 30 experts from the following disciplines: (1) social media researchers or social scientists, (2) practical experts for social media recruitment, (3) legal experts, (4) ethics committee members, and (5) clinical researchers. The interview transcripts were analyzed using thematic analysis. RESULTS: We found diverging expert opinions regarding the challenges and benefits of social media recruitment for research studies in four domains: (1) resources needed, (2) representativeness, (3) web-based community building, and (4) privacy considerations. Moreover, the interviewed experts provided practical advice on how to promote a research study via social media. CONCLUSIONS: Even though recruitment strategies should always be sensitive to individual study contexts, a multiplatform approach (recruiting via several different social media platforms) with mixed-methods recruitment (web-based and offline recruitment channels) is the most beneficial recruitment strategy for many research studies. The different recruitment methods complement each other and may contribute to improving the reach of the study, the recruitment accrual, and the representativeness of the sample. However, it is important to assess the context- and project-specific appropriateness and usefulness of social media recruitment before designing the recruitment strategy.

Ethical Issues in Social Media Recruitment for Clinical Studies: Ethical Analysis and Framework.

BACKGROUND: Social media recruitment for clinical studies holds the promise of being a cost-effective way of attracting traditionally marginalized populations and promoting patient engagement with researchers and a particular study. However, using social media for recruiting clinical study participants also poses a range of ethical issues. OBJECTIVE: This study aims to provide a comprehensive overview of the ethical benefits and risks to be considered for social media recruitment in clinical studies and develop practical recommendations on how to implement these considerations. METHODS: On the basis of established principles of clinical ethics and research ethics, we reviewed the conceptual and empirical literature for ethical benefits and challenges related to social media recruitment. From these, we derived a conceptual framework to evaluate the eligibility of social media use for recruitment for a specific clinical study. RESULTS: We identified three eligibility criteria for social media recruitment for clinical studies: information and consent, risks for target groups, and recruitment effectiveness. These criteria can be used to evaluate the implementation of a social media recruitment strategy at its planning stage. We have discussed the practical implications of these criteria for researchers. CONCLUSIONS: The ethical challenges related to social media recruitment are context sensitive. Therefore, social media recruitment should be planned rigorously, taking into account the target group, the appropriateness of social media as a recruitment channel, and the resources available to execute the strategy.

Correction: Ethical Issues in Social Media Recruitment for Clinical Studies: Ethical Analysis and Framework.

[This corrects the article DOI: 10.2196/31231.].

Out-of-distribution detection with in-distribution voting using the medical example of chest x-ray classification.

BACKGROUND: Deep learning models are being applied to more and more use cases with astonishing success stories, but how do they perform in the real world? Models are typically tested on specific cleaned data sets, but when deployed in the real world, the model will encounter unexpected, out-of-distribution (OOD) data. PURPOSE: To investigate the impact of OOD radiographs on existing chest x-ray classification models and to increase their robustness against OOD data. METHODS: The study employed the commonly used chest x-ray classification model, CheXnet, trained on the chest x-ray 14 data set, and tested its robustness against OOD data using three public radiography data sets: IRMA, Bone Age, and MURA, and the ImageNet data set. To detect OOD data for multi-label classification, we proposed in-distribution voting (IDV). The OOD detection performance is measured across data sets using the area under the receiver operating characteristic curve (AUC) analysis and compared with Mahalanobis-based OOD detection, MaxLogit, MaxEnergy, self-supervised OOD detection (SS OOD), and CutMix. RESULTS: Without additional OOD detection, the chest x-ray classifier failed to discard any OOD images, with an AUC of 0.5. The proposed IDV approach trained on ID (chest x-ray 14) and OOD data (IRMA and ImageNet) achieved, on average, 0.999 OOD AUC across the three data sets, surpassing all other OOD detection methods. Mahalanobis-based OOD detection achieved an average OOD detection AUC of 0.982. IDV trained solely with a few thousand ImageNet images had an AUC 0.913, which was considerably higher than MaxLogit (0.726), MaxEnergy (0.724), SS OOD (0.476), and CutMix (0.376). CONCLUSIONS: The performance of all tested OOD detection methods did not translate well to radiography data sets, except Mahalanobis-based OOD detection and the proposed IDV method. Consequently, training solely on ID data led to incorrect classification of OOD images as ID, resulting in increased false positive rates. IDV substantially improved the model's ID classification performance, even when trained with data that will not occur in the intended use case or test set (ImageNet), without additional inference overhead or performance decrease in the target classification. The corresponding code is available at https://gitlab.lrz.de/IP/a-knee-cannot-have-lung-disease.

Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification.

Purpose: To investigate the chest radiograph classification performance of vision transformers (ViTs) and interpretability of attention-based saliency maps, using the example of pneumothorax classification. Materials and Methods: In this retrospective study, ViTs were fine-tuned for lung disease classification using four public datasets: CheXpert, Chest X-Ray 14, MIMIC CXR, and VinBigData. Saliency maps were generated using transformer multimodal explainability and gradient-weighted class activation mapping (GradCAM). Classification performance was evaluated on the Chest X-Ray 14, VinBigData, and Society for Imaging Informatics in Medicine-American College of Radiology (SIIM-ACR) Pneumothorax Segmentation datasets using the area under the receiver operating characteristic curve (AUC) analysis and compared with convolutional neural networks (CNNs). The explainability methods were evaluated with positive and negative perturbation, sensitivity-n, effective heat ratio, intra-architecture repeatability, and interarchitecture reproducibility. In the user study, three radiologists classified 160 chest radiographs with and without saliency maps for pneumothorax and rated their usefulness. Results: ViTs had comparable chest radiograph classification AUCs compared with state-of-the-art CNNs: 0.95 (95% CI: 0.94, 0.95) versus 0.83 (95%, CI 0.83, 0.84) on Chest X-Ray 14, 0.84 (95% CI: 0.77, 0.91) versus 0.83 (95% CI: 0.76, 0.90) on VinBigData, and 0.85 (95% CI: 0.85, 0.86) versus 0.87 (95% CI: 0.87, 0.88) on SIIM-ACR. Both saliency map methods unveiled a strong bias toward pneumothorax tubes in the models. Radiologists found 47% of the attention-based and 39% of the GradCAM saliency maps useful. The attention-based methods outperformed GradCAM on all metrics. Conclusion: ViTs performed similarly to CNNs in chest radiograph classification, and their attention-based saliency maps were more useful to radiologists and outperformed GradCAM.Keywords: Conventional Radiography, Thorax, Diagnosis, Supervised Learning, Convolutional Neural Network (CNN) Online supplemental material is available for this article. © RSNA, 2023.