Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks.

Natural language understanding (NLU) has made massive progress driven by large benchmarks, but benchmarks often leave a long tail of infrequent phenomena underrepresented. We reflect on the question: Have transfer learning methods sufficiently addressed the poor performance of benchmark-trained models on the long tail? We conceptualize the long tail using macro-level dimensions (underrepresented genres, topics, etc.), and perform a qualitative meta-analysis of 100 representative papers on transfer learning research for NLU. Our analysis asks three questions: (i) Which long tail dimensions do transfer learning studies target? (ii) Which properties of adaptation methods help improve performance on the long tail? (iii) Which methodological gaps have greatest negative impact on long tail performance? Our answers highlight major avenues for future research in transfer learning for the long tail. Lastly, using our meta-analysis framework, we perform a case study comparing the performance of various adaptation methods on clinical narratives, which provides interesting insights that may enable us to make progress along these future avenues.

Factors Affecting Health Care Engagement of Patients With End-Stage Heart Failure: An Exploratory Survey Study.

Background. The decision to receive a permanent left ventricular assist device (LVAD) to treat end-stage heart failure (HF) involves understanding and weighing the risks and benefits of a highly invasive treatment. The goal of this study was to characterize end-stage HF patients across parameters that may affect their decision making and to inform the development of an LVAD decision support tool. Methods. A survey of 35 end-stage HF patients at an LVAD implant hospital was performed to characterize their information-seeking habits, interaction with physicians, technology use, numeracy, and concerns about their health. Survey responses were analyzed using descriptive statistics, grounded theory method, and Bayesian network learning. Results. Most patients indicated an interest in using some type of decision support tool (roadmap of health progression: 46%, n = 16; personal prognosis: 51%, n = 18; short videos of patients telling stories of their experiences with an LVAD: 57%, n = 20). Information patients desired in a hypothetical decision support tool fell into the following topics: prognoses for health outcomes, technical information seeking, expressing emotions, and treatment decisions. Desire for understanding their condition was closely related to whether they had difficult interpreting their electronic medical record in the past. Conclusions. Most patients reported interest in engaging in their health care decision making and seeing their prognosis and electronic health record information. Patients who were less interested in their own treatment decisions were characterized by having less success understanding their health information. Design of a decision support tool for potential LVAD patients should consider a spectrum of health literacy and include information beyond the technical specifications of LVAD support.