The "what, why, and how?" of story completion in health services research: a scoping review.

BACKGROUND: The story completion method provides a different way of doing qualitative research. We note the emergent popularity of this method in health-related research, while much remains to be negotiated in terms of best practices for such studies. This scoping review aims to provide a synthesis on how researchers have used the story completion method in health services research. We offer implications for research and practice for further discussion by the scholarly community. METHODS: We used the JBI methodology for scoping reviews. Six databases were searched for published literature till March 1, 2023: Medline, Embase, CINAHL, PsycINFO, SAGE Journals Online databases, and SAGE Research Methods. We included primary studies of any study design using the story completion method in health services research. RESULTS: A total of 17 studies were included. Findings suggest that the story completion method is useful for research on sensitive topics, and affords the use of comparative study designs and large sample sizes which may be difficult with conventional qualitative research methods. More than 80% of included studies used story completion as the sole method. However, the data collected from this method were limited in terms of the inferences that can be drawn; and richness of participant responses may vary widely. Less than 30% of included studies reported piloting of the story stems. Most studies were conducted online and analyzed qualitatively, though the story stem design and sample size varied widely. CONCLUSION: The story completion method, with its attendant affordances for larger sample sizes, comparative study designs, and streamlined data collection is an innovative and useful stand-alone or adjunct qualitative method for health services research.

A theoretical framework for linking hospitals longitudinally: demonstrated using German Hospital Quality Reports 2016-2020.

BACKGROUND: In longitudinal health services research, hospital identification using an ID code, often supplemented with several additional variables, lacks clarity regarding representativeness and variable influence. This study presents an operational method for hospital identity delimitation and a novel longitudinal identification approach, demonstrated using a case study. METHODS: The conceptualisation considers hospitals as evolving entities, identifying "similar enough" pairs across two time points using an automated similarity matrix. This method comprises key variable selection, similarity scoring, and tolerance threshold definition, tailored to data source characteristics and clinical relevance. This linking method is tested by applying the identification of minimum caseload requirements-related German hospitals, utilizing German Hospital Quality Reports (GHQR) 2016-2020. RESULTS: The method achieved a success rate (min: 97.9% - max: 100%, mean: 99.9%) surpassing traditional hospital ID-code linkage (min: 91.5% - max: 98.8%, mean: 96.6%), with a remarkable 99% reduction in manual work through automation. CONCLUSIONS: This method, rooted in a comprehensive understanding of hospital identities, offers an operational, automated, and customisable process serving diverse clinical topics. This approach has the advantage of simultaneously considering multiple variables and systematically observing temporal changes in hospitals. It also enhances the precision and efficiency of longitudinal hospital identification in health services research.

Barriers and facilitators for recruiting and retaining male participants into longitudinal health research: a systematic review.

BACKGROUND: Successfully recruiting male participants to complete a healthcare related study is important for healthcare study completion and to advance our clinical knowledgebase. To date, most research studies have examined the barriers and facilitators of female participants in longitudinal healthcare-related studies with limited information available about the needs of males in longitudinal research. This systematic review examines the unique barriers and facilitators to male recruitment across longitudinal healthcare-related research studies. METHODS: Following PRIMSA guidelines, MEDLINE, Embase, CINAHL and Web of Science databases were systematically searched using the terms recruitment and/or retention, facilitators and/or barriers and longitudinal studies from 1900 to 2023 which contained separate data on males aged 17-59 years. Health studies or interventions were defined longitudinal if they were greater than or equal to 12 weeks in duration with 3 separate data collection visits. RESULTS: Twenty-four articles published from 1976-2023 met the criteria. One-third of the studies had a predominantly male sample and four studies recruited only male participants. Males appear disinterested towards participation in health research, however this lack of enthusiasm can be overcome by clear, non-directive communication, and studies that support the participants interests. Facilitating factors are diverse and may require substantial time from research teams. CONCLUSIONS: Future research should focus on the specific impact of these factors across the spectrum of longitudinal health-related studies. Based on the findings of this systematic review, researchers from longitudinal health-related clinical trials are encouraged to consider male-specific recruitment strategies to ensure successful recruitment and retention in their studies. REGISTRATION: This systemic review is registered with the PROSPERO database (CRD42021254696).

Health Services Research in Anesthesia: A Brief Overview of Common Methodologies.

The use of large data sources such as registries and claims-based data sets to perform health services research in anesthesia has increased considerably, ultimately informing clinical decisions, supporting evaluation of policy or intervention changes, and guiding further research. These observational data sources come with limitations that must be addressed to effectively examine all aspects of health care services and generate new individual- and population-level knowledge. Several statistical methods are growing in popularity to address these limitations, with the goal of mitigating confounding and other biases. In this article, we provide a brief overview of common statistical methods used in health services research when using observational data sources, guidance on their interpretation, and examples of how they have been applied to anesthesia-related health services research. Methods described involve regression, propensity scoring, instrumental variables, difference-in-differences, interrupted time series, and machine learning.

Funnel plots of patient-reported outcomes to evaluate health-care quality: Basic principles, pitfalls and considerations.

A funnel plot is a graphical method to evaluate health-care quality by comparing hospital performances on certain outcomes. So far, in nephrology, this method has been applied to clinical outcomes like mortality and complications. However, patient-reported outcomes (PROs; eg, health-related quality of life [HRQOL]) are becoming increasingly important and should be incorporated into this quality assessment. Using funnel plots has several advantages, including clearly visualized precision, detection of volume-effects, discouragement of ranking hospitals and easy interpretation of results. However, without sufficient knowledge of underlying methods, it is easy to stumble into pitfalls, such as overinterpretation of standardized scores, incorrect direct comparisons of hospitals and assuming a hospital to be in-control (ie, to perform as expected) based on underpowered comparisons. Furthermore, application of funnel plots to PROs is accompanied by additional challenges related to the multidimensional nature of PROs and difficulties with measuring PROs. Before using funnel plots for PROs, high and consistent response rates, adequate case mix correction and high-quality PRO measures are required. In this article, we aim to provide insight into the use and interpretation of funnel plots by presenting an overview of the basic principles, pitfalls and considerations when applied to PROs, using examples from Dutch routine dialysis care.

2017 National Optometry Workforce Survey.

SIGNIFICANCE: Planning for the effective delivery of eye care, on all levels, depends on an accurate and detailed knowledge of the optometric workforce and an understanding of demographic/behavioral trends to meet future needs of the public. PURPOSE: The purposes of this study were to assess the current and future supply of doctors of optometry and to examine in-depth trends related to (1) demographic shifts, (2) sex-based differences, (3) differences in practice behaviors in between self-employed and employed optometrists, and (4) the concept of additional capacity within the profession. METHODS: The 2017 National Optometry Workforce Survey (31 items) was distributed to 4050 optometrists, randomly sampled from a population of 45,033 currently licensed and practicing optometrists listed in the American Optometric Association's Optometry Master Data File. A stratified sampling method was applied to the population of optometrists using primary license state, age, and sex as variables to ensure a representative sample. RESULTS: With a response rate of 29% (1158 responses), the sample ensured a 95% confidence interval with a margin of error of <5%. Key results include finding no significant differences between men and women for hours worked (38.9 vs. 37.5), productivity (patient visits per hour, 2.0 vs. 1.9), or career options/professional growth satisfaction with 65% for both. The data indicate a likely range of additional patient capacity of 2.29 to 2.57 patients per week (5.05 to 5.65 million annually profession-wide). CONCLUSIONS: The optometric workforce for the next decade is projected to grow 0.6 to 0.7% more annually than the U.S. population. The study found additional capacity for the profession more limited than previously suggested. Findings also illustrate an evolving/equitable workforce based on sex, in terms of both productivity and satisfaction. The trend toward employed versus self-employed was marked with 44% reporting they are employed, up from 29% in 2012.

Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data.

Observational healthcare data, such as electronic health records and administrative claims, offer potential to estimate effects of medical products at scale. Observational studies have often been found to be nonreproducible, however, generating conflicting results even when using the same database to answer the same question. One source of discrepancies is error, both random caused by sampling variability and systematic (for example, because of confounding, selection bias, and measurement error). Only random error is typically quantified but converges to zero as databases become larger, whereas systematic error persists independent from sample size and therefore, increases in relative importance. Negative controls are exposure-outcome pairs, where one believes no causal effect exists; they can be used to detect multiple sources of systematic error, but interpreting their results is not always straightforward. Previously, we have shown that an empirical null distribution can be derived from a sample of negative controls and used to calibrate P values, accounting for both random and systematic error. Here, we extend this work to calibration of confidence intervals (CIs). CIs require positive controls, which we synthesize by modifying negative controls. We show that our CI calibration restores nominal characteristics, such as 95% coverage of the true effect size by the 95% CI. We furthermore show that CI calibration reduces disagreement in replications of two pairs of conflicting observational studies: one related to dabigatran, warfarin, and gastrointestinal bleeding and one related to selective serotonin reuptake inhibitors and upper gastrointestinal bleeding. We recommend CI calibration to improve reproducibility of observational studies.

Hybrid prevalence estimation: Method to improve intervention coverage estimations.

Delivering excellent health services requires accurate health information systems (HIS) data. Poor-quality data can lead to poor judgments and outcomes. Unlike probability surveys, which are representative of the population and carry accuracy estimates, HIS do not, but in many countries the HIS is the primary source of data used for administrative estimates. However, HIS are not structured to detect gaps in service coverage and leave communities exposed to unnecessary health risks. Here we propose a method to improve informatics by combining HIS and probability survey data to construct a hybrid estimator. This technique provides a more accurate estimator than either data source alone and facilitates informed decision-making. We use data from vitamin A and polio vaccination campaigns in children from Madagascar and Benin to demonstrate the effect. The hybrid estimator is a weighted average of two measurements and produces SEs and 95% confidence intervals (CIs) for the hybrid and HIS estimators. The estimates of coverage proportions using the combined data and the survey estimates differ by no more than 3%, while decreasing the SE by 1-6%; the administrative estimates from the HIS and combined data estimates are very different, with 3-25 times larger CI, questioning the value of administrative estimates. Estimators of unknown accuracy may lead to poorly formulated policies and wasted resources. The hybrid estimator technique can be applied to disease prevention services for which population coverages are measured. This methodology creates more accurate estimators, alongside measured HIS errors, to improve tracking the public's health.

Providing Information to Support COVID-19 Pandemic Response: Academic Medical Librarians' Roles in Creating an Intelligence Report.

In March 2020, the Dean of the George Washington (GW) University School of Medicine and Health Sciences (SMHS) and the GW COVID-19 Incident Management Team asked the Senior Associate Dean for Clinical Public Health to initiate a daily report that surveyed COVID-19 literature/resources. This COVID-19 Intelligence Report would serve as a concise, authoritative source of COVID-19 information for clinicians, the Incident Management Team, and operational leaders. The Senior Associate Dean established an Intelligence Gathering Team comprised of clinicians and librarians. Himmelfarb librarians facilitated the collection, distribution, and archiving of COVID-19 resources and Intelligence Reports.

Top priorities for the next decade of nursing health services research.

BACKGROUND: The U.S. health care system faces increasing pressures for reform. The importance of nurses in addressing health care delivery challenges cannot be overstated. PURPOSE: To present a Nursing Health Services Research (NHSR) agenda for the 2020s. METHOD: A meeting of an interdisciplinary group of 38 health services researchers to discuss five key challenges facing health care delivery (behavioral health, primary care, maternal/neonatal outcomes, the aging population, health care spending) and identify the most pressing and feasible research questions for NHSR in the coming decade. FINDINGS: Guided by a list of inputs affecting health care delivery (health information technology, workforce, delivery systems, payment, social determinants of health), meeting participants identified 5 to 6 research questions for each challenge. Also, eight cross-cutting themes illuminating the opportunities and barriers facing NHSR emerged. DISCUSSION: The Agenda can act as a foundation for new NHSR - which is more important than ever - in the 2020s.

Questionnaire development for the Lolland-Falster Health Study, Denmark: an iterative and incremental process.

In composing multi-thematic questionnaires for the Lolland-Falster Health Study (LOFUS), we faced a range of challenges, for which we found limited guidance in the literature. LOFUS is a household-based population study covering multiple medical and social research areas and targeting the mixed rural-provincial population of 103,000 persons on the Danish islands Lolland and Falster. Households were randomly selected for invitation. In this paper, we describe and discuss challenges in developing the questionnaires related to stakeholders, content of the questionnaire, and the process itself. The development process was characterised by loops of learning and can be described as an iterative and incremental process. We propose recommendations to researchers and administrators involved in similar development processes, including awareness of the non-linearity and complexity of the process, a need for negotiations and navigation among multiple stakeholders, and acknowledgement of pragmatism as an inherent part of decisions made in the process.

Accuracy of diagnostic classification and clinical utility assessment of ICD-11 compared to ICD-10 in 10 mental disorders: findings from a web-based field study.

In this web-based field study, we compared the diagnostic accuracy and clinical utility of 10 selected mental disorders between the ICD-11 Clinical Descriptions and Diagnostic Guidelines (CDDG) and the ICD-10 CDDG using vignettes in a sample of 928 health professionals from all WHO regions. On average, the ICD-11 CDDG displayed significantly higher diagnostic accuracy (71.9% for ICD-11, 53.2% for ICD-10), higher ease of use, better goodness of fit, higher clarity, and lower time required for diagnosis compared to the ICD-10 CDDG. The advantages of the ICD-11 CDDG were largely limited to new diagnoses in ICD-11. After limiting analyses to diagnoses existing in ICD-11 and ICD-10, the ICD-11 CDDG were only superior in ease of use. The ICD-11 CDDG were not inferior in diagnostic accuracy or clinical utility compared to the ICD-10 CDDG for any of the vignettes. Diagnostic accuracy was consistent across WHO regions and independent of participants' clinical experience. There were no differences between medical doctors and psychologists in diagnostic accuracy, but members of other health professions had greater difficulties in determining correct diagnoses based on the ICD-11 CDDG. In sum, there were no differences in diagnostic accuracy for diagnoses existing in ICD-10 and ICD-11, but the introduction of new diagnoses in ICD-11 has improved the diagnostic classification of some clinical presentations. The favourable clinical utility ratings of the ICD-11 CDDG give reason to expect a positive evaluation by health professionals in the implementation phase of ICD-11. Yet, training in ICD-11 is needed to further enhance the diagnostic accuracy.

[Memorandum Registry for Health Services Research: Update 2019]. / Memorandum Register für die Versorgungsforschung: Update 2019.

Health registries could be used to analyze questions concerning routine practice in healthcare. Therefore, registries are a core method in health services research. The German Network for Health Services Research (Deutsches Netzwerk Versorgungsforschung, DNVF) promotes the quality of registries by scientific exchange, organization of advanced training, and recommendations in the form of a memorandum "Registry for Health Services Research". The current recommendations are an update of the memorandum's first version of 2010. The update describes the capabilities and aims of registries in health services research. Furthermore, it illustrates the state-of-the-art in designing and implementing health registries. The memorandum provides developers the methodological basis to ensure high quality health registries. It further provides users of health registries with insights that enable assessing the quality of data and results of health registries. Finally, funding agencies and health policy actors can use the quality criteria to establish a framework for the financing and legislative requirements for health registries. The memorandum provides first a definition of health registries and presents an overview of their utility in health services research and health care improvement. Second, several areas of methodological importance for the development and operation of health registries are presented. This includes the conceptual and preliminary design, implementation, technical organization of a health registry, statistical analysis, reporting of results, and data protection. From these areas, criteria are deduced to allow the assessment of the quality of a health registry. Finally, a checklist is presented.

Development of Primary Care Research in North America, Europe, and Australia From 1974 to 2017.

Research is a necessity for high-quality medicine. We used the MEDLINE database to conduct a bibliometric analysis of research output with respect to primary care by 21 countries. For the period 1974 to 2017, the United States and the United Kingdom stood out in terms of publication volume, and the UK, Canada, and Australia had the greatest percentage of publications in primary care. As of 2017, publications in primary care represented a small proportion of total publications. The countries with the greatest publication productivity possess factors that should be considered with respect to strengthening research in primary care.

How to analyse longitudinal data from multiple sources in qualitative health research: the pen portrait analytic technique.

BACKGROUND: Longitudinal qualitative research is starting to be used in applied health research, having been popular in social research for several decades. There is potential for a large volume of complex data to be captured, over a span of months or years across several different methods. How to analyse this volume of data - with its inherent complexity - represents a problem for health researchers. There is a previous dearth of methodological literature which describes an appropriate analytic process which can be readily employed. METHODS: We document a worked example of the Pen Portrait analytic process, using the qualitative dataset for which the process was originally developed. RESULTS: Pen Portraits are recommended as a way in which longitudinal health research data can be concentrated into a focused account. The four stages of undertaking a pen portrait are: 1) understand and define what to focus on 2) design a basic structure 3) populate the content 4) interpretation. Instructive commentary and guidance is given throughout with consistent reference to the original study for which Pen Portraits were devised. The Pen Portrait analytic process was developed by the authors, borne out of a need to effectively integrate multiple qualitative methods collected over time. Pen Portraits are intended to be adaptable and flexible, in order to meet the differing analytic needs of qualitative longitudinal health studies. CONCLUSIONS: The Pen Portrait analytic process provides a useful framework to enable researchers to conduct a robust analysis of multiple sources of qualitative data collected over time.

Attempting rigour and replicability in thematic analysis of qualitative research data; a case study of codebook development.

BACKGROUND: Navigating the world of qualitative thematic analysis can be challenging. This is compounded by the fact that detailed descriptions of methods are often omitted from qualitative discussions. While qualitative research methodologies are now mature, there often remains a lack of fine detail in their description both at submitted peer reviewed article level and in textbooks. As one of research's aims is to determine the relationship between knowledge and practice through the demonstration of rigour, more detailed descriptions of methods could prove useful. Rigour in quantitative research is often determined through detailed explanation allowing replication, but the ability to replicate is often not considered appropriate in qualitative research. However, a well described qualitative methodology could demonstrate and ensure the same effect. METHODS: This article details the codebook development which contributed to thematic analysis of qualitative data. This analysis formed part of a mixed methods multiphase design research project, with both qualitative and quantitative inquiry and involving the convergence of data and analyses. This design consisted of three distinct phases: quantitative, qualitative and implementation phases. RESULTS AND CONCLUSIONS: This article is aimed at researchers and doctoral students new to thematic analysis by describing a framework to assist their processes. The detailed description of the methods used supports attempts to utilise the thematic analysis process and to determine rigour to support the establishment of credibility. This process will assist practitioners to be confident that the knowledge and claims contained within research are transferable to their practice. The approach described within this article builds on, and enhances, current accepted models.

A methodological systematic review of meta-ethnography conduct to articulate the complex analytical phases.

BACKGROUND: Decision making in health and social care requires robust syntheses of both quantitative and qualitative evidence. Meta-ethnography is a seven-phase methodology for synthesising qualitative studies. Developed in 1988 by sociologists in education Noblit and Hare, meta-ethnography has evolved since its inception; it is now widely used in healthcare research and is gaining popularity in education research. The aim of this article is to provide up-to-date, in-depth guidance on conducting the complex analytic synthesis phases 4 to 6 of meta-ethnography through analysis of the latest methodological evidence. METHODS: We report findings from a methodological systematic review conducted from 2015 to 2016. Fourteen databases and five other online resources were searched. Expansive searches were also conducted resulting in inclusion of 57 publications on meta-ethnography conduct and reporting from a range of academic disciplines published from 1988 to 2016. RESULTS: Current guidance on applying meta-ethnography originates from a small group of researchers using the methodology in a health context. We identified that researchers have operationalised the analysis and synthesis methods of meta-ethnography - determining how studies are related (phase 4), translating studies into one another (phase 5), synthesising translations (phase 6) and line of argument synthesis - to suit their own syntheses resulting in variation in methods and their application. Empirical research is required to compare the impact of different methods of translation and synthesis. Some methods are potentially better at preserving links with the context and meaning of primary studies, a key principle of meta-ethnography. A meta-ethnography can and should include reciprocal and refutational translation and line of argument synthesis, rather than only one of these, to maximise the impact of its outputs. CONCLUSION: The current work is the first to articulate and differentiate the methodological variations and their application for different purposes and represents a significant advance in the understanding of the methodological application of meta-ethnography.

A co-produced method to involve service users in research: the SUCCESS model.

BACKGROUND: Public and patient involvement is a routine element of health services research methods to produce better designed and reported studies. Although co-production is recommended when involving people in research, methods for involving people are usually designed and managed by researchers and there is little evidence about methods to co-produce models for effective public and patient involvement. We report the method used by a group of patient and carer service users to develop and implement a model for involving public members in research. METHOD: We recruited people with experience of chronic conditions, as patients and carers, and supported them to develop and implement the involvement model. We collected written records to describe the processes of co-production. RESULTS: Sixteen service users were involved through a series of workshop, meeting and email discussions. They specified principles and operating characteristics of the model which concerned an inclusive culture, adequate resources, accessibility, good communication and clarity of purpose and roles. Components of the model included an on-line Panel of members (n = 20), Steering Group meetings, representation and communication system, facilitator, supportive research environment and access to research activities. Over 8 years, members were active in 218 research activities and held 22 Steering Group meetings. The model was named SUCCESS standing for Service Users with Chronic Conditions Encouraging Sensible Solutions. CONCLUSION: We supported patients and carers to co-produce the SUCCESS model of involvement in research. The model's components, addressing their needs and priorities, led to sustained involvement in research over 8 years. Further work is needed to apply the model in different settings and assess impact of this method of involving people in research.

Advancing complexity science in healthcare research: the logic of logic models.

BACKGROUND: Logic models are commonly used in evaluations to represent the causal processes through which interventions produce outcomes, yet significant debate is currently taking place over whether they can describe complex interventions which adapt to context. This paper assesses the logic models used in healthcare research from a complexity perspective. A typology of existing logic models is proposed, as well as a formal methodology for deriving more flexible and dynamic logic models. ANALYSIS: Various logic model types were tested as part of an evaluation of a complex Patient Experience Toolkit (PET) intervention, developed and implemented through action research across six hospital wards/departments in the English NHS. Three dominant types of logic model were identified, each with certain strengths but ultimately unable to accurately capture the dynamics of PET. Hence, a fourth logic model type was developed to express how success hinges on the adaption of PET to its delivery settings. Aspects of the Promoting Action on Research Implementation in Health Services (PARIHS) model were incorporated into a traditional logic model structure to create a dynamic "type 4" logic model that can accommodate complex interventions taking on a different form in different settings. CONCLUSION: Logic models can be used to model complex interventions that adapt to context but more flexible and dynamic models are required. An implication of this is that how logic models are used in healthcare research may have to change. Using logic models to forge consensus among stakeholders and/or provide precise guidance across different settings will be inappropriate in the case of complex interventions that adapt to context. Instead, logic models for complex interventions may be targeted at facilitators to enable them to prospectively assess the settings they will be working in and to develop context-sensitive facilitation strategies. Researchers should be clear as to why they are using a logic model and experiment with different models to ensure they have the correct type.

A research tool for measuring non-participation of older people in research on digital health.

BACKGROUND: Healthcare services are being increasingly digitalised in European countries. However, in studies evaluating digital health technology, some people are less likely to participate than others, e.g. those who are older, those with a lower level of education and those with poorer digital skills. Such non-participation in research - deriving from the processes of non-recruitment of targeted individuals and self-selection - can be a driver of old-age exclusion from new digital health technologies. We aim to introduce, discuss and test an instrument to measure non-participation in digital health studies, in particular, the process of self-selection. METHODS: Based on a review of the relevant literature, we designed an instrument - the NPART survey questionnaire - for the analysis of self-selection, covering five thematic areas: socioeconomic factors, self-rated health and subjective overall quality of life, social participation, time resources, and digital skills and use of technology. The instrument was piloted on 70 older study persons in Sweden, approached during the recruitment process for a trial study. RESULTS: Results indicated that participants, as compared to decliners, were on average slightly younger and more educated, and reported better memory, higher social participation, and higher familiarity with and greater use of digital technologies. Overall, the survey questionnaire was able to discriminate between participants and decliners on the key aspects investigated, along the lines of the relevant literature. CONCLUSIONS: The NPART survey questionnaire can be applied to characterise non-participation in digital health research, in particular, the process of self-selection. It helps to identify underrepresented groups and their needs. Data generated from such an investigation, combined with hospital registry data on non-recruitment, allows for the implementation of improved sampling strategies, e.g. focused recruitment of underrepresented groups, and for the post hoc adjustment of results generated from biased samples, e.g. weighting procedures.