Evaluation of reproducible and transparent research practices in pulmonology.

BACKGROUND: Study reproducibility is valuable for validating or refuting results. Provision of reproducibility indicators, such as materials, protocols, and raw data in a study improve its potential for reproduction. Efforts to reproduce noteworthy studies in the biomedical sciences have resulted in an overwhelming majority of them being found to be unreplicable, causing concern for the integrity of research in other fields, including medical specialties. Here, we analyzed the reproducibility of studies in the field of pulmonology. METHODS: 500 pulmonology articles were randomly selected from an initial PubMed search for data extraction. Two authors scoured these articles for reproducibility indicators including materials, protocols, raw data, analysis scripts, inclusion in systematic reviews, and citations by replication studies as well as other factors of research transparency including open accessibility, funding source and competing interest disclosures, and study preregistration. FINDINGS: Few publications included statements regarding materials (10%), protocols (1%), data (15%), and analysis script (0%) availability. Less than 10% indicated preregistration. More than half of the publications analyzed failed to provide a funding statement. Conversely, 63% of the publications were open access and 73% included a conflict of interest statement. INTERPRETATION: Overall, our study indicates pulmonology research is currently lacking in efforts to increase replicability. Future studies should focus on providing sufficient information regarding materials, protocols, raw data, and analysis scripts, among other indicators, for the sake of clinical decisions that depend on replicable or refutable results from the primary literature.

Discrepancy between expected and actual child support payments: predicting the health and health-related quality of life of children living in low-income, single-parent families.

OBJECTIVE: Although the amount and frequency of child support payments received by single parents are often erratic and fluctuate, no study to date has quantitatively explored how the discrepancy between expected and actual payments relates to child health. This study aims to examine whether the discrepancy between expected and actual child support payments predicts a range of child health outcomes, including global health, health-related quality of life, involvement in activities and parental psychological distress. METHODS: This study used results from the Longitudinal Study of Australian Children, which included a sample of parents of children aged 4-5 years (n = 4983). The questionnaire was completed by the parent who spent the most time with the child and knew the child best. From the 4983 families, 332 low-income single parents reliant on welfare with a formal or informal child support order in place were identified. RESULTS: After controlling for income, the discrepancy between expected and actual child support predicted school functioning, conduct problems, total mental health problems and involvement in activities. Discrepancy between expected and actual child support payments did not predict the remaining health-related quality of life domains, mental health domains, global child health or parental psychosocial distress. CONCLUSION: This was the first study to examine how the discrepancy between expected and actual child support payments relates to child health, providing important data on the effectiveness of the child support system for children's well-being. These findings highlight the potential impact of the discrepancy on school functioning, conduct problems, total mental health problems and involvement in activities.

Getting the complete picture: combining parental and child data to identify the barriers to social inclusion for children living in low socio-economic areas.

BACKGROUND: Childhood mental health problems are prevalent in Australian children (14-20%). Social exclusion is a risk factor for mental health problems, whereas being socially included can have protective effects. This study aims to identify the barriers to social inclusion for children aged 9-12 years living in low socio-economic status (SES) areas, using both child-report and parent-report interviews. METHODS: Australian-born English-speaking parents and children aged 9-12 years were sampled from a low SES area to participate in semi-structured interviews. Parents and children were asked questions around three prominent themes of social exclusion; exclusion from school, social activities and social networks. RESULTS: Many children experienced social exclusion at school, from social activities or within social networks. Overall, nine key barriers to social inclusion were identified through parent and child interviews, such as inability to attend school camps and participate in school activities, bullying and being left out, time and transport constraints, financial constraints and safety and traffic concerns. Parents and children often identified different barriers. DISCUSSION: There are several barriers to social inclusion for children living in low SES communities, many of which can be used to facilitate mental health promotion programmes. Given that parents and children may report different barriers, it is important to seek both perspectives. CONCLUSION: This study strengthens the evidence base for the investments and action required to bring about the conditions for social inclusion for children living in low SES communities.

Perspectives of a nurse, a social worker and a psychiatrist regarding patient assessment in acute inpatient psychiatry settings: a case study approach.

This case study explores what informs and organizes the assessment of patients, as undertaken by a nurse, a social worker and a psychiatrist in public, metropolitan, acute mental health service settings. The research data are the transcripts of in-depth interviews with three experienced practitioners, one from each of the three disciplines. The analysis draws on Foucauldian concepts: discourse as constructed through practices of discipline and the gaze. We explored examples of taken-for-granted assessment practices and their interplay with discourse. The findings suggest that participating practitioners use language in assessment in ways that support the powerful discourses of the professional disciplines. The competing discourse of management, associated with industry and economics, is evident in hospital admission processes, dictating the times and places of assessment. Professional and management discourses both effectively marginalize the perspective of another player in assessment, the patient.

A psychometric analysis of the measurement level of the rating scale, time trade-off, and standard gamble.

A fundamental assumption of utility-based analyses is that patient utilities for health states can be measured on an equal-interval scale. This assumption, however, has not been widely examined. The objective of this study was to assess whether the rating scale (RS), standard gamble (SG), and time trade-off (TTO) utility elicitation methods function as equal-interval level scales. We wrote descriptions of eight prostate-cancer-related health states. In interviews with patients who had newly diagnosed, advanced prostate cancer, utilities for the health states were elicited using the RS, SG, and TTO methods. At the time of the study, 77 initial and 73 follow-up interviews had. been conducted with a consecutive sample of 77 participants. Using a Rasch model, the boundaries (Thurstone Thresholds) between four equal score sub-ranges of the raw utilities were mapped onto an equal-interval logit scale. The distance between adjacent thresholds in logit units was calculated to determine whether the raw utilities were equal-interval. None of the utility scales functioned as interval-level scales in our sample. Therefore, since interval-level estimates are assumed in utility-based analyses, doubt is raised regarding the validity of findings from previous analyses based on these scales. Our findings need to be replicated in other contexts, and the practical impact of non-interval measurement on utility-based analyses should be explored. If cost-effectiveness analyses are not found to be robust to violations of the assumption that utilities are interval, serious doubt will be cast upon findings from utility-based analyses and upon the wisdom of expending millions in research dollars on utility-based studies.

SODA (severity of dyspepsia assessment): a new effective outcome measure for dyspepsia-related health.

The aim of this research was to develop and evaluate an instrument for measuring dyspepsia-related health to serve as the primary outcome measure for randomized clinical trials. Building on our previous work we developed SODA (Severity of Dyspepsia Assessment), a multidimensional dyspepsia measure. We evaluated SODA by administering it at enrollment and seven follow-up visits to 98 patients with dyspepsia who were randomized to a 6-week course of omeprazole versus placebo and followed over 1 year. The mean age was 53 years, and six patients (6%) were women. Median Cronbach's alpha reliability estimates over the eight visits for the SODA Pain Intensity, Non-Pain Symptoms, and Satisfaction scales were 0.97, 0.90, and 0.92, respectively. The mean change scores for all three scales discriminated between patients who reported they were improved versus those who were unchanged, providing evidence of validity. The effect sizes for the Pain Intensity (.98) and Satisfaction (.87) scales were large, providing evidence for responsiveness. The effect size for the Non-Pain Symptoms scale was small (.24), indicating lower responsiveness in this study sample. SODA is a new, effective instrument for measuring dyspepsia-related health. SODA is multidimensional and responsive to clinically meaningful change with demonstrated reliability and validity.

A trial for comparing methods for eliciting treatment preferences from men with advanced prostate cancer: results from the initial visit.

OBJECTIVE: The objective of this study was to evaluate the convergent validity of 3 types of utility measures: standard gamble, time tradeoff, and rating scale. RESEARCH DESIGN: A prospective cohort of 120 men with advanced prostate cancer were first asked to rank order 8 health states, and then utility values were obtained from each participant for each of the 8 health states through 2 of the 3 techniques evaluated (standard gamble, time tradeoff and rating scale). Participants were randomly assigned to 1 of 3 possible pairs of techniques. The validity of the 3 methods, as measured by the convergence and raw score differences of the techniques, was assessed with ANOVA. The ability of the techniques to differentiate health states was determined. The inconsistencies between rankings and utility values were also measured. Proportions of illogical utility responses were assessed as the percent of times when states with more symptoms were given higher or equal utility values than states with fewer symptoms. RESULTS: There were significant differences in raw scores between techniques, but the values were correlated across health states. Utility values were often inconsistent with the rank order of health states. In addition, utility assessment did not differentiate the health states as well as the rank order. Furthermore, utility values were often illogical in that states with more symptoms received equal or higher utility values than states with fewer symptoms. CONCLUSIONS: Use of the utility techniques in cost-effectiveness analysis and decision making has been widely recommended. The results of this study raise serious questions as to the validity and usefulness of the measures.

Assessing the performance of utility techniques in the absence of a gold standard.

BACKGROUND: Utility techniques are the most commonly used means to assess patient preferences for health outcomes. However, whether utility techniques produce valid measures of preference has been difficult to determine in the absence of a gold standard. OBJECTIVE: To introduce and demonstrate two methods that can be used to evaluate how well utility techniques measure patients' preferences. SUBJECTS AND DESIGN: Patients treated for advanced prostate cancer (n = 57) first ranked eight health states in order of preference. Four utility techniques were then used to elicit patients' utilities for each health state. MEASURES: The rating scale, standard gamble, time trade-off, and a modified version of willingness-to-pay techniques were used to elicit patients' utilities. Technique performance was assessed by computing a differentiation and inconsistency score for each technique. RESULTS: Differentiation scores indicated the rating scale permitted respondents to assign unique utility values to about 70% of the health states that should have received unique values. When the other techniques were used, about 40% or less of the health states that should have received unique utility scores actually did receive unique utility scores. Inconsistency scores, which indicate how often participants assign utility scores that contradict how they value health states, indicated that the willingness-to-pay technique produced the lowest rate of inconsistency (10%). However, this technique did not differ significantly from the rating scale or standard gamble on this dimension. CONCLUSIONS: Differentiation and inconsistency offer a means to evaluate the performance of utility techniques, thereby allowing investigators to determine the extent to which utilities they have elicited for a given decision problem are valid. In the current investigation, the differentiation and inconsistency methods indicated that all four techniques performed at sub-optimal levels, though the rating scale out-performed the standard gamble, time trade-off, and willingness-to-pay techniques.

Evaluation of a multidimensional measure of dyspepsia-related health for use in a randomized clinical trial.

In previous work, we developed a multidimensional measure of dyspepsia-related health. To evaluate the adequacy of this instrument as an outcome measure for a large-scale, multicenter, randomized clinical trial, we used Rasch analysis to address three questions: (1) Are the scales interval-level? (2) Do the scales measure precisely across the entire range of dyspepsia outcomes? (3) Do the scales' items have an optimal number of response categories? We found that the scales were not interval-level and that they did not measure effectively at low or high levels of the dyspepsia-related outcomes. Our results also suggest that patients were capable of discriminating among only four- to seven-item response categories. Further studies are needed to identify items that effectively measure high and low levels of dyspepsia-related outcomes and to validate that decreasing the number of response categories improves the psychometric properties of these scales.

Frozen in time: life in the face of chronic care cutbacks.

Kathy Cook won the $750 first prize in CMAJ's 7th Annual Amy Chouinard Memorial Essay Contest. The deadline for entries to the contest, which is designed to stimulate interest in medical writing among journalism students, is June 1. Entries should be forwarded to the news and features editor. In her winning essay, Cook explores the frustrations and quality-of-life issues that arise in a chronic care institution that is trying to operate in the midst of serious funding cuts.

Assessment of potential inhalant use by students.

Inhalant use is a significant health risk for today's youth, so school nurses need to be aware of its prevalence and life-threatening consequences. Numerous, readily available, inexpensive substances are being inhaled by a growing number of students. The effects of inhaling such substances can be devastating.

A theory of organizational response to hospital regulation: a reply to Smith and Mick.

Smith and Mick identify four basic problems with the theory the present writers developed to explain organizational responses (in this case the behavior of hospitals) to regulation. They challenge the basic assumption regarding autonomy, disagree with the implied cause and effect relations between organizational response and regulation, criticize the omission of goals, and claim that the theory has only limited generality. In so doing they state that their primary concern is with "improving our understanding of the limitations and benefits of the theory." Each of the four topics they raise for consideration will receive comment.

Hospital rate review. A theory and an empirical review.

This paper presents a theory of the effects of rate review on hospital operations and organization. Its purpose is to explain the way in which hospitals have responded to regulation. In the development of this theory, the hospital product was viewed as a bundle of services, rate review was looked upon as a ceiling on the value of the bundle. The ceiling creates an incentive to remove elements from the bundle, i.e., to reduce 'quality'. When quality is variable, the effect on utilization becomes indeterminate. The model argues, among other things, that the hospital will change its service complement and its contractural arrangements with physicians and other hospitals. An extension of the organizational theory literature leads to implications concerning the ordering of hospital responses to regulation. The growing body of empirical literature on the effects of hospital rate review is used as an initial test of the major thrusts of the theory. A suggested agenda for further empirical work also is presented.

A theory of organizational response to regulation: the case of hospitals.

This paper presents a general theory of organizational response to regulation, a theory that integrates adaptation and mutual selection perspectives. Two major forms of regulation in the hospital industry, certificate of need and rate review, are examined. Hypotheses are derived concerning the nature and timing of the various adjustments hospitals make both in internal organizational arrangements and in patterns of interorganizational activity in the face of regulatory constraints. Suggestions and data sources for testing the theory are presented.

Impairment prevention in the training years. A new mental health program at UCLA.

The new Mental Health Program for Physicians in Training, at UCLA, offers free psychiatric evaluation and short-term psychotherapy to UCLA medical students and house officers, utilizing a large group of physician volunteers from the clinical and regular faculties. The program features complete confidentiality and an off-campus setting and provides help in a number of specialized areas. It also conducts research on stress in medical education. A basic aim is to promote a sense of community in the medical profession by demonstrating the concern of mature practitioners for their young colleagues. It also seeks to foster greater sensitivity among young physicians by educating them about the importance of their own psychological needs. The UCLA program provides a model for mental health services that are low in cost, both to medical trainees and to the training institution, at a time of severe budget shrinkage.