Clinical significance in nursing research: A discussion and descriptive analysis.

BACKGROUND: It is widely understood that statistical significance should not be equated with clinical significance, but the topic of clinical significance has not received much attention in the nursing literature. By contrast, interest in conceptualizing and operationalizing clinical significance has been a "hot topic" in other health care fields for several decades. OBJECTIVES: The major purpose of this paper is to briefly describe recent advances in defining and quantifying clinical significance. The overview covers both group-level indicators of clinical significance (e.g., effect size indexes), and individual-level benchmarks (e.g., the minimal important change index). A secondary purpose is to describe the extent to which developments in clinical significance have penetrated the nursing literature. METHODS: A descriptive analysis of a sample of primary research articles published in three high-impact nursing research journals in 2016 was undertaken. A total of 362 articles were electronically searched for terms relating to statistical and clinical significance. RESULTS: Of the 362 articles, 261 were reports of quantitative studies, the vast majority of which (93%) included a formal evaluation of the statistical significance of the results. By contrast, the term "clinical significance" or related surrogate terms were found in only 33 papers, and most often the term was used informally, without explicit definition or assessment. CONCLUSIONS: Raising consciousness about clinical significance should be an important priority among nurse researchers. Several recommendations are offered to improve the visibility and salience of clinical significance in nursing science.

Assessing measurement in health: Beyond reliability and validity.

BACKGROUND: Psychometric concepts have undergone a transformation in health fields, as articulated in a consensus report by an international panel of health measurement experts: COSMIN, the COnsensus-based Standards for the selection of health Measurement INstruments. OBJECTIVES: The aims of this paper are to describe emerging ideas relating to the development and testing of new measures in health fields, to present a revised measurement taxonomy that builds upon COSMIN, and to explore the extent to which the new measurement concepts have played a role in psychometric assessments in nursing. DESIGN: A descriptive analysis of a sample of psychometric papers published in three major nursing journals was undertaken. METHODS: A new measurement taxonomy is presented and explained. A sample of 105 studies, representing a consecutive sample of psychometric studies published in the International Journal of Nursing Studies, Nursing Research, and Research in Nursing & Health between 2010 and 2014 was reviewed to ascertain the extent to which psychometric assessments in nursing map onto the new taxonomy. RESULTS: Most nursing studies reviewed adhered to traditional concepts of psychometric assessment, which focus on reliability and validity. The studies in the sample rarely involved assessments of longitudinal measurement aspects, namely the reliability and validity of change scores (responsiveness). CONCLUSIONS: Many constructs of interest to nurse researchers are amenable to change-and these constructs are frequently the target of nursing interventions designed to foster change. Future psychometric work by nurse researchers would benefit from assessments of the psychometric adequacy of change scores.

Getting serious about test-retest reliability: a critique of retest research and some recommendations.

PURPOSE: To focus attention on the need for rigorous and carefully designed test-retest reliability assessments for new patient-reported outcomes and to encourage retest researchers to be thoughtful, ambitious, and creative in their retest efforts. METHODS: The paper outlines key challenges that confront retest researchers, calls attention to some limitations in meeting those challenges, and describes some strategies to improve retest research. RESULTS: Modest retest coefficients are often reported as acceptable, and many important decisions-such as the retest interval-appear not to be evidence-based. Retest assessments are seldom undertaken before a measure has been finalized, which rules out using retest data to select strong, reproducible items. CONCLUSIONS: Strategies for improving retest research include seeking input from patients or experts regarding the stability of the construct to support decisions about the retest interval, analyzing item-level retest data to identify items to revise or discard, establishing a priori standards of acceptability for reliability coefficients, using large, heterogeneous, and representative retest samples and collecting follow-up data to better understand consistent and inconsistent responses over time.

Infusion phlebitis assessment measures: a systematic review.

RATIONALE, AIMS AND OBJECTIVES: Phlebitis is a common and painful complication of peripheral intravenous cannulation. The aim of this review was to identify the measures used in infusion phlebitis assessment and evaluate evidence regarding their reliability, validity, responsiveness and feasibility. METHOD: We conducted a systematic literature review of the Cochrane library, Ovid MEDLINE and EBSCO CINAHL until September 2013. All English-language studies (randomized controlled trials, prospective cohort and cross-sectional) that used an infusion phlebitis scale were retrieved and analysed to determine which symptoms were included in each scale and how these were measured. We evaluated studies that reported testing the psychometric properties of phlebitis assessment scales using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines. RESULTS: Infusion phlebitis was the primary outcome measure in 233 studies. Fifty-three (23%) of these provided no actual definition of phlebitis. Of the 180 studies that reported measuring phlebitis incidence and/or severity, 101 (56%) used a scale and 79 (44%) used a definition alone. We identified 71 different phlebitis assessment scales. Three scales had undergone some psychometric analyses, but no scale had been rigorously tested. CONCLUSION: Many phlebitis scales exist, but none has been thoroughly validated for use in clinical practice. A lack of consensus on phlebitis measures has likely contributed to disparities in reported phlebitis incidence, precluding meaningful comparison of phlebitis rates.

Is there still gender bias in nursing research? An update.

Data from 300 studies published in four research journals in 2010-2011 were analyzed to assess whether nurse researchers continue to oversample females. One-third of the studies had samples that were 100% female and, on average, 74% of all study participants were female. As was found for studies published 5 years earlier, the bias against male participants was consistent across studies differing in methods, specialty areas, funding, and sample characteristics. Studies with male first authors, however, were significantly less likely to have biased samples. Authors of only 23.6% of studies with mixed-sex samples provided information about sex differences in outcomes. Because of gender bias, the evidence base for nursing practice may suffer from problems with generalizability.

Developing a model of competence in the operating theatre: psychometric validation of the perceived perioperative competence scale-revised.

AIM: This paper describes the development and validation of the Revised Perioperative Competence Scale (PPCS-R). BACKGROUND: There is a lack of a psychometrically tested sound self-assessment tools to measure nurses' perceived competence in the operating room. METHODS: Content validity was established by a panel of international experts and the original 98-item scale was pilot tested with 345 nurses in Queensland, Australia. Following the removal of several items, a national sample that included all 3209 nurses who were members of the Australian College of Operating Room Nurses was surveyed using the 94-item version. Psychometric testing assessed content validity using exploratory factor analysis, internal consistency using Cronbach's alpha, and construct validity using the "known groups" technique. During item reduction, several preliminary factor analyses were performed on two random halves of the sample (n=550). RESULTS: Usable data for psychometric assessment were obtained from 1122 nurses. The original 94-item scale was reduced to 40 items. The final factor analysis using the entire sample resulted in a 40 item six-factor solution. Cronbach's alpha for the 40-item scale was .96. Construct validation demonstrated significant differences (p<.0001) in perceived competence scores relative to years of operating room experience and receipt of specialty education. CONCLUSIONS: On the basis of these results, the psychometric properties of the PPCS-R were considered encouraging. Further testing of the tool in different samples of operating room nurses is necessary to enable cross-cultural comparisons.

Statistical process control in nursing research.

In intervention studies in which randomization to groups is not possible, researchers typically use quasi-experimental designs. Time series designs are strong quasi-experimental designs but are seldom used, perhaps because of technical and analytic hurdles. Statistical process control (SPC) is an alternative analytic approach to testing hypotheses about intervention effects using data collected over time. SPC, like traditional statistical methods, is a tool for understanding variation and involves the construction of control charts that distinguish between normal, random fluctuations (common cause variation), and statistically significant special cause variation that can result from an innovation. The purpose of this article is to provide an overview of SPC and to illustrate its use in a study of a nursing practice improvement intervention.

Innovativeness of nurse leaders.

AIM: The purpose of the present study was to describe the innovativeness and the rate of adoption of change among chief nursing officers (CNOs) of acute care hospitals, and explore the difference in the innovativeness of CNOs of Magnet hospitals vs. non-Magnet hospitals. BACKGROUND: There is little evidence to guide the description of innovativeness for nurse leaders, crucial to the implementation of evidence-based practice standards. METHOD: CNOs of acute care hospitals of New York State participated in a mailed survey which incorporated the Scale for the Measurement of Innovativeness. The response rate was 41% (106/261). RESULTS: The majority of the sample was prepared at the master's level with 5-10 years of experience in the CNO role. A significant relationship was found between the innovativeness scale scores and the innovativeness diversity index. The CNOs who completed more leadership courses had implemented significantly more types of innovations and had higher innovativeness scale scores. CONCLUSION: Graduate level education, years of CNO experience and leadership course completion were identified as significantly influencing innovativeness of CNOs. IMPLICATIONS FOR NURSING MANAGEMENT: The characteristics of innovativeness for nurse leaders presented in the present study may assist organizations, CNOs and the Magnet recognition programme to describe innovativeness that supports organizations to continuously improve the quality of patient care.

Blinding during the analysis of research data.

Blinding in randomized controlled trials (RCTs) is a strategy that is widely endorsed as a method of reducing the biases that can result from people's awareness of study participants' treatment group status. Blinding of participants and interventionists is often impossible in nursing RCTs, but data analysts can almost always be blinded. Yet, such blinding seldom occurs, perhaps because of misperceptions about the objectivity of statistical analysis. Data analysts make many semi-subjective decisions about such issues as handling missing data, transforming variables, undertaking subgroup analysis, and selecting covariates. These decisions ideally should be made without the analyst's knowledge of how treatment groups are coded. Strategies for achieving blinding among data analysts are discussed.

Impact factors in nursing journals.

Journal impact factors (IFs), a measure of citation frequency, are published annually in Journal Citation Reports (JCR). Journal IFs, although controversial because of the uses to which they have been put in academic arenas, remain a metric about which nurses should be informed. This paper discusses key issues in the controversy, explains how IFs are computed, and presents historical and 2009 IF data for nursing journals. The number of nursing journals indexed in JCR has grown from 35 in 2004 to 74 in 2009. The journals currently indexed are diverse in terms of focus (practice vs research), specialty areas, and country of publication. The median IF score for nursing journals (0.91 in 2009) is similar to that for several other health care categories. Given the controversies surrounding IFs, it may be useful for nurses to play a more active role in furthering the debate by undertaking research relating to IFs, including studies of how they affect nurses' scholarly pursuits and publication decisions.

Deliberate ignorance: a systematic review of blinding in nursing clinical trials.

BACKGROUND: Blinding is recommended widely as a strategy in randomized controlled trials (RCTs) to reduce biases that can result from awareness of who is receiving the intervention being tested. The absence of blinding, especially when the primary outcomes are subjective, has been found to be associated with inflated estimates of treatment effects, yet little is known about the use of blinding in nursing RCTs. OBJECTIVES: The purposes of this study were (a) to describe the extent to which nurse researchers state that they used blinding as a bias-reduction strategy or explain why it was not used, (b) to identify the groups that are blinded when blinding is used, (c) to assess whether the term blinding or masking is more prevalent, and (d) to explore factors that might affect the use or acknowledgement of blinding in nursing trials. METHODS: Data regarding blinding were extracted systematically and coded from a consecutive sample of 199 RCTs published in 16 nursing journals in 2007 to 2009. RESULTS: Blinding, the term used predominantly in nursing, was reported to have been used in 33% of the studies. Rates of blinding ranged from 2.5% for data analysts to 28% for data collectors. The absence of blinding was discussed as a limitation in only 13% of the studies in which blinding did not occur. The use or mention of blinding was related strongly to whether the journal in which the article was published had endorsed the Consolidated Standards for Reporting Trials guidelines (CONSORT). CONCLUSIONS: Nurse researchers conducting RCTs should be more diligent in following the CONSORT guidelines regarding the use and description of blinding. Although it is often impossible to blind study participants and intervention agents, the blinding of data collectors, data analysts, and nonresearch healthcare staff should be pursued.

Publication opportunities in nonnursing journals.

The purpose of this article was to identify nonnursing journals that have relevance to nursing, that publish articles that cite the nursing literature and may offer excellent but seldom-considered publication opportunities for nurses. Using 22 indicators derived through citation analysis, 64 nonnursing journals that are highly related to nursing were identified. The authors provide information about these 64 journals related to their subject matter, number of issues annually, and their 2008 impact factor.

Intention-to-treat in randomized controlled trials: recommendations for a total trial strategy.

Intention-to-treat (ITT) in randomized controlled trials involves keeping participants in the treatment groups to which they were randomized regardless of whether they withdraw following randomization. Intention-to-treat is a strategy for maintaining the integrity of randomization and strengthening the trial's internal validity. Although ITT is advocated by the Consolidated Standards of Reporting Trials (CONSORT) guidelines, there is confusion about what ITT means and little specific advice on how to achieve it. The purpose of this article is to present definitions of ITT and to suggest strategies for implementing ITT as a total design strategy in nursing clinical trials. Recommendations are offered regarding study planning, study design, subject retention, sampling, data collection, data analysis, and reporting within the context of ITT.

Generalization in quantitative and qualitative research: myths and strategies.

Generalization, which is an act of reasoning that involves drawing broad inferences from particular observations, is widely-acknowledged as a quality standard in quantitative research, but is more controversial in qualitative research. The goal of most qualitative studies is not to generalize but rather to provide a rich, contextualized understanding of some aspect of human experience through the intensive study of particular cases. Yet, in an environment where evidence for improving practice is held in high esteem, generalization in relation to knowledge claims merits careful attention by both qualitative and quantitative researchers. Issues relating to generalization are, however, often ignored or misrepresented by both groups of researchers. Three models of generalization, as proposed in a seminal article by Firestone, are discussed in this paper: classic sample-to-population (statistical) generalization, analytic generalization, and case-to-case transfer (transferability). Suggestions for enhancing the capacity for generalization in terms of all three models are offered. The suggestions cover such issues as planned replication, sampling strategies, systematic reviews, reflexivity and higher-order conceptualization, thick description, mixed methods research, and the RE-AIM framework within pragmatic trials.

The use of the intention-to-treat principle in nursing clinical trials.

BACKGROUND: In randomized controlled trials (RCTs), the intention-to-treat (ITT) principle, which involves maintaining study participants in the treatment groups to which they were randomized regardless of postrandomization withdrawal, is the recommended analytic approach for preserving the integrity of randomization, yet little is known about the use of ITT in nursing RCTs. OBJECTIVES: The purpose of this study was to describe the extent to which nurse researchers who conduct RCTs state that they have used ITT, the extent to which they adhere to ITT principles, and the methods they use to handle missing data. METHODS: Data regarding ITT analysis, participant flow, rates of attrition, and methods of handling missing data were extracted and coded from a consecutive sample of 124 RCTs published in 16 nursing journals in 2007 and 2008. RESULTS: ITT was declared in only 15.3% of the nursing RCTs, and a definition of ITT was offered in fewer than half of these studies. On the basis of the authors' descriptions of analytic procedures, it was concluded that 10.5% of those claiming ITT use had used a per-protocol analysis rather than an ITT analysis. Overall, 46.8% of the RCTs were classified as having either a classic or a modified ITT analysis, indicating that many nurse researchers are not stating their actual adherence to ITT, despite advice to do so in the Consolidated Standards of Reporting Trials guidelines. CONCLUSIONS: Nurse researchers conducting RCTs should be more diligent in following the Consolidated Standards of Reporting Trials guidelines about ITT, documenting ITT use in their reports, clarifying their definition of ITT, and presenting flowcharts that describe subject flow. Readers of nursing reports, in evaluating evidence from RCTs, should not rely on stated use of ITT but should examine how analyses were conducted.

International differences in nursing research, 2005-2006.

PURPOSE: To compare the characteristics of nursing research being done in different countries, using data from studies published in nonspecialty, English-language nursing journals. DESIGN: Data for this cross-sectional analysis were retrieved from a consecutive sample of 1,072 studies published in eight leading English-language research journals in 2005 and 2006. METHODS: For each study, data were extracted on the characteristics of the study participants and authors, study focus-specialty area, funding, and methodologic attributes. Studies from 15 countries or regional groupings were compared. FINDINGS: International differences in authors, participants, and study characteristics were typically large and statistically significant. Studies that were focused on nurses were especially prevalent in Europe, Australia, and Canada, whereas patient-centered studies were most common in Asian countries and the US. Qualitative studies were predominant in Norway, Sweden, and the UK. Asian nurse researchers, by contrast, undertook mostly quantitative studies, and were especially likely to conduct intervention research. Significant country differences existed in the omission of demographic (age and sex) information about participants, with omissions most prevalent in the UK and Ireland and least prevalent in Asian countries. Research funding was reported for 62% of all studies, ranging from 13% in Turkey to over 75% in Canada and the US. CONCLUSIONS: Although this study had several limitations, especially with regard to potential biases in the sample of studies from non-English-speaking countries, this analysis of over 1,000 nursing studies indicates many important inter-country differences in the focus, methods, and authorship patterns of nursing research published in leading journals. CLINICAL RELEVANCE: Because research "informs" practice, international differences that exist in the focus and methods of nursing research have implications for nursing practice in the respective countries.

International gender bias in nursing research, 2005-2006: a quantitative content analysis.

AIM: This paper reports a study that examined the extent to which nurse researchers internationally disproportionately include females as participants in their research. BACKGROUND: A bias toward predominantly male samples has been well-documented in medical research, but recently a gender bias favoring women in nursing research has been identified in studies published in four North American journals. METHOD: We extracted information about study samples and characteristics of the studies and authors from a consecutive sample of 834 studies published in eight leading English-language nursing research journals in 2005-2006. The primary analyses involved one-sample t-tests that tested the null hypothesis that males and females are equally represented as participants in nursing studies. Studies from different countries, in different specialty areas, and with varying author and methodologic characteristics were compared with regard to the key outcome variable, percent of participants who were female. RESULTS: Overall, 71% of participants, on average, were female, including 68% in client-focused research and 83% in nurse-focused studies (all p<.001). Females were significantly overrepresented as participants in client-focused research in almost all specialty areas, particularly in mental health, community health, health promotion, and geriatrics. The bias favoring female participants in client-focused studies was especially strong in the United States and Canada, but was also present in European countries, most Asian countries, and in Australia. Female overrepresentation was persistent, regardless of methodological characteristics (e.g., qualitative versus quantitative), funding source, and most researcher characteristics (e.g., academic rank). Studies with male authors, however, had more sex-balanced samples. The mean percentage female in client-focused studies with a female lead author was 70.0, compared to 52.1 for male lead authors. CONCLUSION: Nurse researchers not only in North America but around the globe need to pay attention to who will benefit from their research and to whether they are adequately inclusive in studying client groups about which there are knowledge gaps.