Sensitivity and specificity of conventional and new face validation in determining the incomprehensible items by older people: Empirical evidence of testing 106 quality-of-life items.

Objective: This methodological research aimed to investigate and compare the sensitivity and specificity of conventional and new face validation in identifying incomprehensible items empirically. Methods: A purposive sample of 15 older people living in three residential care homes (RCHs) in Hong Kong was used to evaluate a newly developed 106 items covering seven quality-of-life dimensions. The abbreviated Mental Test (Hong Kong version; AMT) was used as a screening tool for excluding those with impaired cognition. The interview was audiotaped, and incomprehensible items were identified by the research panel accordingly (served as the gold standard). The socio-demographics of the respondents were described. Understandability (yes/no, conventional face validation method) and interpretability (4-point Likert scale, new method) were compared and used to compute the Kappa value (representing chance agreement), sensitivity, and specificity analysis. Results: Fifteen older people were interviewed and responded to the structured interview of 106 items regarding understandability and interpretability. 61 items (57%) obtained 100% positive understandability while only 35 items (33%) obtained 100% correct interpretability.The Kappa coefficient was 0.388 (P < 0.001) of the chance agreement between understandability and interpretability. The panel confirmed that 32% of items required revision (i.e., incomprehensible items). The false negative rate of using the conventional approach was up to 70.59% while both the false positive and negative rates of using the new approach were low (0%-5.88%). Conclusion: This empirical evidence indicated that the conventional approach of face validation for checking incomprehensible items by older people encountered a high false negative rate. On the contrary, the new approach was recommended because it demonstrated high sensitivity and specificity and low false positive and negative rates in identifying incomprehensible items.

Design, development, and face validation of an intubation simulation device using real-time force data feedback.

Objectives: To develop a novel laryngoscope device capable of dynamically measuring force and torque measurements in real-time during intubation and to explore the efficacy of such a device through a face validation simulation. Methods: The torque sensor laryngoscope is designed for use during intubation and is modeled after a standard, single-use plastic laryngoscope. After device calibration, a face validation study was performed with intubation experts in the field. Quantitative data (intubation force metrics) and qualitative data (expert feedback on the device) were collected from three intubations using a Mac blade and three intubations with the Miller blade. Results: Three experts (two anesthesiologists and one otolaryngologist) participated in the study. The mean maximum force exerted with the Mac blade was 24.5 N (95% confidence interval [CI], 22.3-26.8). The average force exerted was 13.6 N (95% CI, 11.7-15.5). The average total suspension time was 13.1 s (95% CI, 10.4-15.8). The average total impulse was 164.6 N·s (95% CI, 147.9-181.4). The mean maximum force exerted with the Miller blade was 31.6 N (95% CI, 26.4-36.8). The average force exerted was 15.8 N (95% CI, 13.8-17.9). The average total suspension time was 11.3 s (95% CI, 9.9-12.6). The average total impulse was 216.2 N·s (95% CI, 186.5-245.9). The mean maximum force (p = .0265) and total impulse (p = .009) were significantly higher in the Miller blade trials than in the Mac blade trials. Survey results found that this device, while bulky, intubated similarly to standard-use models and has potential as an intubation teaching tool. Conclusion: The torque sensor laryngoscope can measure and display real-time intubation force metrics for multiple laryngoscope blades. Initial validation studies showed a significantly lower maximum force and total impulse when intubating with the Mac blade than with the Miller blade. Face validation survey results were positive and suggested the potential for this device as a teaching tool. Level of Evidence: Level 5.

Development and face validation of ultrasound-guided renal biopsy virtual trainer.

The overall prevalence of chronic kidney disease in the general population is ∼14% with more than 661,000 Americans having a kidney failure. Ultrasound (US)-guided renal biopsy is a critically important tool in the evaluation and management of renal pathologies. This Letter presents KBVTrainer, a virtual simulator that the authors developed to train clinicians to improve procedural skill competence in US-guided renal biopsy. The simulator was built using low-cost hardware components and open source software libraries. They conducted a face validation study with five experts who were either adult/pediatric nephrologists or interventional/diagnostic radiologists. The trainer was rated very highly (>4.4) for the usefulness of the real US images (highest at 4.8), potential usefulness of the trainer in training for needle visualization, tracking, steadiness and hand-eye coordination, and overall promise of the trainer to be useful for training US-guided needle biopsies. The lowest score of 2.4 was received for the look and feel of the US probe and needle compared to clinical practice. The force feedback received a moderate score of 3.0. The clinical experts provided abundant verbal and written subjective feedback and were highly enthusiastic about using the trainer as a valuable tool for future trainees.

A mixed-reality surgical trainer with comprehensive sensing for fetal laser minimally invasive surgery.

PURPOSE: Smaller incisions and reduced surgical trauma made minimally invasive surgery (MIS) grow in popularity even though long training is required to master the instrument manipulation constraints. While numerous training systems have been developed in the past, very few of them tackled fetal surgery and more specifically the treatment of twin-twin transfusion syndrome (TTTS). To address this lack of training resources, this paper presents a novel mixed-reality surgical trainer equipped with comprehensive sensing for TTTS procedures. The proposed trainer combines the benefits of box trainer technology and virtual reality systems. Face and content validation studies are presented and a use-case highlights the benefits of having embedded sensors. METHODS: Face and content validity of the developed setup was assessed by asking surgeons from the field of fetal MIS to accomplish specific tasks on the trainer. A small use-case investigates whether the trainer sensors are able to distinguish between an easy and difficult scenario. RESULTS: The trainer was deemed sufficiently realistic and its proposed tasks relevant for practicing the required motor skills. The use-case demonstrated that the motion and force sensing capabilities of the trainer were able to analyze surgical skill. CONCLUSION: The developed trainer for fetal laser surgery was validated by surgeons from a specialized center in fetal medicine. Further similar investigations in other centers are of interest, as well as quality improvements which will allow to increase the difficulty of the trainer. The comprehensive sensing appeared to be capable of objectively assessing skill.

Escala de Macarthur de status social subjetivo: validade e associação com indicadores de posição socioeconômica no curso de vida - estudo longitudinal de saúde do adulto (ELSA-BRASIL) / Escala de Macarthur de status social subjetivo: validade e associação com indicadores de posição socioeconômica no curso de vida - estudo longitudinal de saúde do adulto (ELSA-BRASIL)

INTRODUCTION: The MacArthur scale of subjective social status (SSS), which combines verbal instruction and an image of a 10-steps ladder, aims to capture the common sense perception that an individual has about his/her own position in social hierarchies, in three different contexts: society, community and work. OBJECTIVE: To conduct a concurrent and face validity analysis of the MacArthur scale (Paper 1) and to investigate whether individuals who perceive themselves as having a low SSS in adult life, in the society context, were more exposed to low objective socioeconomic positions (SEP) in childhood and youth, after adjusting for current SEP. METHODS: The article 1 included 159 participants from the Longitudinal Study of Adult Health (ELSA-Brasil), from Minas Gerais Investigating Center, selected by convenience during the 2nd wave of interviews and examinations between 11/2012 and 02/2014. The MacArthur scale was compared with a Status scale, created for this study based on the cognitive metaphor theory, which used the same image of a ladder as an indicator of the status and a very similar but more direct instruction. The concurrent validity was examined by comparing the answers to both scales (MacArthur and Status) using weighted kappa statistics. The face validity was assessed by qualitative methods from corpus linguistics. Ordinal logistic regression was used to investigate whether sociodemographic factors were associated with differences of 1, or 2 or more steps in the choices made using the MacArthur and the Status ladders. In the paper 2, 15.105 participants from the base line of ELSA-Brasil (2008-2010) were included. Low SSS was in the society was defined by the choice of a step lower than 5 out of 10. Multivariate logistic regression analysis was used to investigate whether exposures to low objective indicators of SEP in the childhood (maternal education), youth (occupational social class of the household head; occupational social class of the first job; nature of occupation of household head; nature of occupation in the first job) and adult life (current occupational social class; current nature of occupation; current education) increased the chances of being at low SSS. RESULTS: Agreement between the answers to the MacArthur and Status scales moderated in the society context (kw=0,55) and good in the community (kw=0,60) and work (kw=0,67). Face validity analysis showed that MacArthur scale captures beyond the classic socioeconomic indicators, encompassing elements from collective health, as home, transport, health, leisure, merit etc. Although individuals without college education (in the society context) and women (in the community ambience) had greater chances of differing by 1, or 2 or more steps from those with college education or who were women, qualitative differences were not identified by the corpus linguistics, suggesting that the MacArthur scale has good face validity. In paper 2, after adjustments, low SSS in adult life remained statistically associated with low SEP in all stages of the life course, being the magnitude of the associations strong in adult life, moderate in youth and weak in childhood, showing dose response gradients in all associations. CONCLUSION: The MacArthur scale seems to be a valid instrument to measure the SSS in the Brazilian sociocultural context, thus they can be used in studies of social inequality in health. In addition, results suggest that the scale may capture a summary of exposures and experiences of present and past SEP, both intra and intergenerational, allowing to expand the interpretations of findings of health inequalities based on the MacArthur scale.

INTRODUCTION: The MacArthur scale of subjective social status (SSS), which combines verbal instruction and an image of a 10-steps ladder, aims to capture the common sense perception that an individual has about his/her own position in social hierarchies, in three different contexts: society, community and work. OBJECTIVE: To conduct a concurrent and face validity analysis of the MacArthur scale (Paper 1) and to investigate whether individuals who perceive themselves as having a low SSS in adult life, in the society context, were more exposed to low objective socioeconomic positions (SEP) in childhood and youth, after adjusting for current SEP. METHODS: The article 1 included 159 participants from the Longitudinal Study of Adult Health (ELSA-Brasil), from Minas Gerais Investigating Center, selected by convenience during the 2nd wave of interviews and examinations between 11/2012 and 02/2014. The MacArthur scale was compared with a Status scale, created for this study based on the cognitive metaphor theory, which used the same image of a ladder as an indicator of the status and a very similar but more direct instruction. The concurrent validity was examined by comparing the answers to both scales (MacArthur and Status) using weighted kappa statistics. The face validity was assessed by qualitative methods from corpus linguistics. Ordinal logistic regression was used to investigate whether sociodemographic factors were associated with differences of 1, or 2 or more steps in the choices made using the MacArthur and the Status ladders. In the paper 2, 15.105 participants from the base line of ELSA-Brasil (2008-2010) were included. Low SSS was in the society was defined by the choice of a step lower than 5 out of 10. Multivariate logistic regression analysis was used to investigate whether exposures to low objective indicators of SEP in the childhood (maternal education), youth (occupational social class of the household head; occupational social class of the first job; nature of occupation of household head; nature of occupation in the first job) and adult life (current occupational social class; current nature of occupation; current education) increased the chances of being at low SSS. RESULTS: Agreement between the answers to the MacArthur and Status scales moderated in the society context (kw=0,55) and good in the community (kw=0,60) and work (kw=0,67). Face validity analysis showed that MacArthur scale captures beyond the classic socioeconomic indicators, encompassing elements from collective health, as home, transport, health, leisure, merit etc. Although individuals without college education (in the society context) and women (in the community ambience) had greater chances of differing by 1, or 2 or more steps from those with college education or who were women, qualitative differences were not identified by the corpus linguistics, suggesting that the MacArthur scale has good face validity. In paper 2, after adjustments, low SSS in adult life remained statistically associated with low SEP in all stages of the life course, being the magnitude of the associations strong in adult life, moderate in youth and weak in childhood, showing dose response gradients in all associations. CONCLUSION: The MacArthur scale seems to be a valid instrument to measure the SSS in the Brazilian sociocultural context, thus they can be used in studies of social inequality in health. In addition, results suggest that the scale may capture a summary of exposures and experiences of present and past SEP, both intra and intergenerational, allowing to expand the interpretations of findings of health inequalities based on the MacArthur scale.

Nursing Activity Score for estimating nursing care need in intensive care units: findings from a face and content validity study.

AIM: To re-evaluate the face and content validity of the Nursing Activity Score currently adopted in evaluating activities that best describe workloads in intensive care units and their weight in describing average nursing time consumption. BACKGROUND: The Nursing Activity Score calculates the amount of nursing time that each patient will require over the next 24 hours. It has been widely used around the world since its first validation in 2003. However, no re-evaluation of its validity with regard to the advancements achieved in intensive care units nursing care has been documented to date. METHOD: A research project was undertaken from 2012 to 2015, aimed at critically evaluating and validating this tool in the current context of Italian intensive care units nursing care. The 23 items were translated forward and backward into the Italian language, then a panel of 10 experts in critical care evaluated the face validity. Content validity was evaluated through focus groups involving seven critical care expert registered nurses. FINDINGS: The Nursing Activity Score instrument has been considered as not fully adequate to measure current intensive care units nursing activities and their weightings have been considered not fully adequate to score average nursing time consumption. From the content validity process, lack of adequacy has emerged with respect to the concept of nursing care underpinning the tool, the interventions included, its capability to predict the nursing resources needed, advancements achieved in intensive care units nurses' roles and competences, and the contextual factors that may influence consumption of nursing time. CONCLUSIONS: Development of the Nursing Activity Score tool both conceptually and in its structure, in view of the innovations that have occurred in the context of intensive care units, is necessary to continue to have a common tool to help clinicians and managers to capture accurately and compare nursing care required by patients in critical care settings. IMPLICATIONS FOR NURSING MANAGEMENT: There is a need to revise the Nursing Activity Score tool, enabling its use in estimating nursing workloads in current Italian intensive care units practice. A taskforce of clinical nurses and nursing managers, capable of protecting the valuable original Nursing Activity Score project and to advance its further development is recommended.

Towards virtual FLS: development of a peg transfer simulator.

BACKGROUND: Peg transfer is one of five tasks in the Fundamentals of Laparoscopic Surgery (FLS), program. This paper reports the development and validation of a Virtual Basic Laparoscopic Skill Trainer-Peg Transfer (VBLaST-PT(©) ) simulator for automatic real-time scoring and objective quantification of performance. METHODS: New techniques have been introduced in order to allow bi-manual manipulation of pegs and automatic scoring/evaluation while maintaining high quality of simulation. A preliminary face and construct validation study was performed with 22 subjects divided into two groups: experts (PGY 4-5, fellow and practicing surgeons) and novice (PGY 1-3). RESULTS: Face validation shows high scores for all aspects of the simulation. Two-tailed Mann-Whitney U-test scores showed significant differences between the two groups on completion time (P = 0.003), FLS score (P =0.002) and the VBLaST-PT© score (P = 0.006). CONCLUSIONS: VBLaST-PT(©) is a high quality virtual simulator that showed both face and construct validity.