PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool.

BACKGROUND: The Upper Digestive Disease (UDD) Tool™ is used to monitor symptom frequency, intensity, and interference across nine symptom domains and includes two Patient-Reported Outcome Measurement Information System (PROMIS) domains assessing physical and mental health. This study aimed to establish cut scores for updated symptom domains through standard setting exercises and evaluate the effectiveness and acceptability of virtual standard setting. METHODS: The extended Angoff method was employed to determine cut scores. Subject matter experts refined performance descriptions for symptom control categories and achieved consensus. Domains were categorized into good, moderate, and poor symptom control. Two cut scores were established, differentiating good vs. moderate and moderate vs. poor. Panelists estimated average scores for 100 borderline patients per item. Cut scores were computed based on the sum of the average ratings for individual questions, converted to 0-100 scale. RESULTS: Performance descriptions were refined. Panelists discussed that interpretation of the scores should take into account the timing of symptoms after surgery and patient populations, and the importance of items asking symptom frequency, severity, and interference with daily life. The good/moderate cut scores ranged from 21.3 to 35.0 (mean 28.6, SD 3.6) across domains, and moderate/poor ranged from 47.5 to 71.3 (mean 54.5, SD 7.0). CONCLUSIONS: Panelists were confident in the virtual standard setting process, expecting valid cut scores. Future studies can further validate the cut scores using patient perspectives and collect patient and physician preferences for displaying contextual items on patient- and physician-facing dashboard.

The Response Vector for Mastery Method of Standard Setting.

Proposed is a new method of standard setting referred to as response vector for mastery (RVM) method. Under the RVM method, the task of panelists that participate in the standard setting process does not involve conceptualization of a borderline examinee and probability judgments as it is the case with the Angoff and bookmark methods. Also, the RVM-based computation of a cut-score is not based on a single item (e.g., marked in an ordered item booklet) but, instead, on a response vector (1/0 scores) on items and their parameters calibrated in item response theory or under the recently developed D-scoring method. Illustrations with hypothetical and real-data scenarios of standard setting are provided and methodological aspects of the RVM method are discussed.

Accuracy of the 'CUT' Score for Assessing Malignancy in Bethesda 3 and 4 Thyroid Nodules in North American Population: A Retrospective Study.

BACKGROUND: The CUT score is a thyroid nodule malignancy risk assessment scoring system intended to guide surgeons in treating Bethesda 3 and 4 thyroid nodules. It is based on clinical (C) and ultrasonographic (U) features and a five-tiered (T) representing cytology. PURPOSE: Our study aimed to assess the utility of the CUT score in predicting thyroid malignancy in the North American population. The main reason for creating this score is to reduce unnecessary surgeries on these challenging thyroid nodules. MATERIALS AND METHODS: A retrospective record review study applied the CUT score to 219 Bethesda 3 and 4 thyroid nodules. A total of 203 Bethesda 3 and 16 Bethesda 4 nodules from patients treated between January 2015 and December 2019 at a single institution were assessed. A receiver operating characteristic (ROC) curve analysis was performed to evaluate the CUT diagnostic test. Binary logistic regression analysis was performed. Iteration of analysis was performed after stratification according to body mass index to assess CUT score accuracy in obese and non-obese patients. RESULTS: Of 219 nodules analyzed, 148 were characterized as benign and 71 as malignant. Prevalence rates of malignancy were 29.6% (n = 60) and 68.8% (n = 11) in Bethesda 3 and 4 nodules, respectively. The mean CU (clinical, ultrasonography) score was 5.35 ± 1.38 in benign nodules versus 4.96 ± 1.5 in malignant nodules (p = 0.08). The area under the curve (AUC = 0.433) for the association of CUT scores with nodule malignancy was not significant (p = 0.13). The CUT score was insignificant as a diagnostic test for nodule malignancy in obese (AUC = 0.45; p = 0.72) and non-obese patients (AUC = 0.39; p = 0.08). CONCLUSION: The CUT score did not correlate with preoperative malignancy risk estimates in Bethesda 3 thyroid nodules and, therefore, may have limited utility as a predictor of malignancy in these thyroid nodules.

Psychological pain and previous suicide attempts in young adults: Results with the Portuguese version of the Psychache Scale.

OBJECTIVES: In this study, we tested whether the unbearable pain scale of the Psychache Scale provides value for the statistical prediction of a previous suicide attempt; we evaluated whether psychological pain has incremental value in the statistical prediction of a previous suicide attempt relative to the reporting of having had a psychiatric diagnosis, and we established a cut-score for the Psychache Scale. METHOD: Two samples participated: a sample consisting of 1,460 young adults and a sample of 628 Portuguese community adults. RESULTS AND CONCLUSIONS: Results demonstrated that unbearable pain is not a stronger predictor of a previous suicide attempt than is the bearable pain score of the Psychache Scale; that the total score of the Psychache scale provides an additional statistically significant contribution to the statistical prediction of a previous suicide attempt relative to the reporting of having had a psychiatric diagnosis.

What Is the Minimal Competency for a Clinical Ethics Consult Simulation? Setting a Standard for Use of the Assessing Clinical Ethics Skills (ACES) Tool.

Background: The field of clinical ethics is examining ways of determining competency. The Assessing Clinical Ethics Skills (ACES) tool offers a new approach that identifies a range of skills necessary in the conduct of clinical ethics consultation and provides a consistent framework for evaluating these skills. Through a training website, users learn to apply the ACES tool to clinical ethics consultants (CECs) in simulated ethics consultation videos. The aim is to recognize competent and incompetent clinical ethics consultation skills by watching and evaluating a videotaped CEC performance. We report how we set a criterion cut score (i.e., minimally acceptable score) for judging the ability of users of the ACES tool to evaluate simulated CEC performances. Methods: A modified Angoff standard-setting procedure was used to establish the cut score for an end-of-life case included on the ACES training website. The standard-setting committee viewed the Futility Case and estimated the probability that a minimally competent CEC would correctly answer each item on the ACES tool. The committee further adjusted these estimates by reviewing data from 31 pilot users of the Futility Case before determining the cut score. Results: Averaging over all 31 items, the proposed proportion correct score for minimal competency was 80%, corresponding to a cut score that is between 24 and 25 points out of 31 possible points. The standard-setting committee subsequently set the minimal competency cut score to 24 points. Conclusions: The cut score for the ACES tool identifies the number of correct responses a user of the ACES tool training website must attain to "pass" and reach minimal competency in recognizing competent and incompetent skills of the CECs in the simulated ethics consultation videos. The application of the cut score to live training of CECs and other areas of practice requires further investigation.

Delineating the age-related attenuation of vascular function: Evidence supporting the efficacy of the single passive leg movement as a screening tool.

Continuous passive leg movement (PLM) is a promising clinical assessment of the age-related decline in peripheral vascular function. To further refine PLM, this study evaluated the efficacy of a single PLM (sPLM), a simplified variant of the more established continuous movement approach, to delineate between healthy young and old men based on vascular function. Twelve young (26 ± 5 yr) and 12 old (70 ± 7 yr) subjects underwent sPLM (a single passive flexion and extension of the knee joint through 90°), with leg blood flow (LBF, common femoral artery with Doppler ultrasound), blood pressure (finger photoplethysmography), and leg vascular conductance (LVC) assessed. A receiver operator characteristic curve analysis was used to determine an age-specific cut score, and a factor analysis was performed to assess covariance. Baseline LBF and LVC were not different between groups (P = 0.6). The high level of covariance and similar predictive value for all PLM-induced LBF and LVC responses indicates LBF, alone, can act as a surrogate variable in this paradigm. The peak sPLM-induced increase in LBF from baseline was attenuated in the old (Young: 717 ± 227, Old: 260 ± 97 ml/min, P < 0.001; cut score: 372 ml/min), as was the total LBF response (Young: 155 ± 67, Old: 26 ± 17 ml, P < 0.001; cut score: 58 ml). sPLM, a simplified version of PLM, exhibits the prerequisite qualities of a valid screening test for peripheral vascular dysfunction, as evidenced by an age-related attenuation in the peripheral hyperemic response and a clearly delineated age-specific cut score. NEW & NOTEWORTHY Single passive leg movement (sPLM) exhibits the prerequisite qualities of a valid screening test for peripheral vascular dysfunction. sPLM displayed an age-related reduction in the peripheral hemodynamic response for amplitude, duration, initial rate of change, and total change with clearly delineated age-specific cut scores. sPLM has a strong candidate variable that is a simple single numeric value, for which to appraise peripheral vascular function, the 45-s hyperemic response (leg blood flow area under the curve: 45 s).

Osteometric assessment of various mandibular morphological traits for sexual dimorphism in Jordanians by discriminant function analysis / Evaluación osteométrica de diversos rasgos morfológicos mandibulares para el dimorfismo sexual en jordanos mediante el análisis de funciones discriminantes

The aim of this study was to analyze the sexual dimorphism in the mandibles of Jordanians and derive a population specific equation for sex prediction. The employed sample comprised 270 cone beam computed tomography (CBCT) images of Jordanians (147 males and 123 females) aged 27-55 years. The images were rendered to 3D surface models, using customized software that provided various views for each mandible, on which measurements made of nine parameters (ramus height, right and left coronoids height, mandibular length, gonial angle, Intergonial breadth, Intercondylar breadth, mental and mandibular foramena) for the analysis of sex identification. The collected data sets were treated statistically using the Statistical Package SPSS version-21. Canonical discriminant function analysis was used to specify a parameter or combination of parameters that best separate the two sexes. Stepwise analysis together with leave-one-out classification procedure was applied (utilizing the Wilks lambda method), which identified six parameters that formed the best combination most precisely predicting sex. It also identified the predictors' coefficients from which a predictive equation for the Jordanian population was derived. The discriminant function group centroid discriminant scores for males and females and the sectioning cut-score were also defined. According to the equation, the function score for any set of measurements that is above the cut score is probably for a male individual whereas a score that was below the cut score is probably for a female subject.

El objetivo de este estudio fue analizar el dimorfismo sexual en las mandíbulas de los jordanos y derivar una ecuación específica de la población para la predicción del sexo. La muestra consistió en 270 imágenes de tomografía computarizada de haz cónico (CBCT) de jordanos (147 hombres y 123 mujeres) con edades entre los 27 y 55 años. Las imágenes se representaron en modelos de superficie 3D, utilizando software personalizado que proporcionó varias imágenes para cada mandíbula, en las que se midieron nueve parámetros (altura de la rama, altura de los procesos coronoides derecho e izquierdo, longitud mandibular, ángulo gonial, amplitud intergonial, amplitud intercondilar y foramen mandibular) para el análisis de la identificación de sexo. Los conjuntos de datos recopilados se trataron estadísticamente utilizando el programa estadístico SPSS versión-21. El análisis de función discriminante canónica se utilizó para especificar un parámetro, o una combinación de parámetros que separaran mejor los dos sexos. Se aplicó el análisis por pasos junto con el procedimiento de clasificación de dejar salir uno (utilizando el método de Wilks lambda), que identificó seis parámetros que formaban la mejor combinación que predecía con mayor precisión el sexo. También se identificó los coeficientes de los predictores a partir de los cuales se obtuvo una ecuación predictiva para la población jordana. Se definieron además, los puntajes discriminantes centroide del grupo para hombres y mujeres, además del puntaje de corte seccional. De acuerdo con la ecuación, el puntaje de la función para cualquier conjunto de medidas que está por encima del puntaje de corte es probablemente para un individuo masculino, mientras que un puntaje que estaba por debajo del puntaje de corte, probablemente era femenino.

Using Procedure Based on Item Response Theory to Evaluate Classification Consistency Indices in the Practice of Large-Scale Assessment.

In spite of the growing interest in the methods of evaluating the classification consistency (CC) indices, only few researches are available in the field of applying these methods in the practice of large-scale educational assessment. In addition, only few studies considered the influence of practical factors, for example, the examinee ability distribution, the cut score location and the score scale, on the performance of CC indices. Using the newly developed Lee's procedure based on the item response theory (IRT), the main purpose of this study is to investigate the performance of CC indices when practical factors are taken into consideration. A simulation study and an empirical study were conducted under comprehensive conditions. Results suggested that with negatively skewed distribution, the CC indices were larger than with other distributions. Interactions occurred among ability distribution, cut score location, and score scale. Consequently, Lee's IRT procedure is reliable to be used in the field of large-scale educational assessment, and when reporting the indices, it should be treated with caution as testing conditions may vary a lot.

Standardized Testing Practices: Effect on Graduation and NCLEX® Pass Rates.

The use standardized testing in pre-licensure nursing programs has been accompanied by conflicting reports of effective practices. The purpose of this project was to describe standardized testing practices in one states' nursing programs and discover if the use of a cut score or oversight of remediation had any effect on (a) first time NCLEX® pass rates, (b) on-time graduation (OTG) or (c) the combination of (a) and (b). Administrators of 38 nursing programs in one Southwest state were sent surveys; surveys were returned by 34 programs (89%). Survey responses were compared to each program's NCLEX pass rate and on-time graduation rate; t-tests were conducted for significant differences associated with a required minimum score (cut score) and oversight of remediation. There were no significant differences in NCLEX pass or on-time graduation rates related to establishment of a cut score. There was a significant difference when the NCLEX pass rate and on-time graduation rate were combined (Outcome Index "OI") with significantly higher program outcomes (P=.02.) for programs without cut-scores. There were no differences associated with faculty oversight of remediation. The results of this study do not support establishment of a cut-score when implementing a standardized testing.

Standard-setting methodology: Establishing performance standards and setting cut-scores to assist score interpretation.

A critical step in the development and use of tests of physical fitness for employment purposes (e.g., fitness for duty) is to establish 1 or more cut points, dividing the test score range into 2 or more ordered categories reflecting, for example, fail/pass decisions. Over the last 3 decades elaborated theories and methods have evolved focusing on the process of establishing 1 or more cut-scores on a test. This elaborated process is widely referred to as "standard-setting". As such, the validity of the test score interpretation hinges on the standard-setting, which embodies the purpose and rules according to which the test results are interpreted. The purpose of this paper is to provide an overview of standard-setting methodology. The essential features, key definitions and concepts, and various novel methods of informing standard-setting will be described. The focus is on foundational issues with an eye toward informing best practices with new methodology. Throughout, a case is made that in terms of best practices, establishing a test standard involves, in good part, setting a cut-score and can be conceptualized as evidence/data-based policy making that is essentially tied to test validity and an evidential trail.

Identification of anxiety sensitivity classes and clinical cut-scores in a sample of adult smokers: results from a factor mixture model.

Anxiety sensitivity (AS), a multidimensional construct, has been implicated in the development and maintenance of anxiety and related disorders. Recent evidence suggests that AS is a dimensional-categorical construct within individuals. Factor mixture modeling was conducted in a sample of 579 adult smokers (M age=36.87 years, SD=13.47) to examine the underlying structure. Participants completed the Anxiety Sensitivity Index-3 and were also given a Structured Clinical Interview for DSM-IV-TR. Three classes of individuals emerged, a high AS (5.2% of the sample), a moderate AS (19.0%), and a normative AS class (75.8%). A cut-score of 23 to identify high AS individuals, and a cut-score of 17 to identify moderate-to-high AS individuals were supported in this study. In addition, the odds of having a concurrent anxiety disorder (controlling for other Axis I disorders) were the highest in the high AS class and the lowest in the normative AS class.

Reconsidering the cut score of Korean National Medical Licensing Examination.

After briefly reviewing theories of standard setting we analyzed the problems of the current cut scores. Then, we reported the results of need assessment on the standard setting among medical educators and psychometricians. Analyses of the standard setting methods of developed countries were reported as well. Based on these findings, we suggested the Bookmark and the modified Angoff methods as alternative methods for setting standard. Possible problems and challenges were discussed when these methods were applied to the National Medical Licensing Examination.