Efficiency of computerized adaptive testing with a cognitively designed item bank.

An item bank is key to applying computerized adaptive testing (CAT). The traditional approach to developing an item bank requires content experts to design each item individually, which is a time-consuming and costly process. The cognitive design system (CDS) approach offers a solution by automating item generation. However, the CDS approach has a specific way of calibrating or predicting item difficulty that affects the measurement efficiency of CAT. A simulation study was conducted to compare the efficiency of CAT using both calibration and prediction models. The results show that, although the predictive model (linear logistic trait model; LLTM) shows a higher root mean square error (RMSE) than the baseline model (Rasch), it requires only a few additional items to achieve comparable RMSE. Importantly, the number of additional items needed decreases as the explanatory rate of the model increases. These results indicate that the slight reduction in measurement efficiency due to prediction item difficulty is acceptable. Moreover, the use of prediction item difficulty can significantly reduce or even eliminate the need for item pretesting, thereby reducing the costs associated with item calibration.

A two-step item bank calibration strategy based on 1-bit matrix completion for small-scale computerized adaptive testing.

Computerized adaptive testing (CAT) is a widely embraced approach for delivering personalized educational assessments, tailoring each test to the real-time performance of individual examinees. Despite its potential advantages, CAT�s application in small-scale assessments has been limited due to the complexities associated with calibrating the item bank using sparse response data and small sample sizes. This study addresses these challenges by developing a two-step item bank calibration strategy that leverages the 1-bit matrix completion method in conjunction with two distinct incomplete pretesting designs. We introduce two novel 1-bit matrix completion-based imputation methods specifically designed to tackle the issues associated with item calibration in the presence of sparse response data and limited sample sizes. To demonstrate the effectiveness of these approaches, we conduct a comparative assessment against several established item parameter estimation methods capable of handling missing data. This evaluation is carried out through two sets of simulation studies, each featuring different pretesting designs, item bank structures, and sample sizes. Furthermore, we illustrate the practical application of the methods investigated, using empirical data collected from small-scale assessments.

Nurturing Untapped Integration Expertise of MS4 Assessment Writers.

Creating original, integrated multiple-choice questions (MCQs) is time-consuming and onerous for basic science and clinical faculty. We demonstrate that medical students are co-experts to overcome assessment challenges of the faculty. We recruited, trained, and motivated medical students to write 10,000 high-quality MCQs for use in the foundational courses of medical education. These students were ideal because they possessed integrated knowledge (basic sciences and clinical experience). We taught them how to write high-quality MCQs using a writing template and continuous monitoring and support by an item bank curator. The students themselves also benefitted personally and pedagogically from the experience.

Identification of application and interpretation errors that can occur in pairwise meta-analyses in systematic reviews of interventions: a systematic review.

OBJECTIVES: To generate a bank of items describing application and interpretation errors that can arise in pairwise meta-analyses in systematic reviews of interventions. STUDY DESIGN AND SETTING: MEDLINE, Embase, and Scopus were searched to identify studies describing types of errors in meta-analyses. Descriptions of errors and supporting quotes were extracted by multiple authors. Errors were reviewed at team meetings to determine if they should be excluded, reworded, or combined with other errors, and were categorized into broad categories of errors and subcategories within. RESULTS: Fifty articles met our inclusion criteria, leading to the identification of 139 errors. We identified 25 errors covering data extraction/manipulation, 74 covering statistical analyses, and 40 covering interpretation. Many of the statistical analysis errors related to the meta-analysis model (eg, using a two-stage strategy to determine whether to select a fixed or random-effects model) and statistical heterogeneity (eg, not undertaking an assessment for statistical heterogeneity). CONCLUSION: We generated a comprehensive bank of possible errors that can arise in the application and interpretation of meta-analyses in systematic reviews of interventions. This item bank of errors provides the foundation for developing a checklist to help peer reviewers detect statistical errors.

Content development for a new item-bank for measuring multifocal contact lens performance.

BACKGROUND: Presbyopia is an age-related condition that causes a decreased ability to focus on nearby objects. Multifocal contact lenses are commonly used to address this issue. However, there seems to be a notable dissatisfaction among multifocal contact lens wearers. The absence of a reliable instrument to measure the patient's perspective, despite the widespread use of this method, highlights the need for further research in this area. OBJECTIVE: The objective of this study is to develop an item-bank integrating all domains necessary to assess the patient's perspective on multifocal contact lens performance, offering a comprehensive measure. The item-bank will ensure a high level of content validity, be self-administered, and will initially be available in Spanish. The aim of this tool is to serve as a valuable resource for research and optometric clinics, facilitating the follow-up of patients with presbyopia who wear multifocal contact lenses or those who are newly starting to use them. METHODOLOGY: The MCL-PRO item bank, followed a systematic and step-wise inductive approach to gather information, following the recommendations outlined in the COSMIN guidelines and similar studies. The process involved the following steps: (1) Literature review and relevant existing items identification (2) Social media review, (3) Semi-structured focus groups, (4) performing qualitative analysis, (5) refining and revising the items, and (6) generating the content of the item bank. RESULTS: A total of 575 items were included in the item-bank hosted under 8 different domains that were found to be important for presbyopic population: visual symptoms (213), activity limitation (111), ocular symptoms (135), convenience (36), emotional well-being (33), general symptoms (16), cognitive issues (21) and economic issues (10). CONCLUSION: The item-bank created has followed standardised methodology for its development and encloses all the aspects for MCL performance evaluation from patients perspective.

The first steps in the development of a cancer-specific patient-reported experience measure item bank (PREM-item bank): towards dynamic evaluation of experiences.

OBJECTIVE: Since the implementation of value-based healthcare, there has been a growing emphasis on utilizing patient-reported experience measures (PREMs) to enhance the quality of care. However, the current PREMs are primarily generic and static, whereas healthcare is constantly evolving and encompasses a wide variety of aspects that impact care quality. To continuously improve care requires a dynamic PREM. The aim of this study was to propose an item bank for the establishment of a dynamic and care-specific patient-reported evaluation. METHODS: In co-creation with patients, a mixed methods study was conducted involving: (1) an explorative review of the literature, (2) a focus group analysis with (ex-)patients, (3) qualitative analyses to formulate themes, and (4) a quantitative selection of items by patients and experts through prioritization. RESULTS: Eight existing PREMs were evaluated. After removing duplicates, 141 items were identified. Through qualitative analyses of the focus group in which the patient journey was discussed, eight themes were formulated: "Organization of healthcare," "Competence of healthcare professionals," "Communication," "Information & services," "Patient empowerment," "Continuity & informal care," "Environment," and "Technology." Seven patients and eleven professionals were asked to prioritize what they considered the most important items. From this, an item bank with 76 items was proposed. CONCLUSION: In collaboration with patients and healthcare professionals, we have proposed a PREM-item bank to evaluate the experiences of patients' receiving cancer care in an outpatient clinic. This item bank is the first step to dynamically assess the quality of cancer care provided in an outpatient setting.

Development of standard computerised adaptive test (CAT) settings for the EORTC CAT Core.

AIMS: Computerised adaptive test (CAT) provides individualised patient reported outcome measurement while retaining direct comparability of scores across patients and studies. Optimal CAT measurement requires an appropriate CAT-setting, the set of criteria defining the CAT including start item, item selection criterion, and stop criterion. The European Organisation for Research and Treatment of Cancer (EORTC) CAT Core allows for assessing the 14 functional and symptom domains covered by the EORTC QLQ-C30 questionnaire. The aim was to present a general approach for selecting CAT-settings and to use this to develop a portfolio of standard settings for the EORTC CAT Core optimised for different purposes and populations. METHODS: Using simulations, the measurement properties of CATs of different length and precision were evaluated and compared allowing for identifying the most suitable settings. All CATs were initiated with the most informative QLQ-C30 item. For each domain two fixed-length and two fixed-precision standard CATs were selected focusing on efficiency (brief version) and precision (long), respectively. RESULTS: The brief fixed-length CATs included 3-5 items each while the long versions included 5-8 items. The fixed-precision CATs aimed for reliability of 0.65-0.95 (brief versions) and 0.85-0.98 (long versions), respectively. Median sample size savings using the CATs compared to the QLQ-C30 scales ranged 20%-31%, although savings varied considerably across the domains. CONCLUSION: The EORTC CAT Core standard settings simplify selection of relevant and appropriate CATs. The CATs prioritise either brevity and efficiency or precision, but all provide increased measurement precision and hence, reduced sample size requirements compared to the QLQ-C30 scales. The CATs may be used as they are or modified to accommodate specific requirements.

Measurement of visual functioning following first and second eye cataract surgery using Vision-Related Activity Limitation Item Bank.

PURPOSE: This study aims to compare visual functioning (VF) after first or second eye cataract surgery using the vision-related activity limitation (VRAL) item bank. METHODS: This prospective, interventional study included 787 patients (mean age, 58.2 years) with cataract undergoing cataract surgery (first eye surgery with/out ocular comorbidity, second eye surgery with/out ocular comorbidity) at a tertiary eye care center, South India, who were administered the item bank pre- and at 6 weeks postoperatively to assess change in VF. Rasch analysis was used to estimate VF at both time points, and responsiveness to cataract surgery was calculated as effect size (ES) which was interpreted as small (≤ 0.2), moderate (0.3-0.7), and large (≥ 0.8). RESULTS: Mean best-corrected logMAR VA in surgical eye improved significantly postoperatively compared to preoperative VA (0.20 ± 0.40 vs. 1.19 ± 0.96; P < 0.0001) across all groups. Patients reported significant and large improvements in VF postoperatively across all groups: largest ES for first eye surgery without comorbidity (1.87 [95% CI, 1.61, 2.13]) and smallest for second eye without ocular comorbidity (1.55 [95% CI, 1.22, 1.88]). Compared to patients undergoing second eye surgery, first eye surgery patients reported significantly lower VF preoperatively (-0.72 ± 2.39 vs. 0.17 ± 2.34 logits; P < 0.0001), and a larger change in VF postoperatively (3.71 ± 2.33 logits vs. 4.27 ± 2.83 vs.; P = 0.004). CONCLUSIONS: Cataract surgery resulted in large and significant improvements in VF, regardless of ocular comorbidity and first or second eye surgery. The VRAL item bank is a useful tool to measure responsiveness to cataract surgery.

Psychometric properties of the Chinese version of the PROMIS-Cancer-Anxiety item bank assessed using a graded response model.

Objective: This study aimed to examine the psychometric properties of the Chinese version of the Patient-Reported Outcome Measurement Information System (PROMIS)-Cancer-Anxiety item bank using a graded response model in a sample of patients with cancer. Methods: A cross-sectional study was conducted and the Chinese version of the PROMIS-Cancer-Anxiety item bank was used to measure anxiety in patients with cancer. The unidimensional structure of the item bank was evaluated using principal component analysis. Residual correlations and the graphs of item mean scores conditional on the rest scores were examined to evaluate the local independence and monotonicity of the items, respectively. Item characteristics were described using item parameter estimates and item information. Operating characteristic curves (OCCs) and test information curve (TIC) were also plotted. Measurement invariance across age, gender, and education level was assessed to identify possible differential item functioning (DIF). Results: A total of 1075 patients with cancer were enrolled. Under the assumptions of unidimensionality, local independence, and monotonicity, the discrimination parameters a ranged from 2.30 to 5.47, and the threshold parameters b ranged from b1 = -2.87 to b4 = 3.21 with proper intervals. Completely overlapped category curves were not observed among the OCCs of any items. Item information and TIC showed that the item bank had a wide measurement range. The DIFs for age, gender, and education level for all items were not remarkable. Conclusions: The results supported using the Chinese version of the PROMIS-Cancer-Anxiety item bank to measure anxiety and develop a computerized adaptive testing (CAT) system for anxiety in patients with cancer.

Development and psychometric evaluation of item banks for memory and attention - supplements to the EORTC CAT Core instrument.

BACKGROUND: Cancer patients may experience a decrease in cognitive functioning before, during and after cancer treatment. So far, the Quality of Life Group of the European Organisation for Research and Treatment of Cancer (EORTC QLG) developed an item bank to assess self-reported memory and attention within a single, cognitive functioning scale (CF) using computerized adaptive testing (EORTC CAT Core CF item bank). However, the distinction between different cognitive functions might be important to assess the patients' functional status appropriately and to determine treatment impact. To allow for such assessment, the aim of this study was to develop and psychometrically evaluate separate item banks for memory and attention based on the EORTC CAT Core CF item bank. METHODS: In a multistep process including an expert-based content analysis, we assigned 44 items from the EORTC CAT Core CF item bank to the memory or attention domain. Then, we conducted psychometric analyses based on a sample used within the development of the EORTC CAT Core CF item bank. The sample consisted of 1030 cancer patients from Denmark, France, Poland, and the United Kingdom. We evaluated measurement properties of the newly developed item banks using confirmatory factor analysis (CFA) and item response theory model calibration. RESULTS: Item assignment resulted in 31 memory and 13 attention items. Conducted CFAs suggested good fit to a 1-factor model for each domain and no violations of monotonicity or indications of differential item functioning. Evaluation of CATs for both memory and attention confirmed well-functioning item banks with increased power/reduced sample size requirements (for CATs ≥ 4 items and up to 40% reduction in sample size requirements in comparison to non-CAT format). CONCLUSION: Two well-functioning and psychometrically robust item banks for memory and attention were formed from the existing EORTC CAT Core CF item bank. These findings could support further research on self-reported cognitive functioning in cancer patients in clinical trials as well as for real-word-evidence. A more precise assessment of attention and memory deficits in cancer patients will strengthen the evidence on the effects of cancer treatment for different cancer entities, and therefore contribute to shared and informed clinical decision-making.

Constructing country-specific quality-of-life item banks for adults with amblyopia and strabismus in Australia and India.

CLINICAL RELEVANCE: Understanding the quality-of-life (QoL) impacts of amblyopia and strabismus from the perspectives of patients using validated tools would enable eye care practitioners to provide better clinical management. BACKGROUND: Item banks are advanced patient reported outcome measures with several advantages over traditional QoL assessment. This paper describes the development of amblyopia and strabismus-specific QoL item banks for two distinct country settings: Australia (high-income) and India (low-middle income) and examines the unique QoL issues. METHODS: Using a bottom-up systematic approach, the content for the item banks was identified from three sources: existing self-report amblyopia and strabismus questionnaires (n = 22), qualitative literature (n = 5) and prospective qualitative studies in Australia (n = 49) and India (n = 30). The initial item pool underwent item evaluation, construction, and pre-testing to form optimal sets of representative items. The Indian item pools were first developed in English and translated into Hindi and Tamil using a rigorous translation protocol. The differences in QoL experiences that emanated from the qualitative studies and the number of common and unique items in the final item pools were compared. RESULTS: The final Australian and Indian item pools comprised 312 and 277 items, respectively, covering 11 QoL domains. Two hundred and sixty items (79%) were common to both countries, and 21% were unique. Of the 11 domains, except for activity limitation (64.5%), visual symptoms (73.3%) and emotional impact (75.5%) domains, all other domains had over 80% of items common to both countries. The unique items can be attributed to differences in QoL experiences, individual perspectives, culture, lifestyle, country setting and health systems. CONCLUSION: Amblyopia has a multifaceted impact on QoL irrespective of the country settings. Despite a huge overlap in QoL impacts between Australia and India, both countries had unique issues, especially activity limitations. The study developed comprehensive, country-specific item pools for Australia and India.

A successful intervention to improve conceptual knowledge of medical students who rely on memorization of disclosed items.

Background: The mere memorization of isolated facts without the claim of integrating them is detrimental to the desired learning outcomes in medical education. The current study was conducted within an environment where items from summative assessments are regularly disclosed by the university and consequently collected into an item bank that is shared between students. Within this environment, we aimed to quantify 1) to which extent students use disclosed items for their preparation for the summative exam, 2) a putative mismatch between (isolated factual) knowledge regarding distinct questions from the item bank and conceptual knowledge, and 3) to which extent this mismatch can be ameliorated by a project aiming to steer student learning away from the memorization of isolated facts toward the acquisition of conceptual knowledge. Methods: This steering project in the midst of the first semester consisted of the implementation of an oral exam based on selected learning objectives, preceded by two seminars. After their summative exam at the end of semester, 135 students performed a voluntary exam for study purposes. Here, authentic (i.e., presumably preknown) physiology questions taken from the item bank were used to assess students' ability to 1) recognize the correct answer in a multiple choice (MC) question, 2) recall the answer (short answer), or 3) display conceptual knowledge closely corresponding to the question presented in the other formats. Additionally, students received a questionnaire addressing their learning habits and attitudes. Results: The median reported percentage of learning time for the summative exam exclusively spent with this item bank was 80%. The results of the voluntary exam indicate that students frequently recognize and recall correct answers of included items without displaying knowledge of the underlying concept. Compared to recall of the correct answer, the probability of giving a correct answer regarding the corresponding basal physiologic concept was lower by 47 percentage points (p <0.001) for topics not included in the steering project. Regarding topics included in the steering project, this discrepancy was reduced to 25.5% (p <0.001). Conclusion: The results of this study demonstrate the influence of disclosed items on student learning and learning outcomes and suggest that a carefully implemented assessment is able to improve conceptual knowledge in physiology.

Construction of a reading literacy test item bank for fourth graders based on item response theory.

Introduction: Reading literacy is not only central to students' academic success during their school years but also crucial to their personal development in their later life. The development of assessment instruments for reading literacy has been of interest to researchers, educators and educational administrators. The purpose of the present study was to construct and validate a comparable item bank for assessing fourth-grade students' reading literacy. Methods: One hundred fifteen reading comprehension items were developed and administered to 2,174 Grade 4 students to construct an item bank. Using the test equating technique and balanced incomplete block design, we divided participants into 10 subgroups, and the 115 items were further assigned into 10 test forms. Item response theory software was used to estimate discrimination, items' threshold parameters, and students' ability parameters. The criterion-related validity was also examined in 135 Grade 4 students who completed the reading literacy test and verbal self-description questionnaire. Results: The final item bank included 99 reading performance indicators to express high achievement. The correlation between the students' reading literacy and the verbal self-description questionnaire was significant and demonstrated the item bank's good criterion-related validity. The item bank developed in this study shows good psychometric characteristics and can be used to assess the reading literacy of fourth graders.

Development of a diverse set of standard short forms based on the EORTC CAT Core item banks.

PURPOSE: The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Group has developed item banks covering the 14 domains of the EORTC QLQ-C30 quality of life questionnaire. These allow for dynamic assessment and for forming population/study specific static short forms. To simplify selection of relevant short forms, we here present a portfolio of standard short forms with measurement properties optimized for different populations. METHODS: For each domain, a brief and a long version were constructed for each of three populations having mild, moderate, and severe symptoms, respectively. The most informative items were prioritised while also taking content into consideration. All short forms included at least one QLQ-C30 item. The measurement precision/power of the short forms was compared to the corresponding QLQ-C30 scales using simulations. RESULTS: In total, 84 short forms were constructed. The brief versions included 3-5 items each, the long versions 5-9 items. Estimated sample size savings using the suggested short forms while maintaining the same power as with the QLQ-C30 ranged 3-50% across domains with median savings of 19% (brief versions) and 28% (long versions), respectively. CONCLUSION: The suggested short forms allow for simple selection of items particularly relevant for patients with mild, moderate, or severe symptoms, respectively. They facilitate the use of smaller samples without loss of power compared to the QLQ-C30 scales. The suggested short forms may be used as they are or adapted to the specific aims of individual studies/settings.

Development of a Patient-Reported Symptom Item Bank for Patients with Hepatobiliary or Pancreatic Malignancies: A Systematic Review.

Background: Patients with hepatobiliary or pancreatic cancers often experience severe symptoms, resulting in a sharp decline in functioning, poor quality of life, and increased mortality risk. Early and effective management of symptoms allows a better quality of life and reduced mortality, depending on the selection of appropriate assessment of specific symptoms for a defined purpose. We aimed to develop a symptom measurement item bank for hepatobiliary or pancreatic cancers. Methods: The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was applied to organize this systematic review. The articles validated patient-reported outcome measures (PROMs) for hepatobiliary or pancreatic cancer and published before December 2021 were retrieved from the Web of Science, PubMed, Embase databases and Cochrane Library. Items from the existing PROMs were selected and classified into different patient-reported symptoms based on the concepts and specific underlying constructs of the objects measured. Results: Sixteen unique PROMs were identified across the 29 eligible studies included in our analysis. Items from the literature review (14 PROMs with 421 items for which information was obtained) were selected and classified. As a result of this study, we developed a symptom item bank with 40 patient-reported symptoms and 229 assessment items for hepatobiliary or pancreatic cancer, and fatigue, pain and nausea were the most common symptom items. Conclusion: We developed an item bank to assess the patient-reported symptoms of hepatobiliary or pancreatic cancer. This item bank could allow researchers to select appropriate measures of symptom and provide a basis for the development of a single-item symptom-measurement system.

On-the-fly parameter estimation based on item response theory in item-based adaptive learning systems.

Online learning systems are able to offer customized content catered to individual learner's needs, and have seen growing interest from industry and academia alike in recent years. In contrast to the traditional computerized adaptive testing setting, which has a well-calibrated item bank with new items added periodically, the online learning system has two unique features: (1) the number of items is large, and they have likely not gone through costly field testing for item calibration; and (2) the individual's ability may change as a result of learning. The Elo rating system has been recognized as an effective method for fast updating of item and person parameters in online learning systems to enable personalized learning. However, the updating parameter in Elo has to be tuned post hoc, and Elo is only suitable for the Rasch model. In this paper, we propose the use of a moment-matching Bayesian update algorithm to estimate item and person parameters on the fly. With sequentially updated item and person parameters, a modified maximum posterior weighted information criterion (MPWI) is proposed to adaptively assign items to individuals. The Bayesian updated algorithm along with MPWI is validated in a simulated multiple-session online learning setting, and the results show that the new combo can achieve fast and reasonably accurate parameter estimations that are comparable to random selection, match-difficulty selection, and traditional online calibration. Moreover, the combo can still function reasonably well with as low as 20% of items being pre-calibrated in the item bank.

The development of a glaucoma-specific health-related quality of life item bank supporting a novel computerized adaptive testing system in Asia.

BACKGROUND: A glaucoma-specific health-related quality of life (HRQoL) item bank (IB) and computerized adaptive testing (CAT) system relevant to Asian populations is not currently available. We aimed to develop content for an IB focusing on HRQoL domains important to Asian people with glaucoma; and to compare the content coverage of our new instrument with established glaucoma-specific instruments. METHODS: In this qualitative study of glaucoma patients recruited from the Singapore National Eye Centre (November 2018-November 2019), items/domains were generated from: (1) glaucoma-specific questionnaires; (2) published articles; (3) focus groups/semi-structured interviews with glaucoma patients (n = 27); and (4) feedback from glaucoma experts. Data were analyzed using the constant comparative method. Items were systematically refined to a concise set, and pre-tested using cognitive interviews with 27 additional glaucoma patients. RESULTS: Of the 54 patients (mean ± standard deviation [SD] age 66.9 ± 9.8; 53.7% male), 67 (62.0%), 30 (27.8%), and 11 (10.2%) eyes had primary open angle glaucoma, angle closure glaucoma, and no glaucoma respectively. Eighteen (33.3%), 11 (20.4%), 8 (14.8%), 12 (22.2%), and 5 (9.3%) patients had no, mild, moderate, severe, or advanced/end-stage glaucoma (better eye), respectively. Initially, 311 items within nine HRQoL domains were identified: Visual Symptoms, Ocular Comfort Symptoms, Activity Limitation, Driving, Lighting, Mobility, Psychosocial, Glaucoma management, and Work; however, Driving and Visual Symptoms were subsequently removed during the refinement process. During cognitive interviews, 12, 23 and 10 items were added, dropped and modified, respectively. CONCLUSION: Following a rigorous process, we developed a 221-item, 7-domain Asian glaucoma-specific IB. Once operationalised using CAT, this new instrument will enable precise, rapid, and comprehensive assessment of the HRQoL impact of glaucoma and associated treatment efficacy.

Delirium Item Bank: Utilization to Evaluate and Create Delirium Instruments.

INTRODUCTION: The large number of heterogeneous instruments in active use for identification of delirium prevents direct comparison of studies and the ability to combine results. In a recent systematic review we performed, we recommended four commonly used and well-validated instruments and subsequently harmonized them using advanced psychometric methods to develop an item bank, the Delirium Item Bank (DEL-IB). The goal of the present study was to find optimal cut-points on four existing instruments and to demonstrate use of the DEL-IB to create new instruments. METHODS: We used a secondary analysis and simulation study based on data from three previous studies of hospitalized older adults (age 65+ years) in the USA, Ireland, and Belgium. The combined dataset included 600 participants, contributing 1,623 delirium assessments, and an overall incidence of delirium of about 22%. The measurements included the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition diagnostic criteria for delirium, Confusion Assessment Method (long form and short form), Delirium Observation Screening Scale, Delirium Rating Scale-Revised-98 (total and severity scores), and Memorial Delirium Assessment Scale (MDAS). RESULTS: We identified different cut-points for each existing instrument to optimize sensitivity or specificity, and compared instrument performance at each cut-point to the author-defined cut-point. For instance, the cut-point on the MDAS that maximizes both sensitivity and specificity was at a sum score of 6 yielding 89% sensitivity and 79% specificity. We then created four new example instruments (two short forms and two long forms) and evaluated their performance characteristics. In the first example short form instrument, the cut-point that maximizes sensitivity and specificity was at a sum score of 3 yielding 90% sensitivity, 81% specificity, 30% positive predictive value, and 99% negative predictive value. DISCUSSION/CONCLUSION: We used the DEL-IB to better understand the psychometric performance of widely used delirium identification instruments and scorings, and also demonstrated its use to create new instruments. Ultimately, we hope that the DEL-IB might be used to create optimized delirium identification instruments and to spur the development of a unified approach to identify delirium.

Identifying the content for an item bank and computerized adaptive testing system to measure the impact of age-related macular degeneration on health-related quality of life.

PURPOSE: We are developing an age-related macular degeneration (AMD) health-related quality of life (HRQoL) item bank, applicable to Western and Asian populations. We report primarily on content generation and refinement, but also compare the HRQoL issues reported in our study with Western studies and current AMD-HRQoL questionnaires. METHODS: In this cross-sectional, qualitative study of AMD patients attending the Singapore National Eye Centre (May-December 2019), items/domains were generated from: (1) AMD-specific questionnaires; (2) published articles; (3) focus groups/semi-structured interviews with AMD patients (n = 27); and (4) written feedback from retinal experts. Following thematic analysis, items were systematically refined to a minimally representative set and pre-tested using cognitive interviews with 16 AMD patients. RESULTS: Of the 27 patients (mean ± standard deviation age 67.9 ± 7.0; 59.2% male), 18 (66.7%), two (7.4%), and seven (25.9%) had no, early-intermediate, and late/advanced AMD (better eye), respectively. Whilst some HRQoL issues, e.g. activity limitation, mobility, lighting, and concerns were similarly reported by Western patients and covered by other questionnaires, others like anxiety about intravitreal injections, work tasks, and financial dependency were novel. Overall, 462 items within seven independent HRQoL domains were identified: Activity limitation, Lighting, Mobility, Emotional, Concerns, AMD management, and Work. Following item refinement, items were reduced to 219, with 31 items undergoing amendment. CONCLUSION: Our 7-domain, 219-item AMD-specific HRQoL instrument will undergo psychometric testing and calibration for computerized adaptive testing. The future instrument will enable users to precisely, rapidly, and comprehensively quantify the HRQoL impact of AMD and associated treatments, with item coverage relevant across several populations.

Swedish translation and cross-cultural adaptation of eight pediatric item banks from the Patient-Reported Outcomes Measurement Information System (PROMIS)®.

BACKGROUND: This study is part of the Swedish initiative for the establishment of standardized, modern patient-reported measures for national use in Swedish healthcare. The goal was to translate and culturally adapt eight pediatric Patient-Reported Outcomes Measurement Information System (PROMIS®) item banks (anger, anxiety, depressive symptoms, family relationships, fatigue, pain interference, peer relationships and physical activity) into Swedish. METHODS: Authorization to translate all currently available pediatric PROMIS item banks (autumn, 2016) into Swedish was obtained from the PROMIS Health Organization. The translation followed the Functional Assessment of Chronic Illness Therapy translation recommendations with one major modification, which was the use of a bilingual multi-professional review workshop. The following steps were applied: translation, reconciliation, a two-day multi professional reviewer workshop, back translation, and cognitive debriefing with eleven children (8-17 years) before final review. The bilingual multi-professional review workshop provided a simultaneous, in-depth assessment from different professionals. The group consisted of questionnaire design experts, researchers experienced in using patient-reported measures in healthcare, linguists, and pediatric healthcare professionals. RESULTS: All item banks had translation issues that needed to be resolved. Twenty-four items (20.7%) needed resolution at the final review stage after cognitive debriefing. The issues with translations included 1. Lack of matching definitions with items across languages (6 items); 2. Problems related to language, vocabulary, and cultural differences (6 items); and 3. Difficulties in adaptation to age-appropriate language (12 items). CONCLUSIONS: The translated and adapted versions of the eight Swedish pediatric PROMIS item banks are linguistically acceptable. The next stage will be cross-cultural validation studies in Sweden. Despite the fact that there are cultural differences between Sweden and the United States, our translation processes have successfully managed to address all issues. Expert review groups from already-established networks and processes regarding pediatric healthcare throughout the country will facilitate the future implementation of pediatric PROMIS item banks in Sweden.