Reliability and validity of the Netherlands Triage Standard in emergency care settings: a case scenario study.

BACKGROUND: The Netherlands Triage Standard (NTS) is a triage system that can be used by different types of emergency care organisations. Our objective was to determine the interrater reliability and construct validity of the NTS when applied to self-presenting patients. METHODS: We performed a cross-sectional case scenario study consisting of two parts: (1) paediatric triage in January-February 2019 and (2) adult triage in October-November 2020. In each part, we invited nurse triagists from three general practitioner cooperatives, three ambulance dispatching centres and three hospital emergency departments in the Netherlands to participate. We used 40 case scenarios involving paediatric patients and 41 involving adult patients who could self-present to any emergency care organisation. In advance, an expert panel determined the urgency (six levels) of the case scenarios (reference standard). The main outcome for reliability was the intraclass correlation coefficient (ICC) for urgency level. The main outcomes for validity were degree of agreement with the reference standard, for urgency level, and sensitivity and specificity for high versus low urgency. We used descriptive statistics and logistic multilevel modelling with both case and triagist as random effects. RESULTS: 218 out of 240 invited triagists participated. The ICC among all triagists was 0.73 for paediatric cases and 0.88 for adult cases and was highest in general practitioner cooperatives. For paediatric cases, there was 62.3% agreement with the reference standard about urgency, 17.4% underestimation and 20.2% overestimation. The sensitivity of the NTS for identifying highly urgent paediatric cases was 85.2%; the specificity was 89.7%. For adult cases, there was 68.3% agreement, 13.7% underestimation and 18.0% overestimation. The sensitivity of triage for high urgency in adults was 94.5% and the specificity 83.3%. CONCLUSION: NTS appears to have good reliability and construct validity for estimating the urgency of health complaints of non-referred patients presenting themselves in emergency care.

[The Netherlands Triage Standard applied to paediatric patients in emergency medicine: is it reliable and valid?] / Toepassing van de Nederlandse Triage Standaard bij kinderen in de spoedzorg.

OBJECTIVE: To determine the inter-rater reliability and validity of the Netherlands Triage Standard (NTS) for paediatric triage. DESIGN: A cross-sectional study using fictional cases for telephone and physical triage. METHOD: An expert panel established in advance the urgency of 40 cases concerning emergency help requests from non-referred children (the reference standard). These requests were presented in an online survey to triagists from three general practitioner (GP) out-of-hours practices, three ambulance dispatching centres and three hospital emergency departments. Triagists assessed all cases, using the NTS. We determined the agreement on degrees of urgency between different triagists and compared them with the reference standard. The outcome measure for inter-rater reliability was the intraclass correlation coefficient (ICC). The outcome measures for validity were the degree of agreement with the reference standard, under-triage and over-triage, and sensitivity and specificity in identifying high-urgency (U0-U2) versus low-urgency cases (U30U5). RESULTS: In total, 116 triagists participated in the study (response: 86%). The ICC was 0.73 among all triagists, and was highest in the out-of-hours GP cooperatives. There was 62.3% agreement with the reference standard, 17.4% under-triage and 20.2% over-triage. Of the divergent urgencies, 77% differed by only one urgency category. The sensitivity was 85.2% and the specificity 89.7%. The sensitivity and specificity of triage by the GP out-of-hours practices (82.7% and 92.7%, respectively) were almost the same as that by the hospital emergency departments (79.6% and 92.5%, respectively). Triage by the ambulance dispatching centres had relatively high sensitivity (93%), but relatively low specificity (82.4%). CONCLUSION: The results of the study contribute to the evidence that the NTS is a reliable and valid triage standard for paediatric patients. The urgency assessments by triagists in the GP out-of-hours practices, ambulance dispatching centres and hospital emergency departments were broadly in agreement. Results were limited by cases being on paper and triage only on anamnestic characteristics.