40 years external quality assessment--what have we achieved? A personal view.

The present report deals with a personal interaction with different external quality assessment (EQA) programmes over the past four decades for different groups of analytes. The effects of national and international legislation and standardization, as well as certification and accreditation standards have also been taken into account. The effects and limitations of reference method procedures and reference materials are also discussed as are the trends in accuracy and precision, the latter especially in terms of automation and decentralization of laboratory analyses. The effects seen are not only positive and, often, legislation and commercial interests are in conflict with the primary aims of EQA and standardization.

An educational web-based external quality assessment outcome and evaluation: first experiences with urinary sediment and hemostaseology.

External quality assessment is a standard procedure for many medical laboratories, especially those accredited according to ISO 15189. INSTAND has developed web-based quality control surveys (WQ) with integrated case-based learning in collaboration with the Institute for Teaching and Educational Research in Health Sciences (IDBG), University of Witten/Herdecke. The WQs were presented via a weblink using the learning management system CASUS. Laboratories registered with INSTAND for conventional EQAs covering the same topic were invited to participate in the WQs. Statistics were calculated with the INSTAND EQA-evaluation program and with the "Statistical Program for Social Sciences" (SPSS Version 20). While the two pilot surveys (hemostaseology) were designed for laboratory technicians, WQ III (urine sediment) addressed the whole laboratory team. From the invited laboratories more than 80% participated successfully and the evaluations showed that, in general, more than half the participants solved the tasks as a team. The web-based EQAs were well accepted by the users, with some technical problems in WQ III and moderate criticism on the images and relevance for daily practice in the evaluation sheets. Well above 80% would like to participate in further WQs and would recommend them to other laboratories. Thus web-based quality control surveys can emerge as an important supplement to conventional EQAs.

Experience with an alternative form of samples for external quality assessment of urinary sediment (visual sample EQA).

BACKGROUND: Urinary sediment components are mainly unstable, especially in their original forms. This makes the use of such material in routine external quality assessment surveys (EQAS) almost impossible. The development of an alternative EQAS for urinary sediment using photomicrographs (visual-sample EQAS) is described here, together with results, improvements and future plans. METHODS: The original survey allowed a free-text interpretation, which proved to be time consuming and impractical. The next phase included a multiple choice answer (one from five alternatives). The current version challenges the participant with a list of over 50 alternatives, together with a "not-in-the-list" possibility. RESULTS: Certain elements are always interpreted more correctly (examples: erythrocytes, squamous cells, renal epithelial cells, bacteria--above 90%) than others (examples: leukocytes, fat droplets, certain crystals--below 55%). Pictures are included which are atypical (for example: decoy cells, which must be interpreted correctly, other artefacts). Occasionally interesting structures are included, but which are not evaluated. CONCLUSIONS: The two-dimensional pictures often limited the interpretation possibilities, so that in such cases more than one answer was allowed. The possibility of "online microscopy" using staple pictures is at present in evaluation. The acceptance of this type of EQAS is seen by the number of participants, at present over 500 per survey, four surveys per year.

Has the introduction of the European Parliament Directive 98/79/EC from 1998-10-27--concerning in-vitro diagnostic devices (IVD-directive) influenced performance of hormone immunoassay kits in terms of precision and accuracy? Results from selected external quality assessment (EQA) schemes between 1993 and 2008.

The aim of this report is to examine what--if any--effects the introduction of the European Directive 98/79/EC has had on the performance of in-vitro diagnostic devices (kits) used in determining hormones and related measurands in medical diagnostic laboratories. The observations covered the period from 1993-2008, results being taken from EQA surveys at three year intervals. The measurands chosen were: cortisol, progesterone, testosterone, 17beta oestradiol (E2), free triiodothyronine (fT3), total triiodothyronine (TT3), free thyroxine (fT4), total thyroxine (TT4), thyrotropin (TSH), luteotropin (LH), follitropin (FSH), prolactin (PRL), parathyrin (PTH), thyroglobulin (Tg), anti-thyroid microsomes (MAb), anti-thyroid peroxidase (Anti-TPO), anti-thyroglobulin (Anti-Tg) and anti-TSH-receptor (TRAb). The results showed the following trends and tendencies: a. The precision has improved over the observation period, mainly due to automation made possible by the use of non-radioisotopic labelling. b. The accuracy--in terms of comparison with reference method values--has not been influenced by the Directive 98/79/EC. c. Introduction of international standards and reference preparations does not itself result in better comparison of results between kits from different manufacturers. d. Harmonisation in methodology has led to improved comparison between different manufacturers--an example being the assay of thyroid-receptor antibodies, where components are often only available from one producer.

The determination of uric acid in urine--forgotten problems rediscovered in an external quality assessment scheme.

This article mainly describes the effects of boric acid and borates often used as bacteriostatic agents in urine collection for microbiological examination, on the results obtained in the measurement of uric acid in urine using the uricase [EC.1.7.3.3] method. The bias in results is not unidirectional, the spread of results being much larger than in urine samples not containing borate. Borate ions are also known to inhibit other enzymes such as urease [EC 3.5.1.5]. Results are presented from six national external quality assessment (EQA) surveys carried out in 2007 by INSTAND in Düsseldorf, Germany. Whereas the performance--expressed as the success rate--in samples not containing borate was acceptable (between 90 and 95%), this was not the case where borate was present (success rate 68-72%). The results also showed systematic differences for different kits which were partly due to differences in standardisation/calibration and partly due to interference by borate. Some effects of ascorbic acid and sodium azide on the determination of uric acid in urine using uricase have also been presented. The results show that care must be taken in the preanalytical phase, especially in referral laboratories, in order to prevent wrong interpretation of results due to methodological failure in the unknown presence of borate ions.

Comparison of performance of classical clinical chemistry analysers with test-strip devices (Reflotron) and those based on film technology (Vitros) in external quality assessment (EQA) surveys.

This article reports on the performance of two "dry" chemistry devices, (Reflotron, Roche Diagnostics and Vitros, Johnson & Johnson) and compared them with classical "wet" chemistry analysers in four commercially produced quality assessment samples (Roche PNU and PPU and Seronorm Human and Human High Controls) sent repeatedly over a 12-month observation period. Eleven analytes (including five enzymes) were studied, eight of which had target values set by reference method procedures. The results showed that both devices gave comparative results for the same sample sent in different EQA-surveys. Statistically significant differences which occurred were due to the high precision of measurement with a minimal shift in the measured concentrations. They had no clinical relevance in interpretation of results. Comparisons between "dry" and "wet" chemistry results for the same analyte were almost always statistically significantly different and often large enough to influence the clinical interpretation of results. Examples here were glucose and uric acid measured with the Reflotron and compared with other Roche devices (Cobas, Hitachi). The Vitros showed deviant values for urea and creatinine, when compared with other measuring devices using liquid reagents. Differences seen were constant over time, but must be seen in context with the matrices of the samples sent. The results show the long term stability of both reagents and test kits, a necessary prerequisite for long-term controlling of precision and indirectly accuracy of patient measurements.

Sample commutability in external quality assessment surveys (EQAS) for thyroid-related antibodies--state of the art.

External quality assessment surveys for thyroid-related antibodies have been offered by INSTAND for 20 years. During this time, some problems have remained, especially those between the similarity of samples sent and routine patient samples. Here the questions of "matrix effects" and "commutability of results" are topics discussed at most EQA-meetings. This short communication deals with the effects of lyophilization on results for thyroid-associated antibodies in an EQA-survey carried out by over 230 participants in October 2005 by INSTAND in Düsseldorf, Germany. The results show that there are small but statistically significant differences (at the level p < 0.05) in results from liquid and lyophilized samples from the same source, the tendency to higher results in the lyophilized samples for anti-TPO and anti-Tg and to lower results for TRAb. The precision in measurement was significantly better for anti-TPO and TRAb in the lyophilized samples, there being no significant difference for anti-Tg. The differences in results between liquid and lyophilized samples were minimal when compared with the numerical results for anti-TPO and anti-Tg, despite the fact that all kits were calibrated with the NIBSC reference materials, although the latter are now both 40 years old.

Problems with the external quality assessment of accuracy of point of care devices (POCD) for blood glucose are independent of sample composition.

This short article describes the results obtained in both internal and external quality assessment of point of care devices (POCD) for the monitoring of blood glucose. The results show that the use of synthetic, serum and whole blood matrices for the samples do not markedly change the inaccuracy of measurement. It is only possible to check precision of POC devices for glucose in external quality assessment (EQA) surveys for POC devices for blood glucose. The majority of devices used in point of care testing for blood glucose had acceptable (im)precision. The applications and limitations of the use of POC devices for blood glucose must be clearly defined before allowing their use in patient care programmes. Finally, the results confirm that the decision of the German Federal Medical Council (Bundesärztekammer) to exclude POC devices from accuracy checks against a reference method procedure in national EQA surveys was correct, at least at the present time.

Some practical thoughts on restructuring the current guideline of the Federal Medical Council (Richtlinie der Bundesärztekammer [RiliBAK]) in Germany for quality control of clinical laboratory analyses based on results from external quality assessment surveys.

The present article presents a critical review of the current guidelines of the Federal Medical Council (Richtlinie der Bundesärztekammer--[RiliBAK]) in Germany, both for internal and external quality control. Examples have been chosen for analytes which present problems. These include thyrotropin (TSH) and human chorionic gonadotropin (hCG). Data are presented for the difference in analyte concentrations found between methods/kits and longitudinally for external quality assessment (EQA-) surveys for analytes using reference method values as target values. These include: calcium, aspartate aminotransferase (ASAT; GOT), aldosterone, cortisol, 17beta-oestradiol and total thyroxine (TT4). Furthermore, internal data from participating laboratories concerning laboratory internal precision and accuracy have been analysed to show the state of the art with regard to both parameters. Data analysis shows the need for a reintroduction of concentration-dependent assessment of performance (crossover system) for certain analytes, where clinically relevant concentrations are measured at the extremes of the calibration curve. The current guideline is at present being restructured and should include considerations made in this review of the state-of-the-art of clinical laboratory analysis.

External quality assessment of tumour marker analysis: state of the art and consequences for estimating diagnostic sensitivity and specificity.

This review shows the current analytical quality for the following analytes used as tumour markers in the external quality assessment (EQA)-programmes of Instand e.V., a national EQA-organiser in Germany: Corticotropin (ACTH), growth hormone (GH, hGH), prolactin (PRL), chorionic gonadotropin (CG, hCG), calcitonin (CT, hCT), thyroglobulin (Tg), carcinoembryonic antigen (CEA), CA-Antigens 125, 72-4, 15-3 and 19-9, alpha foetoprotein (AFP) and prostate-specific antigen (PSA). The results from the participants show a large variation in the precision of the methods used as well as in the comparability of results between methods for the same analyte. In general, the hormones used as tumour markers show better performance than the "CA-markers", which are often inadequately standardised and defined. In the case of one CA-marker (CA 72-4/TAG 72-4), the differences between the lowest kit median concentration and highest kit median concentration for one sample pair were 440% and 580%. The corresponding figures for ACTH were 123% and 156% and for CEA 180% and 184%. The classical tumour markers such as carcinoembryonic antigen (CEA) and alpha foetoprotein (AFP) performed markedly better than the CA-markers and PSA with regards to both inter- and intra-method comparability. The inter-laboratory precision for a given kit and marker was acceptable in many cases. The results show that only results from the same kit/method for each tumour marker can be used for cumulative or time-dependent comparison of results - for example pre-operative and post-operative follow up. In the case of prostate specific antigen (PSA), the kits used for free and total PSA must come from the same producer, if the generally accepted ratios are to have any diagnostic value. The need for kit- and laboratory-specific reference ranges and cut-off values for setting diagnostic specificity and sensitivity is highlighted from the EQA-results. The situation for inter-method comparability for the CA-Markers has not improved over the past decade. With the exception of calcitonin for detecting medullary thyroid carcinoma, chorionic gonadotropin in germ-cell tumours in men and thyroglobulin after total thyroidectomy, none of the remaining analytes appear to be suitable for screening purposes.

Comparability of method results and performance in a national external quality assessment scheme between 1993 and 2003 using thyroid associated analytes as examples.

Six thyroid analytes (free and total triiodothyronine and thyroxine, thyrotropin and thyroglobulin) have been followed up over a 10 year period in a national external quality assessment scheme (EQAS) organised by the Institute for Standardisation and Documentation in the Medical Laboratory (INSTAND). I. The following points were observed: II. The introduction of samples with properties similar to patient serum (filtered, recalcified defibrinated plasma without stripping) improved performance and inter-method comparability for the free thyroid hormones. III. In general, the performance in EQAS has improved over the past decade, an exception being thyroglobulin, where precision has improved at the expense of inter-method comparability. IV. Regular statistical analysis of EQAS data allows adjustment of target ranges to be made when necessary. V. Analytes which are not dependent on binding proteins--thyrotropin and the total thyroid hormones--give rise to similar performance when stripped and spiked plasma or recalcified non-stripped and spiked plasma is used as sample. VI. Whereas certain analytes have had a relatively constant number of participants over the past decade (total thyroid hormones), others have shown a drastic increase (free thyroxine from 67 to 620; thyrotropin from 295 to 724) reflecting the medical demand for the analytes.

Comparability of method results and performance in a national external quality assessment scheme between 1993 and 2003 using thyroid associated antibodies as examples.

Four thyroid antibodies (antibodies to microsomes [MAb], thyroid peroxidase [anti-TPO], thyroglobulin [anti-Tg] and TSH-receptor [TRAB, THYBIA]) have been followed up over a 10-year period in a national external quality assessment scheme (EQAS) organised by the Institute for Standardisation and Documentation in the Medical Laboratory (INSTAND e.V.). The following points were observed: I. The introduction of samples with properties similar to patient serum (filtered, recalcified defibrinated plasma without stripping) improved performance and inter-method comparability for thyroid antibodies. II. Regular statistical analysis of EQAS data allows adjustment of target ranges to be made when necessary. III. There are large inter-method variations in reporting, both on qualitative and quantitative results. IV. The samples often gave rise to different constellations of antibodies, which were kit-dependent. V. Despite use of international reference preparations, there was no numerical comparability between quantitative methods for the same analyte. In general, the performance in EQAS for thyroid antibodies has improved over the past decade. There is still a real need for standardisation in the field of thyroid antibody analysis.

Experience with an external quality assessment programme for point-of-care-testing (POCT) devices for the determination of blood glucose.

This article describes the preparation and internal and external evaluation of materials, critical issues in the external quality assessment (EQA) of point-of-care testing (POCT) devices for measuring blood glucose. A comparison was made between different materials, both of natural and synthetic origin and with and without stabilisers. The aims were to produce a material which was compatible with as many POCT-devices as possible and so reduce the number of materials sent out in each campaign as well as to optimise the precision and comparability of results between methods and devices. Although the use of near natural material--sterile-filtered plasma spiked with glucose--survived internal testing, this material proved to be unsuitable for EQA surveys. The study resulted in the reduction of materials for each survey to stabilised whole blood for one device, stabilised plasma for two devices and a synthetic material based on a polyethylene glycol matrix for all other devices. Samples were sent as pairs six times annually. The POCT-devices tested measured precisely but inaccurately in the synthetic material, when compared with the reference method (gas-chromatography coupled with isotope-dilution mass-spectrometry; GC-IDMS), so that the devices could only be evaluated for precision. The construction of ratios between the concentrations measured on the two samples distributed allowed an indirect assessment of accuracy. The need for surveillance of POCT devices is stressed in this publication, which combines theory and practice in setting up and running an EQA programme for blood glucose.