Concordancia consensuada en metodología observacional: efectos del tamaño del grupo en el tiempo y la calidad del registro / Concordância consensual na metodologia observacional: efeitos do tamanhodo grupo, tempo e qualidade do registro / The consensus agreement method in observational methodology: impact of sample size on time requirements and dataset quality

En metodología observacional, para abordar la fiabilidad de los datos ya registrados, suele recurrirse a coeficientes de concordancia, coeficientes de correlación o a la teoría de la generalizabilidad; además, cada vez está tomando mayor protagonismo la concordancia consensuada. Esta forma de concordancia trata de lograr la coincidencia entre los observadores antes del registro. A pesar de su creciente presencia en estudios observacionales, son pocos los trabajos quehan profundizado en el desarrollo y optimización de esta forma cualitativa de concordancia. El presente trabajo, además de constituirse en un ejemplo de la utilización de la concordancia por consenso, ha comparado el resultado obtenido (tiempo empleado y ajuste con el registro ideal) por grupos de consenso formados por diferente número de integrantes (dos, tres y cuatro observadores). No se han encontrado diferencias significativas al comparar los grupos de concordancia por consenso de dos, tres y cuatro integrantes, ni en relación al tiempo empleado en el registro, ni en lo relativo al porcentaje de acuerdo con el registro ideal. La determinación del tamaño muestral necesario para obtener diferencias significativas entre los grupos ha permitido elevar conclusiones en términos de eficiencia. (AU)

The reliability of datasets in observational methodology is typically tested using coefficients of agreement, correlation coefficients, or generalizability theory. Another increasingly popular method used to demonstrate the quality of data is the consensus agreement method, in which two or more observers agree on their coding decisions while creating the dataset. Although the consensus agreement method is being increasingly used in observational studies, few studies have conducted an in-depth analysis of how this qualitative procedure is approached or of how it can be optimized. In this study, in addition to presenting a practical example of the application of the consensus agreement method, we compare the results from three groups (of two, three, and four observers) to analyze performance in terms of time required to code the data and goodness of fit with respect to an optimal dataset. No significant differences were found between the three groups for either of the variables analyzed. Prior calculation of the sample size required to detect significant differences between the groups adds strength to our conclusions regarding the efficiency of the consensus agreement method. (AU)

Na metodologia observacional, para lidar com a confiabilidade de dois dados já registrados, costumamos passarpelos coeficientes de concordância, coeficientes de correlação ou pela teoria da generalização. Além disso, a concordância consensual vem ganhando cada vez mais destaque. Esta forma de concordância tenta chegar a um acordo entre os observadores antes do registro. Apesar de sua crescente presença em estudos observacionais, poucos estudos se aprofundam no desenvolvimento e otimização de uma forma qualitativa de concordância. Ou apresentar trabalho, além de ser um exemplo do uso de concordância de consenso, comparação ou resultado obtido (tempo despendido e ajuste como registro ideal) por grupos de consenso formados por diferentes números de membros (dois, três e quatro observadores). Não foram encontradas diferenças significativas na comparação de dois gruposde concordância por consenso de dois, três e quatro membros, não em relação ao tempo gasto não registrado, mas em relação ao percentual de concordância conforme lista ideal. A determinação do tamanho dá a amostra necessária para obter diferenças significativas entre os grupos permitindo-nos tirar conclusões em termos de eficiência. (AU)

Interrater agreement in visual scoring of neonatal seizures based on majority voting on a web-based system: The Neoguard EEG database.

OBJECTIVE: To assess interrater agreement based on majority voting in visual scoring of neonatal seizures. METHODS: An online platform was designed based on a multicentre seizure EEG-database. Consensus decision based on 'majority voting' and interrater agreement was estimated using Fleiss' Kappa. The influences of different factors on agreement were determined. RESULTS: 1919 Events extracted from 280h EEG of 71 neonates were reviewed by 4 raters. Majority voting was applied to assign a seizure/non-seizure classification. 44% of events were classified with high, 36% with moderate, and 20% with poor agreement, resulting in a Kappa value of 0.39. 68% of events were labelled as seizures, and in 46%, all raters were convinced about electrographic seizures. The most common seizure duration was <30s. Raters agreed best for seizures lasting 60-120s. There was a significant difference in electrographic characteristics of seizures versus dubious events, with seizures having longer duration, higher power and amplitude. CONCLUSIONS: There is a wide variability in identifying rhythmic ictal and non-ictal EEG events, and only the most robust ictal patterns are consistently agreed upon. Database composition and electrographic characteristics are important factors that influence interrater agreement. SIGNIFICANCE: The use of well-described databases and input of different experts will improve neonatal EEG interpretation and help to develop uniform seizure definitions, useful for evidence-based studies of seizure recognition and management.

Interobserver agreement on dermoscopic features of small basal cell carcinoma (<5 mm) among low-experience dermoscopists.

The basal cell carcinoma (BCC) assessment in dermoscopy is based on the recognition of specific characteristics synthesized and described as classical and non-classical criteria, but which may not necessarily present all at the same time. Consequently, a deep knowledge in detecting the aforementioned dermoscopic criteria is crucial in diagnosis. The aim of the study was to evaluate which criteria are more frequently recognized among a group of low-experienced dermoscopists when confronted with the difficult diagnosis of BCC with a diameter lower than 5 mm. We examined 100 BCC finding that data displays a full agreement only for one classical criterion, the lack of pigmented network (Fleiss' κ = 1), while among other classical criteria only arborizing vessels and ulceration exhibit a good agreement among observers (Fleiss' κ > 0.40). Analyzing non-classical criteria, only blue-whitish veil and blue in-focus dots show a good agreement among low-experience observers (Fleiss' κ > 0.40). It is evident that in small size BCC classic dermoscopic criteria are often substituted by non-classical criteria, which represent the neoplasm's early phase. Thus, it is of importance, especially for low-experience dermoscopists, to analyze even the non-classical criteria in order to obtain a diagnosis of early BCC.

What is synergy? The Saariselkä agreement revisited.

Many biological or chemical agents when combined interact with each other and produce a synergistic response that cannot be predicted based on the single agent responses alone. However, depending on the postulated null hypothesis of non-interaction, one may end up in different interpretations of synergy. Two popular reference models for null hypothesis include the Bliss independence model and the Loewe additivity model, each of which is formulated from different perspectives. During the last century, there has been an intensive debate on the suitability of these synergy models, both of which are theoretically justified and also in practice supported by different schools of scientists. More than 20 years ago, there was a community effort to make a consensus on the terminology one should use when claiming synergy. The agreement was formulated at a conference held in Saariselkä, Finland in 1992, stating that one should use the terms Bliss synergy or Loewe synergy to avoid ambiguity in the underlying models. We review the theoretical relationships between these models and argue that one should combine the advantages of both models to provide a more consistent definition of synergy and antagonism.