A discussion of alternatives for establishing empirical benchmarks for interpreting single-case effect sizes / Discusión de diferentes maneras de establecer criterios interpretativos empíricos para tamaños del efecto de diseños de caso único

ABSTRACT

In this paper we reflect on the numerous calls for the development of benchmarks for interpreting effect size indices, reviewing several possibilities. Such benchmarks are aimed to provide criteria so that analysts can judge whether the size of the effect o bserved is rather "small", "medium" or "large". The context of this discussion is single-case experimental designs, for which a great variety of procedures have been proposed, with their different nature (e.g., being based on amount of overlap vs. a standardized mean difference) posing challenges to interpretation. For each of the alternatives discussed we point at their strengths and limitations. We also comment how such empirical benchmarks can be obtained, usually by methodologists, and illustrate how these benchmarks can be used by applied researchers willing to have more evidence on the magnitude of effect observed and not only whether an effect is present or not. One of the alternatives discussed is a proposal we make in the current paper. Although it has certain limitations, as all alternatives do, we consider that it is worth discussing it and the whole set of alternatives in order to advance in interpreting effect sizes, now that computing and reporting their numerical values is (or is expected to be) common practicIn this paper we reflect on the numerous calls for the development of benchmarks for interpreting effect size indices, reviewing several possibilities. Such benchmarks are aimed to provide criteria so that analysts can judge whether the size of the effect o bserved is rather «small», «medium» or «large». The context of this discussion is single-case experimental designs, for which a great variety of procedures have been proposed, with their different nature (e.g., being based on amount of overlap vs. a standardized mean difference) posing challenges to interpretation. For each of the alternatives discussed we point at their strengths and limitations. We also comment how such empirical benchmarks can be obtained, usually by methodologists, and illustrate how these benchmarks can be used by applied researchers willing to have more evidence on the magnitude of effect observed and not only whether an effect is present or not. One of the alternatives discussed is a proposal we make in the current paper. Although it has certain limitations, as all alternatives do, we consider that it is worth discussing it and the whole set of alternatives in order to advance in interpreting effect sizes, now that computing and reporting their numerical values is (or is expected to be) common practice

RESUMEN

El presente trabajo responde a la necesidad expresada de desarrollar criterios interpretativos para los índices de tamaño del efecto, repasando diferentes maneras para conseguirlo. El objetivo de los criterios es proporcionar herramientas a los analistas para que éstos puedan valorar si el efecto observado en su estudio es más bien «pequeño», «mediano» o «grande». El contexto en el cual tiene lugar la discusión son los diseños de caso único, para los cuales se ha propuesto una gran variedad de técnicas analíticas cuya base diferente (e.g., grado de solapamiento versus diferencia de medias estandarizada) supone un reto para la interpretación. Para cada una de las alternativas que se comentan, se destacan las ventajas e inconvenientes. Adicionalmente, se comenta cómo estos criterios pueden ser obtenidos, una tarea propia de los metodólogos, y cómo pueden ser utilizados por investigadores aplicados que desean disponer de más evidencias sobre la magnitud del efecto observado, más allá de decidir si el efecto existe o no. Una de las alternativas es una propuesta que se realiza en el marco del presente artículo. A pesar de que también presenta desventajas, como todas las alternativas, consideramos que es necesario discutir esta alternativa y todas las demás con la finalidad de avanzar en la interpretación de tamaños del efecto, en un momento en el cual el hecho de calcular y reportar sus valores numéricos es (o se supone) habitual