Conditioning on the Pre-Test versus Gain Score Modelling: Revisiting the Controversy in a Multilevel Setting.

We consider estimating the effect of a treatment on a given outcome measured on subjects tested both before and after treatment assignment in observational studies. A vast literature compares the competing approaches of modelling the post-test score conditionally on the pre-test score versus modelling the difference, namely, the gain score. Our contribution lies in analyzing the merits and drawbacks of two approaches in a multilevel setting. This is relevant in many fields, such as education, where students are nested within schools. The multilevel structure raises peculiar issues related to contextual effects and the distinction between individual-level and cluster-level treatments. We compare the two approaches through a simulation study. For individual-level treatments, our findings align with existing literature. However, for cluster-level treatments, the scenario is more complex, as the cluster mean of the pre-test score plays a key role. Its reliability crucially depends on the cluster size, leading to potentially unsatisfactory estimators with small clusters.

Statistical methods to estimate the impact of remote teaching on university students' performance.

The COVID-19 pandemic manifested around the World since February 2020, leading to disruptive effects on many aspects of people social life. The suspension of face-to-face teaching activities in schools and universities was the first containment measure adopted by the Governments to deal with the spread of the virus. Remote teaching has been the emergency solution implemented by schools and universities to limit the damages of schools and universities closure to students' learning. In this contribution we intend to suggest to policy makers and researchers how to assess the impact of emergency policies on remote learning in academia by analysing students' careers. In particular, we exploit the quasi-experimental setting arising from the sudden implementation of remote teaching in the second semester of academic year 2019/2020: we compare the performance of the cohort 2019/2020, which represents the treatment group, with the performance of the cohort 2018/2019, which represents the control group. We distinguish the impact of remote teaching at two levels: degree program and single courses within a degree program. We suggest to use Difference-In-Differences approach in the former case and multilevel modeling in the latter one. The proposal is illustrated analysing administrative data referred to freshmen of cohorts 2018/2019 and 2019/2020 for a sample of degree programs of the University of Florence (Italy).

Quantile regression for count data: jittering versus regression coefficients modelling in the analysis of credits earned by university students after remote teaching.

The extension of quantile regression to count data raises several issues. We compare the traditional approach, based on transforming the count variable using jittering, with a recently proposed approach in which the coefficients of quantile regression are modelled by parametric functions. We exploit both methods to analyse university students' data to evaluate the effect of emergency remote teaching due to COVID-19 on the number of credits earned by the students. The coefficients modelling approach performs a smoothing that is especially convenient in the tails of the distribution, preventing abrupt changes in the point estimates and increasing precision. Nonetheless, model selection is challenging because of the wide range of options and the limited availability of diagnostic tools. Thus the jittering approach remains fundamental to guide the choice of the parametric functions.

Evaluation of Student Performance through a Multidimensional Finite Mixture IRT Model.

In the Italian academic system, a student can enroll for an exam immediately after the end of the teaching period or can postpone it; in this second case the exam result is missing. We propose an approach for the evaluation of a student performance throughout the course of study, accounting also for nonattempted exams. The approach is based on an item response theory model that includes two discrete latent variables representing student performance and priority in selecting the exams to take. We explicitly account for nonignorable missing observations as the indicators of attempted exams also contribute to measure the performance (within-item multidimensionality). The model also allows for individual covariates in its structural part.

Group versus single handed primary care: a performance evaluation of the care delivered to chronic patients by Italian GPs.

OBJECTIVES: In family medicine contrasting evidence exists on the effectiveness of team practice compared with solo practice on chronic disease management. In Italy, several experiences of team practice have been introduced since the late 1990s but few studies detail their impact on the quality of care. The aim of this paper is to evaluate the impact of team practice in family medicine in six Italian regions using chronic disease management process indicators as a measure of outcome. METHODS: Cross-sectional studies were performed to assess impact on quality of care for diabetes, congestive heart failure and ischaemic heart disease. The impact of team vs. solo practice was approximated through performance comparison of general practitioners (GPs) adhering to a team with respect to GPs working in a solo practice. Among the 2082 practitioners working in the 6 regions those assisting 300+ patients were selected. Quality of care towards 164,267 patients having at least one of three chronic conditions was estimated for the year 2008 using administrative databases. Quality indicators (% of patients receiving appropriate care) were selected (4 for diabetes, 4 for congestive heart failure, 3 for ischaemic heart disease) and a total score was computed for each patient. For each disease the response variable associated to each physician was the average score of the patients on his/her list. A multilevel model was estimated assessing the impact of team vs. solo practice. RESULTS: No impact was found for diabetes and heart failure. For ischaemic heart disease a slightly significant impact was observed (0.040; 95% CI: 0.015, 0.065). CONCLUSIONS: No significant difference was found between team practice and solo practice on chronic disease management in six Italian regions.