Unit Response and Costs in Web Versus Face-To-Face Data Collection: Comparison of Two Cross-sectional Health Surveys.

BACKGROUND: Potential is seen in web data collection for population health surveys due to its combined cost-effectiveness, implementation ease, and increased internet penetration. Nonetheless, web modes may lead to lower and more selective unit response than traditional modes, and this may increase bias in the measured indicators. OBJECTIVE: This research assesses the unit response and costs of a web study versus face-to-face (F2F) study. METHODS: Alongside the Belgian Health Interview Survey by F2F edition 2018 (BHISF2F; net sample used: 3316), a web survey (Belgian Health Interview Survey by Web [BHISWEB]; net sample used: 1010) was organized. Sociodemographic data on invited individuals was obtained from the national register and census linkages. Unit response rates considering the different sampling probabilities of both surveys were calculated. Logistic regression analyses examined the association between mode system and sociodemographic characteristics for unit nonresponse. The costs per completed web questionnaire were compared with the costs for a completed F2F questionnaire. RESULTS: The unit response rate is lower in BHISWEB (18.0%) versus BHISF2F (43.1%). A lower response rate was observed for the web survey among all sociodemographic groups, but the difference was higher among people aged 65 years and older (15.4% vs 45.1%), lower educated people (10.9% vs 38.0%), people with a non-Belgian European nationality (11.4% vs 40.7%), people with a non-European nationality (7.2% vs 38.0%), people living alone (12.6% vs 40.5%), and people living in the Brussels-Capital (12.2% vs 41.8%) region. The sociodemographic characteristics associated with nonresponse are not the same in the 2 studies. Having another European (OR 1.60, 95% CI 1.20-2.13) or non-European nationality (OR 2.57, 95% CI 1.79-3.70) compared to a Belgian nationality and living in the Brussels-Capital (OR 1.72, 95% CI 1.41-2.10) or Walloon (OR 1.47, 95% CI 1.15-1.87) regions compared to the Flemish region are associated with a higher nonresponse only in the BHISWEB study. In BHISF2F, younger people (OR 1.31, 95% CI 1.11-1.54) are more likely to be nonrespondents than older people, and this was not the case in BHISWEB. In both studies, lower educated people have a higher probability of being nonrespondent, but this effect is more pronounced in BHISWEB (low vs high education level: Web, OR 2.71, 95% CI 2.21-3.39 and F2F OR 1.70, 95% CI 1.48-1.95). The BHISWEB study had a considerable advantage; the cost per completed questionnaire was almost 3 times lower (€41 [US $48]) compared with F2F data collection (€111 [US $131]). CONCLUSIONS: The F2F unit response rate was generally higher, yet for certain groups the difference between web and F2F was more limited. Web data collection has a considerable cost advantage. It is therefore worth experimenting with adaptive mixed-mode designs to optimize financial resources without increasing selection bias (eg, only inviting sociodemographic groups who are keener to participate online for web surveys while continuing to focus on increasing F2F response rates for other groups).

Is the European Health Interview Survey online yet? Response and net sample composition of a web-based data collection.

BACKGROUND: The European Health Interview Survey (EHIS) provides cross-national data on health status, health care and health determinants. So far, 10 of the 30 member states (MS) opted for web-based questionnaires within mixed-mode designs but none used it as the sole mode. In the context of future EHIS, the response rate and net sample composition of a web-only approach was tested. METHODS: A Belgian study with a target sample size of 1000 (age: 16-85) was organized using the EHIS wave 3 model questionnaire. The sample was selected according to a multistage, clustered sampling procedure with geographical stratification. Field substitution was applied; non-participating households were replaced by similar households regarding statistical sector, sex and age. There was one reminder letter and a €10 conditional incentive. RESULTS: Considering all substitutions, a 16% response rate was obtained after sending one reminder. Elderly, Brussels Capital inhabitants, people living without a partner and those with a non-Belgian nationality were less responding. By design, there were no differences between the initial and final net sample regarding substitution characteristics. Nevertheless, people living without a partner, non-Belgians and lower educated people remain underrepresented. CONCLUSION: There was a low response rate, particularly for some population groups. The response rate was lower than those of MS using mixed-mode designs including web, especially these comprising interviewer-based approaches. Despite the long and complex questionnaire, there was a low break off rate. So far, web-only data collection is not an acceptable strategy for population-based health surveys but efforts to increase the response should be further explored.

Mixing mixed-mode designs in a national health interview survey: a pilot study to assess the impact on the self-administered questionnaire non-response.

BACKGROUND: Many population health surveys consist of a mixed-mode design that includes a face-to-face (F2F) interview followed by a paper-and-pencil (P&P) self-administered questionnaire (SAQ) for the sensitive topics. In order to alleviate the burden of a supplementary P&P questioning after the interview, a mixed-mode SAQ design including a web and P&P option was tested for the Belgian health interview survey. METHODS: A pilot study (n = 266, age 15+) was organized using a mixed-mode SAQ design following the F2F interview. Respondents were invited to complete a web SAQ either immediately after the interview or at a later time. The P&P option was offered in case respondents refused or had previously declared having no computer access, no internet connection or no recent usage of computers. The unit response rate for the web SAQ and the overall unit response rate for the SAQ independent of the mode were evaluated. A logistic regression analysis was conducted to explore the association of socio-demographic characteristics and interviewer effects with the completed SAQ mode. Furthermore, a logistic regression analysis assessed the differential user-friendliness of the SAQ modes. Finally, a logistic multilevel model was used to evaluate the item non-response in the two SAQ modes while controlling for respondents' characteristics. RESULTS: Of the eligible F2F respondents in this study, 76% (107/140) agreed to complete the web SAQ. Yet among those, only 78.5% (84/107) actually did. At the end, the overall (web and P&P) SAQ unit response rate reached 73.5%. In this study older people were less likely to complete the web SAQ. Indications for an interviewer effect were observed as regard the number of web respondents, P&P respondents and respondents who refused to complete the SAQ. The web SAQ scored better in terms of user-friendliness and presented higher item response than the P&P SAQ. CONCLUSIONS: The web SAQ performed better regarding user-friendliness and item response than the P&P SAQ but the overall SAQ unit response rate was low. Therefore, future research is recommended to further assess which type of SAQ design implemented after a F2F interview is the most beneficial to obtain high unit and item response rates.

Data collection in pandemic times: the case of the Belgian COVID-19 health surveys.

BACKGROUND: Survey data were needed to assess the mental and social health, health related behaviors and compliance with preventive measures of the population during the COVID-19 pandemic. Yet, the pandemic challenged classical survey methods. Time and budgetary constraints at the beginning of the pandemic led to ad hoc recruitment of participants and easily manageable data collection modes. This paper describes the methodological choices and results in terms of participation for the COVID-19 health surveys conducted in Belgium. METHODS: The COVID-19 health surveys refer to a series of ten non-probability web surveys organized between April 2020 and March 2022. The applied recruitment strategies were diverse including, amongst others, a launch through the website and the social media of the organizing research institute. In addition, the survey links were shared in articles published in the national press and participants were requested to share the surveys in their network. Furthermore, participants were asked consent to be re-contacted for next survey editions using e-mail invitations. RESULTS: These mixed approaches allowed to reach a substantial number of participants per edition ranging from 49339 in survey 1 to 13882 in survey 10. In addition, a longitudinal component was created; a large share of the same individuals were followed up over time; 12599 participants completed at least 5 surveys. There were, however, sex, age, educational level and regional differences in participation. Post-stratification weighting on socio-demographic factors was applied to at least partly take this into account. CONCLUSION: The COVID-19 health surveys allowed rapid data collection after the onset of the pandemic. Data from these non-probability web surveys had their limitations in terms of representativeness due to self-selection but were an important information source as there were few alternatives. Moreover, by following-up the same individuals over time it was possible to study the effect of the different crisis phases on, amongst others, the mental health. It is important to draw lessons from these experiences: initiatives in order to create a survey infrastructure better equipped for future crises are needed.

Organizing the health interview survey at the local level: design of a pilot study.

BACKGROUND: The local Health Interview Study (LHIS) was developed to gain health information at the level of the municipality in Flanders, the northern part of Belgium. It enables municipalities to make evidence-based decisions in their public health policy. To test the feasibility of implementing the LHIS, a pilot study was conducted in Melle, a small Flemish municipality with 11.736 inhabitants. METHODS: The target sample size was 1000 (≥ 15 years). A systematic sampling technique was applied with substitutes for non-respondents who were matched in terms of statistical sector, age and sex. Selected persons were contacted by post to complete the questionnaire and in case of non-response, a reminder was sent. Questionnaires were collected using a concurrent mixed-mode design: a paper and pencil, and web option. All questions were selected from the Belgian Health Interview Survey relating to health status and determinants of health. RESULTS: One thousand twenty-two questionnaires were obtained after inviting 3137 individuals (response rate = 32.6%). Older adults were more likely to participate than younger adults, and women more than men. The final sample resembled the initial sample in terms of sex and statistical sector, but not in terms of age. Younger adults were underrepresented whereas older adults were overrepresented. Lastly, older adults were more likely to fill in the questionnaire on paper than younger adults, and women more than men. CONCLUSION: The LHIS can be successfully implemented in Flemish municipalities. The method, however, does not guarantee that the composition of the final sample reflects the initial sample. Therefore, weights should be added in the analyses to correct for potential deviations in sample composition. Furthermore, implementing a sequential mixed-mode design with a web option preceding a paper and pencil option in future studies could reduce costs and improve data quality.

Domestic violence during the COVID-19 confinement: do victims feel more socially isolated?

BACKGROUND: Since March 13th 2020, confinement measures have been introduced in Belgium to curb the spread of the coronavirus (COVID-19). These measures also have an impact on people's daily life (closure of school/businesses, teleworking, recommendation to stay at home). This can cause stress on social, economic and psychological levels and thereby can trigger domestic violence. Besides, confinement also fosters social isolation, which can complicate help seeking behaviour. The aim of this study is to determine the prevalence of domestic violence during the coronavirus crisis and to assess whether there is an association between domestic violence and social isolation. METHODS: Several online COVID-19 Health Surveys were organised among Belgian residents aged 18+ via snowball sampling. This study is based on the second (April 2020) and the sixth survey (March 2021). After excluding 1-person households and missing data, the sample size was respectively 25,251 and 12,589. Weighted prevalence of domestic violence was evaluated for the two surveys. The association (OR; 95% CI; p-value) between domestic violence and subjective social isolation was assessed with logistic regression stratified by survey and adjusted for covariates. RESULTS: In April 2020, 4.0% of the adult population reported being a victim of domestic violence (1.2% in the Health Interview Survey 2018); in March 2021, this was 6.2%. In April 2020, victims of domestic violence had higher odds of being unsatisfied with their social contacts (OR = 1.25; 95% CI: 1.08-1.44; p < .05), weak social support (OR = 2.26; 95% CI: 1.97-2.58; p < .0001) and having less confidence in health care services (OR = 1.38; 95% CI: 1.13-1.71; p < .05). In March 2021, victims had higher odds of being unsatisfied with their social contacts (OR = 1.30; 95% CI: 1.08-1.56; p < .05) and weak social support (OR = 2.41; 95% CI: 2.04-2.84; p < .0001), and social (OR = 2.64; 95% CI: 2.23-3.13; p < .0001) and emotional loneliness (OR = 2.22; 95% CI: 1.80-2.73; p < .0001). CONCLUSIONS: More people have reported domestic violence since the start of the coronavirus crisis than did in 2018. An association between domestic violence and social isolation was determined. Although confinement is needed to counteract the virus, it can put people in a dangerous situation since they do not get the help they need. Therefore, adequate support is essential.

Anxiety and Depression in Belgium during the First 15 Months of the COVID-19 Pandemic: A Longitudinal Study.

The COVID-19 pandemic and policy measures enacted to contain the spread of the coronavirus have had nationwide psychological effects. This study aimed to assess the impact of the first 15 months of the COVID-19 pandemic on the level of anxiety (GAD-7 scale) and depression (PHQ-9 scale) of the Belgian adult population. A longitudinal study was conducted from April 2020 to June 2021, with 1838 respondents participating in 6 online surveys. Linear mixed models were used to model the associations between the predictor variables and the mental health outcomes. Results showed that the prevalence of symptoms of anxiety and depression was higher in times of stricter policy measures. Furthermore, after the initial stress from the outbreak, coping and adjustment were observed in participants, as symptoms of anxiety and depression decreased during times of lower policy restrictions to almost the same level as in pre-COVID times (2018). Though time trends were similar for all population subgroups, higher levels of both anxiety and depression were generally found among women, young people, people with poor social support, extraverts, people having pre-existing psychological problems, and people who were infected/exposed to the COVID-19 virus. Therefore, investment in mental health treatment programs and supports, especially for those risk groups, is crucial.

Short-term impact of the COVID-19 confinement measures on health behaviours and weight gain among adults in Belgium.

BACKGROUND: In Belgium, confinement measures were introduced on the 13th of March 2020 to curb the spread of the coronavirus disease (COVID-19). These measures may affect health behaviours of the population such as eating habits, physical activity and alcohol consumption, which in turn can lead to weight gain resulting in overweight and obesity, increasing the risk of several chronic diseases, but also of severe COVID-19. The purpose of this study is to assess the impact of confinement measures on health behaviours and their associations with weight gain. METHODS: Data were derived from the second national COVID-19 health survey. Data were collected between the 16th and the 23rd of April 2020. The recruitment of participants was based on snowball sampling via Sciensano's website, invitations via e-mail and social media. The study sample includes participants aged 18 years and over with no missing data on the variables of interest (n = 28,029). The association between self-reported weight gain and health behaviour changes, adjusted for gender, age group and household composition was assessed through OR's (95% CI) calculated with logistic regression models, using post-stratification weights. RESULTS: Overall, 28.6% reported weight gain after 6 weeks of confinement. Higher odds of weight gain were observed among participants who increased or decreased their consumption of sugar-sweetened beverages (OR = 1.39 (1.15-1.68) and 1.29 (1.04-1.60), respectively), among those who increased their consumption of sweet or salty snacks (OR = 3.65 (3.27-4.07)), among those who became less physically active (OR = 1.91 (1.71-2.13)), and among those who increased their alcohol consumption (OR = 1.86 (1.66-2.08)). CONCLUSIONS: The most important correlates of weight gain during confinement were an increased consumption of sweet or salty snacks and being less physically active. These findings confirm the impact of diet and exercise on short term weight gain and plead to take more action, in supporting people to achieve healthier behaviours in order to tackle overweight and obesity, especially during the COVID-19 pandemic.

Comparing web-based versus face-to-face and paper-and-pencil questionnaire data collected through two Belgian health surveys.

OBJECTIVES: Using the European Health Interview Survey (EHIS) questionnaire, a web-based survey was organized alongside a face-to-face (F2F) survey including a paper-and-pencil (P&P) questionnaire for sensitive topics. Associated with these different modes, other design features varied too (e.g., recruitment, incentives, sampling). We assessed whether these whole data collection systems developed around the modes produced equivalent health estimates. METHODS: Data were obtained from two population-based surveys: the EHISWEB (web-administered, n = 1010) and the Belgian Health Interview Survey 2018 (BHIS2018) (interviewer-administered, n = 2748). Logistic regression analyses were used to assess mode system differences while adjusting for socio-demographic differences in the net samples. RESULTS: For the P&P mode of the BHIS, significant mode system differences were detected for 2 of the 9 health indicators. Among the indicators collected via the F2F mode, 9 of the 18 indicators showed significant differences. CONCLUSIONS: Indicators collected via the web-based and P&P self-administered modes were generally more comparable than indicators collected via the web-based and F2F mode. Furthermore, fewer differences were detected for indicators based on simple and factual questions compared to indicators based on subjective or complex questions.

The Belgian health examination survey: objectives, design and methods.

BACKGROUND: In 2018 the first Belgian Health Examination Survey (BELHES) took place. The target group included all Belgian residents aged 18 years and older. The BELHES was organized as a second stage of the sixth Belgian Health Interview Survey (BHIS). This paper describes the study design, recruitment method and the methodological choices that were made in the BELHES. METHODS: After a pilot period during the first quarter of the BHIS fieldwork, eligible BHIS participants were invited to participate in the BELHES until a predefined number (n = 1100) was reached. To obtain the required sample size, 4918 eligible BHIS participants had to be contacted. Data were collected at the participant's home by trained nurses. The data collection included: 1) a short set of questions through a face-to-face interview, 2) a clinical examination consisting of the measurement of height, weight, waist circumference, blood pressure and for people aged 50 years and older handgrip strength and 3) a collection of blood and urine samples. The BELHES followed as much as possible the guidelines provided in the framework of the European Health Examination Survey (EHES) initiative. Finally 1184 individuals participated in the BELHES, resulting in a participation rate of 24.1%. Results for all the core BELHES measurements were obtained for more than 90% of the participants. CONCLUSION: It is feasible to organize a health examination survey as a second stage of the BHIS. The first successfully organized BELHES provides useful information to support Belgian health decision-makers and health professionals. As the BELHES followed EHES recommendations to a large extent, the results can be compared with those from similar surveys in other EU (European Union) member states.

Determinants of unit nonresponse in multi-mode data collection: A multilevel analysis.

BACKGROUND: Multi-mode data collection is widely used in surveys. Since several modes of data collection are successively applied in such design (e.g. self-administered questionnaire after face-to-face interview), partial nonresponse occurs if participants fail to complete all stages of the data collection. Although such nonresponse might seriously impact estimates, it remains currently unexplored. This study investigates the determinants of nonresponse to a self-administered questionnaire after having participated in a face-to-face interview. METHODS: Data from the Belgian Health Interview Survey 2013 were used to identify determinants of nonresponse to self-administered questionnaire (n = 1,464) among those who had completed the face-to-face interview (n = 8,133). The association between partial nonresponse and potential determinants was explored through multilevel logistic regression models, encompassing a random interviewer effect. RESULTS: Significant interviewer effects were found. Almost half (46.6%) of the variability in nonresponse was attributable to the interviewers, even in the analyses controlling for the area as potential confounder. Partial nonresponse was higher among youngsters, non-Belgian participants, people with a lower educational levels and those belonging to a lower income household, residents of Brussels and Wallonia, and people with poor perceived health. Higher odds of nonresponse were found for interviews done in the last quarters of the survey-year. Regarding interviewer characteristics, only the total number of interviews carried out throughout the survey was significantly associated with nonresponse to the self-administered questionnaire. CONCLUSIONS: The results indicate that interviewers play a crucial role in nonresponse to the self-administered questionnaire. Participant characteristics, interview circumstances and interviewer characteristics only partly explain the interviewer variability. Future research should examine further interviewer characteristics that impact nonresponse. The current study emphasises the importance of training and motivating interviewers to reduce nonresponse in multi-mode data collection.

Measurement agreement of the self-administered questionnaire of the Belgian Health Interview Survey: Paper-and-pencil versus web-based mode.

Before organizing mixed-mode data collection for the self-administered questionnaire of the Belgian Health Interview Survey, measurement effects between the paper-and-pencil and the web-based questionnaire were evaluated. A two-period cross-over study was organized with a sample of 149 employees of two Belgian research institutes (age range 22-62 years, 72% female). Measurement agreement was assessed for a diverse range of health indicators related to general health, mental and psychosocial health, health behaviors and prevention with kappa coefficients and intraclass correlation (ICC). The quality of the data collected by both modes was evaluated by quantifying the missing, 'don't know' and inconsistent values and data entry mistakes. Good to very good agreement was found for all categorical indicators with kappa coefficients superior to 0.60, except for two mental and psychosocial health indicators namely the presence of a sleeping disorder and of a depressive disorder (kappa≥0.50). For the continuous indicators high to acceptable agreement was observed with ICC superior to 0.70. Inconsistent answers and data-entry mistakes were only occurring in the paper-and-pencil mode. There were no less missing values in the web-based mode compared to the paper-and-pencil mode. The study supports the idea that web-based modes provide, in general, equal responses to paper-and-pencil modes. However, health indicators based upon factual and objective items tend to have higher measurement agreement than indicators requiring an assessment of personal subjective feelings. A web-based mode greatly facilitates the data-entry process and guides the completing of a questionnaire. However, item non-response was not positively affected.