RESUMO
OBJECTIVES: To design a questionnaire to evaluate compliance with quality criteria on health web sites and to analyse its reliability. DESIGN: A descriptive study on the reliability of a questionnaire. PARTICIPANTS: Twenty web sites on health topics. SETTING: Internet. MAIN MEASUREMENTS: The questionnaire was based on analysis of content of the criteria of the e-Europe 2002 code of conduct, comparing these with the criteria underlying the AMA, the Summit code and the e-Health Code of Ethics regulations and current directives. The dimensions studied were transparency, absence of conflicts of interest, authorship, data protection, updating, accountability, and accessibility. A preliminary questionnaire was drawn up for a pilot test conducted by 3 researchers for 20 web sites, its reliability was appraised, adjustments were made and the definitive questionnaire was designed. Reliability was evaluated again for 26 web sites evaluated by 3 researchers. To evaluate concordance in the answers of the participants, Cohen's Kappa index with a 95% confidence level was used. RESULTS: All the quality directives of e-Europe 2002, the European regulations and relevant points from other questionnaires were included in the questionnaire designed. Reliability was acceptable (kappa > or = 0.60) for 12 of the 18 quality criteria included. The least concordant aspects of the questionnaire were information update (kappa = 0.310) and accessibility (search for contents and access for persons with disability). CONCLUSIONS: Reliability of the questionnaire designed was acceptable.