[Development and validation of performance indicators of the National School Food Program management]. / Desenvolvimento e validação de indicadores de desempenho da gestão do Programa Nacional de Alimentação Escolar.

The evaluation of the National School Food Program (PNAE) is a strategic action to ensure the quality of management. This study aims to develop and validate an instrument of performance indicators to assess the management of the PNAE in the national context. A psychometric study was carried out for the development and validation of performance indicators. Twelve indicators were subsequently established and evaluated by an expert panel concerning seven attributes. The Delphi technique was used for the consensus of the expert panel, and the content validity of the indicators was analyzed with the Content Validity Ratio. Nutritionists working on the implementation of the PNAE assessed the relevance of the indicators by answering an online questionnaire to construct validity and reliability. Thirteen (43.3%, 13/30) experts participated in the Delphi round 1 and nine (69.2%, 9/13) completed the Delphi round 2. A new indicator was developed after the experts' suggestions. Furthermore, 281 nutritionists participated in this study assessing the relevance of the indicators. Overall, content and construct validity were achieved for 13 indicators. All were considered relevant and had the potential to promote the assessment of PNAE operational management.

A avaliação do Programa Nacional de Alimentação Escolar (PNAE) é ação estratégica para garantir a qualidade da gestão. O objetivo deste estudo é desenvolver e validar um instrumento de indicadores de desempenho para avaliação da gestão do PNAE no contexto nacional. Trata-se de um estudo psicométrico de construção e validação de indicadores de desempenho. Foram estabelecidos 12 indicadores, posteriormente, avaliados em painel de especialistas em relação a sete atributos. Para obter consenso foi utilizada a técnica Delphi e a validade de conteúdo foi analisada com a Razão de Validade de Conteúdo. Para a validade de constructo e confiabilidade, nutricionistas que trabalham na execução do PNAE avaliaram a relevância dos indicadores respondendo um questionário online. A taxa de resposta dos especialistas na primeira rodada da técnica Delphi foi de 43,3% (13/30) e de 69,2% (9/13) na segunda rodada. Um novo indicador foi desenvolvido após a avaliação do painel de especialistas. Um total de 281 nutricionistas participaram do estudo da relevância dos indicadores. No geral, a validade de conteúdo e constructo foi alcançada para 13 indicadores. Todos os 13 indicadores se apresentaram relevantes, com potencial para promover a avaliação da gestão operacional do PNAE.

Evaluation of quality indicators for management of the National School Feeding Program in Brazil: a systematic review.

This article aims to identify studies that developed quality indicator for the management of the National School Feeding Program (PNAE, in Brazilian context) and to critically appraise the properties of their instruments. Systematic review using Scopus, Lilacs, Pubmed and Web of Science for data collection. The search was limited to studies between 2009 and 2019. The search strategy included search terms related to school feeding, program evaluation, and indicator. The indicators were evaluated using the Appraisal of Indicators through Research and Evaluation instrument. The search identified 1,355 studies, of which 14 were potentially relevant records and 10 met the inclusion criteria. Most studies used a literature review with consensus techniques in the development of the instrument and a frame work format to evaluate the PNAE. None of them presented evidence of validity of the instrument. The highest level was achieved on the domain 'Purpose, relevance and organizational context', followed by 'Stakeholder involvement', 'Additional evidence, formulation and usage', and 'Scientific evidence'. This review found gaps in the methodology of studies that had developed quality indicators for the management of PNAE. Future development of these instruments should include validity evidence.

Systematic review of instruments for assessing culinary skills in adults: What is the quality of their psychometric properties?

BACKGROUND: Culinary skills are important objects of study in the field of Public Health. Studies that propose to develop instruments for assessing such construct show lack of methodological uniformity to report validity and reliability of their instruments. OBJECTIVE: To identify studies that have developed instruments to measure culinary skills in adult population, and critically assess their psychometric properties. DESIGN: We conducted a systematic review according to the PRISMA statement. We searched literature PubMed/Medline, Scopus, LILACS, and Web of Science databases until January 2021, and consulted Google Scholar for relevant grey literature. Two reviewers independently selected the studies, conducted data extraction, and assessed the psychometric quality of the instruments. A third reviewer resolved any doubts or disagreements in all steps of the systematic review. RESULTS: The search identified 1148 potentially relevant studies, out of which 9 met the inclusion criteria. In addition, we included 3 studies by searching the related articles and the reference lists of these studies, totaling 12 included studies in this review. Ten studies reported the development of tools measuring culinary skills in adults and 2 studies performed cross-cultural adaptations of original instruments. We considered adequate quality of internal consistency reliability in four studies. One study received adequate rating for test-retest reliability. No studies presented adequate rating for content validity and four studies showed satisfactory results for at least one type of construct validity. One study reported criterion validity and the quality of this psychometric property was inadequate. CONCLUSIONS: We identified many studies that surveyed culinary skills. Although the isolated measures appraised in this review show good promise in terms of quality of psychometric properties, no studies presented adequate measures for each aspect of reliability and validity. A more consistent and consensual definition of culinary skills is recommended. The flaws observed in these studies show that there is a need for ongoing research in the area of the psychometric properties of instruments assessing culinary skills.

Desenvolvimento e validação de indicadores de desempenho da gestão do Programa Nacional de Alimentação Escolar / Development and validation of performance indicators of the National School Food Program management

Resumo A avaliação do Programa Nacional de Alimentação Escolar (PNAE) é ação estratégica para garantir a qualidade da gestão. O objetivo deste estudo é desenvolver e validar um instrumento de indicadores de desempenho para avaliação da gestão do PNAE no contexto nacional. Trata-se de um estudo psicométrico de construção e validação de indicadores de desempenho. Foram estabelecidos 12 indicadores, posteriormente, avaliados em painel de especialistas em relação a sete atributos. Para obter consenso foi utilizada a técnica Delphi e a validade de conteúdo foi analisada com a Razão de Validade de Conteúdo. Para a validade de constructo e confiabilidade, nutricionistas que trabalham na execução do PNAE avaliaram a relevância dos indicadores respondendo um questionário online. A taxa de resposta dos especialistas na primeira rodada da técnica Delphi foi de 43,3% (13/30) e de 69,2% (9/13) na segunda rodada. Um novo indicador foi desenvolvido após a avaliação do painel de especialistas. Um total de 281 nutricionistas participaram do estudo da relevância dos indicadores. No geral, a validade de conteúdo e constructo foi alcançada para 13 indicadores. Todos os 13 indicadores se apresentaram relevantes, com potencial para promover a avaliação da gestão operacional do PNAE.

Abstract The evaluation of the National School Food Program (PNAE) is a strategic action to ensure the quality of management. This study aims to develop and validate an instrument of performance indicators to assess the management of the PNAE in the national context. A psychometric study was carried out for the development and validation of performance indicators. Twelve indicators were subsequently established and evaluated by an expert panel concerning seven attributes. The Delphi technique was used for the consensus of the expert panel, and the content validity of the indicators was analyzed with the Content Validity Ratio. Nutritionists working on the implementation of the PNAE assessed the relevance of the indicators by answering an online questionnaire to construct validity and reliability. Thirteen (43.3%, 13/30) experts participated in the Delphi round 1 and nine (69.2%, 9/13) completed the Delphi round 2. A new indicator was developed after the experts' suggestions. Furthermore, 281 nutritionists participated in this study assessing the relevance of the indicators. Overall, content and construct validity were achieved for 13 indicators. All were considered relevant and had the potential to promote the assessment of PNAE operational management.