Unveiling the female experience through adult mortality and survivorship in Milan over the last 2000 years.

This study challenges historical paradigms using a large-scale integrated bioarchaeological approach, focusing on the female experience over the last 2,000 years in Milan, Italy. Specifically, 492 skeletons from the osteological collection of Milan were used to elucidate female survivorship and mortality by integrating bioarchaeological and paleopathological data, paleoepidemiological analyses, and historical contextualization. Findings revealed changes in female longevity, with a notable increase from Roman to contemporary eras, albeit plateauing in the Middle Ages/modern period. Significant sex-specific differences in mortality risk and survivorship were observed: females had higher mortality risk and lower survivorship in the Roman (first-fifth century AD) and Modern (16th-18th century AD) eras, but this trend reversed in the contemporary period (19th-20th century AD). Cultural and social factors negatively impacted female mortality in Roman and modern Milan, while others buffered it during the Middle Ages (sixth-15th century AD). This study underscored the importance of bioarchaeological inquiries in reconstructing the past, providing answers that may challenge historical assumptions and shedding light on how the interplay of cultural, social, and biological factors shaped the female experience across millennia.

FHIR Implementation Guide for Stroke: A dual focus on the patient's clinical pathway and value-based healthcare.

BACKGROUND: Stroke management requires a coordinated strategy, adhering to clinical pathways (CP) and value-based healthcare (VBHC) principles from onset to rehabilitation. However, the discrepancies between these pathways and actual patient experiences highlight the need for ongoing monitoring and addressing interoperability issues across multiple institutions in stroke care. To address this, the Fast Healthcare Interoperability Resource (FHIR) Implementation Guide (IG) standardizes the information exchange among these systems, considering a specific context of use. OBJECTIVE: Develop an FHIR IG for stroke care rooted in established stroke CP and VBHC principles. METHOD: We represented the stroke patient journey by considering the core stroke CP, the International Consortium for Health Outcomes Measurement (ICHOM) dataset for stroke, and a Brazilian case study using the Business Process Model and Notation (BPMN). Next, we developed a data dictionary that aligns variables with existing FHIR resources and adapts profiling from the Brazilian National Health Data Network (BNHDN). RESULTS: Our BPMN model encompassed three critical phases that represent the entire patient journey from symptom onset to rehabilitation. The stroke data dictionary included 81 variables, which were expressed as questionnaires, profiles, and extensions. The FHIR IG comprised nine pages: Home, Stroke-CP, Data Dictionary, FHIR, ICHOM, Artifacts, Examples, Downloads, and Security. We developed 96 artifacts, including 7 questionnaires, 27 profiles with corresponding example instances, 3 extensions, 18 value sets, and 14 code systems pertinent to ICHOM outcome measures. CONCLUSION: The FHIR IG for stroke in this study represents a significant advancement in healthcare interoperability, streamlining the tracking of patient outcomes for quality enhancement, facilitating informed treatment choices, and enabling the development of dashboards to promote collaborative excellence in patient care.

Prediction of the occurrence of leprosy reactions based on Bayesian networks.

Introduction: Leprosy reactions (LR) are severe episodes of intense activation of the host inflammatory response of uncertain etiology, today the leading cause of permanent nerve damage in leprosy patients. Several genetic and non-genetic risk factors for LR have been described; however, there are limited attempts to combine this information to estimate the risk of a leprosy patient developing LR. Here we present an artificial intelligence (AI)-based system that can assess LR risk using clinical, demographic, and genetic data. Methods: The study includes four datasets from different regions of Brazil, totalizing 1,450 leprosy patients followed prospectively for at least 2 years to assess the occurrence of LR. Data mining using WEKA software was performed following a two-step protocol to select the variables included in the AI system, based on Bayesian Networks, and developed using the NETICA software. Results: Analysis of the complete database resulted in a system able to estimate LR risk with 82.7% accuracy, 79.3% sensitivity, and 86.2% specificity. When using only databases for which host genetic information associated with LR was included, the performance increased to 87.7% accuracy, 85.7% sensitivity, and 89.4% specificity. Conclusion: We produced an easy-to-use, online, free-access system that identifies leprosy patients at risk of developing LR. Risk assessment of LR for individual patients may detect candidates for close monitoring, with a potentially positive impact on the prevention of permanent disabilities, the quality of life of the patients, and upon leprosy control programs.

'Quer N0 AVC' for Monitoring Stroke Patients' Healthcare Using a Mobile App.

Stroke is one of the leading causes of death and impairments worldwide. After hospital discharge, it is necessary to monitor these patients during their recovery. This research addresses the implementation of a mobile app, entitled 'Quer N0 AVC', to improve the quality of stroke patient care in Joinville, Brazil. The study method was divided into two parts. The adaptation phase included all the necessary information in the app for monitoring stroke patients. The implementation phase aimed to prepare a routine for the Quer mobile app installation. One of the questionnaires collected data from 42 patients and identified that before hospital admission 29% of them did not have medical appointments, 36% had one or two appointments, 11% had three appointments, and 24% had four or more appointments. This research portrayed adaptation feasibility and the implementation of a cell phone app for following up on stroke patients.

Prediction of the occurrence of leprosy reactions based on Bayesian networks

Introduction: Leprosy reactions (LR) are severe episodes of intense activation of the host inflammatory response of uncertain etiology, today the leading cause of permanent nerve damage in leprosy patients. Several genetic and non-genetic risk factors for LR have been described; however, there are limited attempts to combine this information to estimate the risk of a leprosy patient developing LR. Here we present an artificial intelligence (AI)-based system that can assess LR risk using clinical, demographic, and genetic data. Methods: The study includes four datasets from different regions of Brazil, totalizing 1,450 leprosy patients followed prospectively for at least 2 years to assess the occurrence of LR. Data mining using WEKA software was performed following a two-step protocol to select the variables included in the AI system, based on Bayesian Networks, and developed using the NETICA software. Results: Analysis of the complete database resulted in a system able to estimate LR risk with 82.7% accuracy, 79.3% sensitivity, and 86.2% specificity. When using only databases for which host genetic information associated with LR was included, the performance increased to 87.7% accuracy, 85.7% sensitivity, and 89.4% specificity. Conclusion: We produced an easy-to-use, online, free-access system that identifies leprosy patients at risk of developing LR. Risk assessment of LR for individual patients may detect candidates for close monitoring, with a potentially positive impact on the prevention of permanent disabilities, the quality of life of the patients, and upon leprosy control programs.

Decision-Making Support in Stroke Diagnosis Process: An Approach Based on the PROMETHEE Method and Decision Model Notation.

Decision-making in the field of healthcare is a very complex activity. Several tools have been developed to support the decision-making process. DMN, a modeling technique focused on decisions, is among these and has been gaining prominence in both, literature and business, as has the multi-criteria method PROMETHEE II that helps decision-makers with multi-criteria in analyses. Thus, this research targets combining these two techniques and analyzing the decision support that these two tools afford together. The diagnostic stage of stroke patients was used to perform this work. The research demonstrated that this proposal can drive major gains in efficiency and assertiveness in decision-making in time-sensitive hospital processes. After all, there is a noticeable dearth of hospitals with specialized teams as well as a shortfall of adequate infrastructure for this treatment.

SemClinBr - a multi-institutional and multi-specialty semantically annotated corpus for Portuguese clinical NLP tasks.

BACKGROUND: The high volume of research focusing on extracting patient information from electronic health records (EHRs) has led to an increase in the demand for annotated corpora, which are a precious resource for both the development and evaluation of natural language processing (NLP) algorithms. The absence of a multipurpose clinical corpus outside the scope of the English language, especially in Brazilian Portuguese, is glaring and severely impacts scientific progress in the biomedical NLP field. METHODS: In this study, a semantically annotated corpus was developed using clinical text from multiple medical specialties, document types, and institutions. In addition, we present, (1) a survey listing common aspects, differences, and lessons learned from previous research, (2) a fine-grained annotation schema that can be replicated to guide other annotation initiatives, (3) a web-based annotation tool focusing on an annotation suggestion feature, and (4) both intrinsic and extrinsic evaluation of the annotations. RESULTS: This study resulted in SemClinBr, a corpus that has 1000 clinical notes, labeled with 65,117 entities and 11,263 relations. In addition, both negation cues and medical abbreviation dictionaries were generated from the annotations. The average annotator agreement score varied from 0.71 (applying strict match) to 0.92 (considering a relaxed match) while accepting partial overlaps and hierarchically related semantic types. The extrinsic evaluation, when applying the corpus to two downstream NLP tasks, demonstrated the reliability and usefulness of annotations, with the systems achieving results that were consistent with the agreement scores. CONCLUSION: The SemClinBr corpus and other resources produced in this work can support clinical NLP studies, providing a common development and evaluation resource for the research community, boosting the utilization of EHRs in both clinical practice and biomedical research. To the best of our knowledge, SemClinBr is the first available Portuguese clinical corpus.

Analysis of Stroke Assistance in Covid-19 Pandemic by Process Mining Techniques.

Medical assistance to stroke patients must start as early as possible; however, several changes have impacted healthcare services during the Covid-19 pandemic. This research aimed to identify the stroke onset-to-door time during the Covid-19 pandemic considering the different paths a patient can take until receiving specialized care. It is a retrospective study based on process mining (PM) techniques applied to 221 electronic healthcare records of stroke patients during the pandemic. The results are two process models representing the patient's path and performance, from the onset of the first symptoms to admission to specialized care. PM techniques have discovered the patient journey in providing fast stroke assistance.

Representation of Diagnosis and Nursing Interventions in OpenEHR Archetypes.

OBJECTIVE: The study aimed to represent the content of nursing diagnosis and interventions in the openEHR standard. METHODS: This is a developmental study with the models developed according to ISO 18104: 2014. The Ocean Archetype Editor tool from the openEHR Foundation was used. RESULTS: Two archetypes were created; one to represent the nursing diagnosis concept and the other the nursing intervention concept. Existing archetypes available in the Clinical Knowledge Manager were reused in modeling. CONCLUSION: The representation of nursing diagnosis and interventions based on the openEHR standard contributes to representing nursing care phenomena and needs in health information systems.

Early Identification of Patients at Risk of Sepsis in a Hospital Environment

Abstract Sepsis is a systematic response to an infectious disease, being a concerning factor because of the increase in the mortality ratio for every delayed hour in the identification and start of patient's treatment. Studies that aim to identify sepsis early are valuable for the healthcare domain. Further, studies that propose machine learning-based models to identify sepsis risk are scarce for the Brazilian scenario. Hence, we propose the early identification of sepsis considering data from a Brazilian hospital. We developed a temporal series based on LSTM to predict sepsis in patients considering a three-day timestep. The patients were selected using both criteria, ICD-10, and qSOFA, where we supplemented qSOFA with the additional identification of words referring to infections in the clinical texts. Additionally, we tested a Random Forest classifier to classify patients with sepsis with a single timestep before the sepsis event, evaluating the most relevant features. We achieved an accuracy of 0.907, a sensitivity of 0.912, and a specificity of 0.971 when considering a three-day timestep with LSTM. The Random Forest classifier achieved an accuracy of 0.971, a sensitivity of 0.611, and a specificity of 0.998. The features age, blood glucose, systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, and admission days had the most influence over the algorithm classification, with age being the most relevant feature. We achieved satisfactory results compared with the literature considering a scenario of spaced measures and a high amount of missing data.

Ischemic stroke: Process perspective, clinical and profile characteristics, and external factors.

OBJECTIVE: To describe a method of analysis for understanding the health care process, enriched with information on the clinical and profile characteristics of the patients. To apply the proposed technique to analyze an ischemic stroke dataset. MATERIALS AND METHODS: We analyzed 4,830 electronic health records (EHRs) from patients with ischemic stroke (2010-2017), containing information about events realized during treatment and clinical and profile information of the patients. The proposed method combined process mining techniques with data analysis, grouping the data by primary care units (PCU - units responsible for the primary care of patients residing in a geographical area). RESULTS: A novel method, named process, data, and management (PDM) analysis method was used for ischemic stroke data and it provided the following outcomes: health care process for patients with ischemic stroke with time statistics; analysis of potential factors for slow hospital admission indicating an increase in the time to hospital admission of 3.4 h (mean value) for patients with an origin at the urgent care center (UCC) - 30% of patients; analysis of PCUs with distinct secondary stroke rates indicating that the social class of patients is the main difference between them; and the visualization of risk factors (before the stroke) by the PCU to inform the health manager about the potential of prevention. DISCUSSION: PDM analysis describes a step-by-step method for combining process analysis with data analysis considering a management focus. The results obtained on the stroke context can support the definition of more refined action plans by the health manager, improving the stroke health care process and preventing new events. CONCLUSION: When a patient is diagnosed with ischemic stroke, immediate treatment is needed. Moreover, it is possible to prevent new events to some degree by monitoring and treating risk factors. PDM analysis provides an overview of the health care process with time, combining elements that affect the treatment flow and factors, which can indicate a potential for preventing new events. We also can apply PDM analysis in different scenarios, when there is information about activities from treatment flow and other characteristics related to the treatment or the prevention of the analyzed disease. The management focus of the results aids in the formulation of service policies, action plans, and resource allocation.

ISO/TR 12300:2016 for clinical cross-terminology mapping: contribution to nursing.

This theoretical and reflective study aimed to assess the contribution of the ISO/TR 12300:2016 document for the mapping of nursing terminology. The referred document and related articles were used as an empirical framework. The study analyzed the content of the document, highlighting cardinality and equivalence principles. The standard presents conceptual and operational basis for mapping, with cardinality and equivalence as the support for the categorization of cross-terminology mapping in the area of nursing. Cardinality verifies candidate target terms to represent the source term, while the equivalence degree scale checks semantic correspondence. Among the principles included in the ISO/TR 12300:2016, cardinality and equivalence contribute to the accurate representation of the results of the cross-terminology mapping process and its use should decrease inconsistencies.

Comparison of the Results of Manual and Automated Processes of Cross-Mapping Between Nursing Terms: Quantitative Study.

BACKGROUND: Cross-mapping establishes equivalence between terms from different terminology systems, which is useful for interoperability, updated terminological versions, and reuse of terms. Due to the number of terms to be mapped, this work can be extensive, tedious, and thorough, and it is susceptible to errors; this can be minimized by automated processes, which use computational tools. OBJECTIVE: The aim of this study was to compare the results of manual and automated term mapping processes. METHODS: In this descriptive, quantitative study, we used the results of two mapping processes as an empirical basis: manual, which used 2638 terms of nurses' records from a university hospital in southern Brazil and the International Classification for Nursing Practice (ICNP); and automated, which used the same university hospital terms and the primitive terms of the ICNP through MappICNP, an algorithm based on rules of natural language processing. The two processes were compared via equality and exclusivity assessments of new terms of the automated process and of candidate terms. RESULTS: The automated process mapped 569/2638 (21.56%) of the source bank's terms as identical, and the manual process mapped 650/2638 (24.63%) as identical. Regarding new terms, the automated process mapped 1031/2638 (39.08%) of the source bank's terms as new, while the manual process mapped 1251 (47.42%). In particular, manual mapping identified 101/2638 (3.82%) terms as identical and 429 (16.26%) as new, whereas the automated process identified 20 (0.75%) terms as identical and 209 (7.92%) as new. Of the 209 terms mapped as new by the automated process, it was possible to establish an equivalence with ICNP terms in 48 (23.0%) cases. An analysis of the candidate terms offered by the automated process to the 429 new terms mapped exclusively by the manual process resulted in 100 (23.3%) candidates that had a semantic relationship with the source term. CONCLUSIONS: The automated and manual processes map identical and new terms in similar ways and can be considered complementary. Direct identification of identical terms and the offering of candidate terms through the automated process facilitate and enhance the results of the mapping; confirmation of the precision of the automated mapping requires further analysis by researchers.

ISO/TR 12300:2016 for clinical cross-terminology mapping: contribution to nursing / ISO/TR 12300:2016 para mapeo entre terminologías clínicas: aporte para la enfermeira / ISO/TR 12300:2016 para mapeamento entre terminologias clínicas: contribuição para a enfermagem

Abstract This theoretical and reflective study aimed to assess the contribution of the ISO/TR 12300:2016 document for the mapping of nursing terminology. The referred document and related articles were used as an empirical framework. The study analyzed the content of the document, highlighting cardinality and equivalence principles. The standard presents conceptual and operational basis for mapping, with cardinality and equivalence as the support for the categorization of cross-terminology mapping in the area of nursing. Cardinality verifies candidate target terms to represent the source term, while the equivalence degree scale checks semantic correspondence. Among the principles included in the ISO/TR 12300:2016, cardinality and equivalence contribute to the accurate representation of the results of the cross-terminology mapping process and its use should decrease inconsistencies.

Resumen Este estudio teórico reflexivo tiene como fin reflexionar acerca del aporte de la norma ISO/TR 12300:2016 para el mapeo de terminologías en el área de enfermería. Fueron utilizados como base empírica la mencionada norma y artículos relacionados, analizando el contenido de la norma y destacando los principios de cardinalidad y equivalencia. La norma presenta bases conceptuales y operativas para el mapeo, con la cardinalidad y la equivalencia, anclando la categorización de los resultados de los mapeos entre terminologías en el área de enfermería. La cardinalidad verifica los términos meta candidatos para representar el término fuente, mientras que la escala de grado de equivalencia verifica la correspondencia semántica. Entre los principios incluidos en la ISO/TR 12300:2016, la cardinalidad y la equivalencia contribuyen a la representación precisa de los resultados del proceso de mapeo cruzado y su empleo debe de reducir inconsistencias.

Resumo Este estudo teórico-reflexivo teve como objetivo refletir sobre a contribuição da norma ISO/TR 12300:2016 para mapeamento de terminologias na área de enfermagem. Foram utilizados como base empírica a referida norma e artigos relacionados, analisando o conteúdo da norma e destacando os princípios de cardinalidade e equivalência. A norma apresenta bases conceituais e operacionais para o mapeamento, com a cardinalidade e a equivalência, ancorando a categorização dos resultados dos mapeamentos entre terminologias na área de enfermagem. A cardinalidade verifica os termos-alvo candidatos para representar o termo-fonte, enquanto a escala de grau de equivalência verifica a correspondência semântica. Entre os princípios inclusos na ISO/TR 12300:2016, a cardinalidade e a equivalência contribuem para a representação precisa dos resultados do processo de mapeamento cruzado e seu uso deve diminuir inconsistências.

Learning Portuguese Clinical Word Embeddings: A Multi-Specialty and Multi-Institutional Corpus of Clinical Narratives Supporting a Downstream Biomedical Task.

In this paper, we trained a set of Portuguese clinical word embedding models of different granularities from multi-specialty and multi-institutional clinical narrative datasets. Then, we assessed their impact on a downstream biomedical NLP task of Urinary Tract Infection disease identification. Additionally, we intrinsically evaluated our main model using an adapted version of Bio-SimLex for the Portuguese language. Our empirical results showed that the larger, coarse-grained model achieved a slightly better outcome when compared with the small, fine-grained model in the proposed task. Moreover, we obtained satisfactory results with Bio-SimLex intrinsic evaluation.

Automatic Mapping Between Brazilian Portuguese Clinical Terms and International Classification for Nursing Practice.

This study describes MappICNP, an automatic method for mapping between Brazilian Portuguese clinical narratives in free text and International Classification for Nursing Practice (ICNP) concepts. It's composed of six natural language processing rules, related to terms comparison. A set of 2,638 terms extracted from hospitals nursing notes was mapped. MappICNP helps to map 1,607 terms, 113 less than a manual approach. The results demostrate its advantages in minimizing the time spent and reducing the scope of analysis through candidate terms of ICNP.

Mineração de dados na avaliação de óbitos após cirurgia de amputação / Data mining to evaluate mortality after amputation surgery

Contexto: A amputação e a desarticulação objetivam melhorar a saúde de um indivíduo, mas esses tratamentos apresentam taxas significantes de mortalidade que variam de acordo com os fatores relacionados. Objetivo: Identificar as associações entre os determinantes da mortalidade pós-operatória da amputação. Métodos: Estudo do tipo caso-controle (óbito versus não óbito) em que foi adotada a descoberta de regras de associação (abordagem da mineração de dados) e métricas epidemiológicas sobre 173 registros de pacientes amputados em um hospital público de Santa Catarina em 2014. Resultados: Os principais determinantes foram: idade > 60 anos [ odds ratio (OR) = 3,0], sexo feminino (OR = 2,0), baixa escolaridade, hipertensão (OR = 3,0), diabetes (OR = 1,6) e tabagismo (OR = 1,8). Dos pacientes com idade entre 60 a 69 anos (38%), 87,9% evoluíram para alta, estando o óbito associado a doença vascular periférica. Quando a idade foi > 70 anos, embolia e trombose de artérias dos membros inferiores foram o fator de exceção (óbito). As patologias com maior associação ao óbito foram doença vascular (47,0%), diabetes (29,4%), doença cardíaca (razão de risco = 11,4), doença renal (OR = 10,4) e doença pulmonar (OR = 5,2). As cirurgias proximais estiveram mais associadas ao óbito do que as distais. Entre os pacientes que foram a óbito, 76,0% foram submetidos a raquianestesia e 24,0% a anestesia geral. Conclusão: A mineração de dados permitiu identificar as associações vinculadas ao óbito entre as diferentes variáveis e diagnósticos, como por exemplo, entre idade > 70 anos e diagnóstico de embolia e trombose de artérias dos membros inferiores

Background: The objective of amputation and disarticulation is to improve health. However, these treatments are associated with significant mortality rates that vary in relation to risk factors. Objective: To identify associations between determinants of postoperative mortality after amputation surgery. Methods: Case-control study (death vs. no death) considering data from 173 patients who underwent amputation surgery at a public hospital in Santa Catarina state, Brazil. These data were analyzed using a data mining approach to discover association rules and epidemiologic association metrics. Results: The main determinants were age > 60 years (odds ratio (OR) = 3.0), female sex (OR = 2.0), low education, hypertension (OR = 3.0), diabetes (OR = 1.6), and smoking (OR = 1.8). Among patients aged 60-69 years, 87.9% survived to discharge from hospital. The exceptions occurred when patients in this age range had peripheral vascular disease. The same was true when age was > 70 years, among whom diagnoses of embolism and thrombosis of arteries of the lower extremities were the exception factors (associated with death). The most common pathologies associated with death were vascular disease (47.0%) and diabetes (29.4%), heart disease (relative risk = 11.4), renal disease (OR = 10.4), and lung disease (OR = 5.2). Proximal surgeries were more strongly associated with death than distal ones. Among the deaths, 76.0% had been given spinal anesthesia and 24.0% general anesthesia. Conclusion: Data mining enabled identification of associations between death and a variety of different variables and diagnostic hypotheses; for example, age > 70 years and diagnosis of embolism and thrombosis of arteries of the lower extremities

Exploiting Temporal Constraints of Clinical Guidelines by Applying OpenEHR Archetypes.

Studies describing Computer-Interpretable Clinical Guidelines (CIG) with temporal constrains (TC) generally have not addressed issues related to their integration into Electronic Health Record (EHR) systems. This study aimed to represent TCs contained in clinical guidelines by applying archetypes and Guideline Definition Language (GDL) to incorporate decision support into EHRs. An example of each TC class in the clinical guideline for management of Atrial Fibrillation was represented using archetypes and GDL.

Métodos para Avaliação de Sistema de Informação em Saúde / Methods for Evaluation of Information Systems in Healthcare / Métodos para la Evaluación de los Sistemas de Información en Salud

Objetivos: Definir o que é a avaliação de SIS, descrevendo os aspectos considerados e os métodos aplicados. Método: Revisão sistemática na base de dados PubMed no período de janeiro de 2004 a junho de 2014. Resultados: A maioria dos estudos avaliou os aspectos da usabilidade, eficácia e qualidade da informação. Em 29% dos estudos foi encontrada aplicação de métodos combinados com: satisfação e aceitação do usuário; questionários; usabilidade; e entrevista com o grupo. Avaliação da funcionalidade; do impacto; usabilidade e desempenho clínico/diagnóstico são utilizados de forma isolada. Conclusão: É fundamental ter bem claro o que será avaliado no SIS, sendo essencial conhecer os aspectos envolvidos, os métodos existentes e como são aplicados.

Objectives: To define what is the assessment of HIS, describing the aspects considered and the methods applied. Methods: Systematic review in PubMed database from January 2004 to June 2014. Results: Most studies evaluating aspects of usability, efficacy and quality of information. In 29% of studies were applied methods together with: satisfaction and user acceptance; questionnaires; usability; and interview with the group. Reviewed functionality; impact; usability and clinical/diagnostic performance were generally used separated. Conclusion: It is essential to have clear what it will be evaluated in the SIS, and to know the aspects involved, the existing methods and how they are applied.

Objetivos: Definir qué es la evaluación del SIS, describir los aspectos considerados y los métodos aplicados. Métodos: Revisión sistemática de la base de datos PubMed de enero 2004 a junio 2014. Resultados: La mayoría de los estudios evaluaron los aspectos de usabilidad, la eficacia y la calidad de la información. En 29% de los estudios se encontró la aplicación de métodos combinados con: satisfacción y aceptación de los usuarios; cuestionarios; facilidad de uso; y una entrevista con el grupo. La evaluación de  funcionalidad; impacto; usabilidad y el rendimiento clínico / diagnóstico se utilizan de forma aislada. Conclusión: Es esencial tener claro los objetivos de la evaluación del SIS, además es esencial conocer los aspectos involucrados, los métodos existentes y cómo éstos son aplicados.