Logic Models for Evaluation of Complex Health Information Systems.

BACKGROUND: Complex health IT needs to be planned and evaluated. OBJECTIVES: To propose logic models for the evaluation of complex health IT. METHODS: Logic models describe input, activities, output, outcome, and impact. RESULTS AND CONCLUSION: This first example of a logic model for patient portals shows how health IT planning and evaluation may benefit from logic models.

Effects of Hospital Digitization on Clinical Outcomes and Patient Satisfaction: Nationwide Multiple Regression Analysis Across German Hospitals.

BACKGROUND: The adoption of health information technology (HIT) by health care providers is commonly believed to improve the quality of care. Policy makers in the United States and Germany follow this logic and deploy nationwide HIT adoption programs to fund hospital investments in digital technologies. However, scientific evidence for the beneficial effects of HIT on care quality at a national level remains mostly US based, is focused on electronic health records (EHRs), and rarely accounts for the quality of digitization from a hospital user perspective. OBJECTIVE: This study aimed to examine the effects of digitization on clinical outcomes and patient experience in German hospitals. Hence, this study adds to the small stream of literature in this field outside the United States. It goes beyond assessing the effects of mere HIT adoption and also considers user-perceived HIT value. In addition, the impact of a variety of technologies beyond EHRs was examined. METHODS: Multiple linear regression models were estimated using emergency care outcomes, elective care outcomes, and patient satisfaction as dependent variables. The adoption and user-perceived value of HIT represented key independent variables, and case volume, hospital size, ownership status, and teaching status were included as controls. Care outcomes were captured via risk-adjusted, observed-to-expected outcome ratios for patients who had stroke, myocardial infarction, or hip replacement. The German Patient Experience Questionnaire of Weisse Liste provided information on patient satisfaction. Information on the adoption and user-perceived value of 10 subdomains of HIT and EHRs was derived from the German 2020 Healthcare IT Report. RESULTS: Statistical analysis was based on an overall sample of 383 German hospitals. The analyzed data set suggested no significant effect of HIT or EHR adoption on clinical outcomes or patient satisfaction. However, a higher user-perceived value or quality of the installed tools did improve outcomes. Emergency care outcomes benefited from user-friendly overall digitization (ß=-.032; P=.04), which was especially driven by the user-friendliness of admission HIT (ß=-.023; P=.07). Elective care outcomes were positively impacted by user-friendly EHR installations (ß=-.138; P=.008). Similarly, the results suggested user-friendly, overall digitization to have a moderate positive effect on patient satisfaction (ß=-.009; P=.01). CONCLUSIONS: The results of this study suggest that hospital digitization is not an end in itself. Policy makers and hospitals are well advised to not only focus on the mere adoption of digital technologies but also continuously work toward digitization that is perceived as valuable by physicians and nurses who rely on it every day. Furthermore, hospital digitization strategies should consider that the assumed benefits of single technologies are not realized across all care domains.

Should App Self-Management Mean Self-Control? A Quantiative Study on App Supported Diabetes Self-Management.

Apps have been attested to empower patients regarding disease self-management through numerous studies. However, it is still unclear what factors determine the perception of patients whether an app is a useful tool for this purpose. A multiple regression model that was informed by the Technology Acceptance Model (TAM 2) was tested based on the answers of 235 app users with Diabetes type 1 or 2. The model accounted for 59.2% of the variance of the perceived degree of self-management. Factors belonging to the relevance-usefulness-quality complex as well as factors reflecting the patient's self-control were found to be significant in the model. Patient demographics, i.e. age, gender, app experience and type of Diabetes did not play any significant role. In conclusion, this study raises the question whether apps should be designed to strengthen self-management in the sense of self-control (e.g. own measurements, diary) as opposed to guiding and advice giving.

The Effect of Innovation Capabilities of Health Care Organizations on the Quality of Health Information Technology: Model Development With Cross-sectional Data.

BACKGROUND: Large health organizations often struggle to build complex health information technology (HIT) solutions and are faced with ever-growing pressure to continuously innovate their information systems. Limited research has been conducted that explores the relationship between organizations' innovative capabilities and HIT quality in the sense of achieving high-quality support for patient care processes. OBJECTIVE: The aim of this study is to explain how core constructs of organizational innovation capabilities are linked to HIT quality based on a conceptual sociotechnical model on innovation and quality of HIT, called the IQHIT model, to help determine how better information provision in health organizations can be achieved. METHODS: We designed a survey to assess various domains of HIT quality, innovation capabilities of health organizations, and context variables and administered it to hospital chief information officers across Austria, Germany, and Switzerland. Data from 232 hospitals were used to empirically fit the model using partial least squares structural equation modeling to reveal associations and mediating and moderating effects. RESULTS: The resulting empirical IQHIT model reveals several associations between the analyzed constructs, which can be summarized in 2 main insights. First, it illustrates the linkage between the constructs measuring HIT quality by showing that the professionalism of information management explains the degree of HIT workflow support (R²=0.56), which in turn explains the perceived HIT quality (R²=0.53). Second, the model shows that HIT quality was positively influenced by innovation capabilities related to the top management team, the information technology department, and the organization at large. The assessment of the model's statistical quality criteria indicated valid model specifications, including sufficient convergent and discriminant validity for measuring the latent constructs that underlie the measures of HIT quality and innovation capabilities. CONCLUSIONS: The proposed sociotechnical IQHIT model points to the key role of professional information management for HIT workflow support in patient care and perceived HIT quality from the viewpoint of hospital chief information officers. Furthermore, it highlights that organizational innovation capabilities, particularly with respect to the top management team, facilitate HIT quality and suggests that health organizations establish this link by applying professional information management practices. The model may serve to stimulate further scientific work in the field of HIT adoption and diffusion and to provide practical guidance to managers, policy makers, and educators on how to achieve better patient care using HIT.

Professionalism of Information Management in Health Care: Development and Validation of the Construct and Its Measurement.

BACKGROUND: Against the background of a steadily increasing degree of digitalization in health care, a professional information management (IM) is required to successfully plan, implement, and evaluate information technology (IT). At its core, IM has to ensure a high quality of health data and health information systems to support patient care. OBJECTIVES: The goal of the present study was to define what constitutes professional IM as a construct as well as to propose a reliable and valid measurement instrument. METHODS: To develop and validate the construct of professionalism of information management (PIM) and its measurement, a stepwise approach followed an established procedure from information systems and behavioral research. The procedure included an analysis of the pertaining literature and expert rounds on the construct and the instrument, two consecutive and comprehensive surveys at the national and international level, exploratory and confirmatory factor analyses as well as reliability and validity testing. RESULTS: Professionalism of information management was developed as a construct consisting of the three dimensions of strategic, tactical, and operational IM as well as of the regularity and cyclical phases of IM procedures as the two elements of professionalism. The PIM instrument operationalized the construct providing items that incorporated IM procedures along the three dimensions and cyclical phases. These procedures had to be evaluated against their degree of regularity in the instrument. The instrument proved to be reliable and valid in two consecutive measurement phases and across three countries. CONCLUSION: It can be concluded that professionalism of information management is a meaningful construct that can be operationalized in a scientifically rigorous manner. Both science and practice can benefit from these developments in terms of improved self-assessment, benchmarking capabilities, and eventually, obtaining a better understanding of health IT maturity.

It Needs More Than Just User Participation: Combining Perspectives of Clinical Leaders and Chief Information Officers on Determinants of Hospitals' IT Innovativeness.

Although user participation may facilitate the realisation of IT innovations, various literature analyses show only minimal to moderate evidence for such effects possibly due to disregard of mediating factors. Against this background, this study examines the extent to which joint intrapreneurship of clinical leaders and IT leaders as well as a distinct innovation culture mediate the effect of user participation on hospitals' IT innovativeness. IT innovativeness was measured by the availability and usability of IT functions and by the perceived 'innovative power' of a hospital. An empirical model was developed and tested with data from 168 clinical leaders and IT leaders who participated pairwise in a survey representing 84 German hospitals. Three parallel mediation analyses indicated that the participation of users could only lead to IT innovativeness if they were accompanied by intrapreneurial leadership on the part of clinical directors and IT leaders and if a pronounced innovation culture prevailed.

Measuring the Availability of Electronic Patient Data Across the Hospital and Throughout Selected Clinical Workflows.

The workflow-oriented dissemination of electronic patient data is a central goal of IT deployment in hospitals. Against this background, the present study examines two research questions: (1.) Are there differences in the availability of electronic patient data (AEPD) between different clinical workflows and data types and (2.) which structural and organizational factors determine AEPD? Based on a Germany wide hospital survey, AEPD was assessed along six clinical workflows. While AEPD was lowest for ward rounds, discharge showed the highest AEPD with pre- and post-surgery processes ranging in between. With regard to the data types analyzed, patient demographics and observation findings obtained the highest AEPD scores. Electrophysiological results, checklists and warnings were less common electronically and received lower AEPD scores. Multiple linear regression analysis resulted in a significant model that explained 34.4% of the variance of AEPD. Large hospitals and those with a professional information management, a high health IT related innovation culture and a nursing informatics officer possess higher AEPD scores and thus have better clinical information logistics mechanisms at their command.

Determinants of Clinical Information Logistics: Tracing Socio-Organisational Factors and Country Differences from the Perspective of Clinical Directors.

The establishment of successful clinical information logistics (CIL) within the care processes is one of the main objectives of strategic health IT management in hospitals. While technical realisations in terms of useful, usable and interoperable IT solutions are essential precursors of CIL, there is limited empirical research on what socio-organisational factors underlie an innovation-friendly culture and how they can affect successful information provision. We applied factor analysis on survey data from 403 clinical directors from Germany, Austria and Switzerland and used the dimensions identified to explain the level of CIL with ordered logistic regression analysis. The intensity of collaboration and exchange with the IT department as well as the degree of executive IT leadership showed to be strongly associated with better CIL while personal views and attitudes of clinical directors were not. Analysing country differences revealed the degree of the exchange with the IT department to be significantly lower in German hospitals. This points at a potential strategic lever for German hospital executives to focus on.

Going Mobile: An Empirical Model for Explaining Successful Information Logistics in Ward Rounds.

BACKGROUND: Medical ward rounds are critical focal points of inpatient care that call for uniquely flexible solutions to provide clinical information at the bedside. While this fact is undoubted, adoption rates of mobile IT solutions remain rather low. OBJECTIVES: Our goal was to investigate if and how mobile IT solutions influence successful information provision at the bedside, i.e. clinical information logistics, as well as to shed light at socio-organizational factors that facilitate adoption rates from a user-centered perspective. METHODS: Survey data were collected from 373 medical and nursing directors of German, Austrian and Swiss hospitals and analyzed using variance-based Structural Equation Modelling (SEM). RESULTS: The adoption of mobile IT solutions explains large portions of clinical information logistics and is in itself associated with an organizational culture of innovation and end user participation. CONCLUSION: Results should encourage decision makers to understand mobility as a core constituent of information logistics and thus to promote close end-user participation as well as to work towards building a culture of innovation.

Hospital CEOs Need Health IT Knowledge and Trust in CIOs: Insights from a Qualitative Study.

BACKGROUND: IT is getting an increasing importance in hospitals. In this context, major IT decisions are often made by CEOs who are not necessarily IT experts. OBJECTIVES: Therefore, this study aimed at a) exploring different types of IT decision makers at CEO level, b) identifying hypotheses if trust exists between these different types of CEOs and their CIOs and c) building hypotheses on potential consequences regarding risk taking and innovation. METHODS: To this end, 14 qualitative interviews with German hospital CEOs were conducted to explore the research questions. RESULTS: The study revealed three major types: IT savvy CEOs, IT enthusiastic CEOs and IT indifferent CEOs. Depending on these types, their relationship with the CIO varied in terms of trust and common language. In case of IT indifferent CEOs, a potential vicious circle of lack of IT knowledge, missing trust, low willingness to take risks and low innovation power could be identified. CONCLUSION: In order to break of this circle, CEOs seem to need more IT knowledge and/or greater trust in their CIO.

Ready for HIT Innovations? Developing a Tool to Assess the Professionalism of Information Management in Hospitals.

Current frameworks postulate the success of health IT innovations to be determined by the professionalism of the information management (PIM). Still, empirical knowledge about PIM is scarce up until today. This study seeks to answer three research questions: (1.) How can PIM be measured in a reliable and valid way, (2.) how pronounced is PIM in German hospitals and (3.) do hospital characteristics have an impact on the degree of PIM? Based on the results of an expert workshop and frameworks for information management (IM) items for a PIM inventory were developed and the inventory sent to 1349 chief information officers of German hospitals. A principle component analysis based on the responses of 196 hospitals confirmed the three components that had been proposed by the frameworks: the strategic, the tactical and the operational level. The full inventory implied satisfying reliability and allowed a PIM composite-score to be calculated. The PIM scores for strategic and tactical IM were found to be far lower than for operational IM which hints at strong deficits in these areas. A stepwise regression model indicated that the degree of PIM significantly increased with the size of the hospital, which had been expected and hints the validity of the PIM inventory. This tool offers potentials for hospitals to classify and improve their IM.

Implementing a Data Management Platform for Longitudinal Health Research.

Health IT adoption research is rooted in Rogers' Diffusion of Innovation theory, which is based on longitudinal analyses. However, many studies in this field use cross-sectional designs. The aim of this study therefore was to design and implement a system to (i) consolidate survey data sets originating from different years (ii) integrate additional secondary data and (iii) query and statistically analyse these longitudinal data. Our system design comprises a 5-tier-architecture that embraces tiers for data capture, data representation, logics, presentation and integration. In order to historicize data properly and to separate data storage from data analytics a data vault schema was implemented. This approach allows the flexible integration of heterogeneous data sets and the selection of comparable items. Data analysis is prepared by compiling data in data marts and performed by R and related tools. IT Report Healthcare data from 2011, 2013 and 2017 could be loaded, analysed and combined with secondary longitudinal data.

Antecedents of CIOs' Innovation Capability in Hospitals: Results of an Empirical Study.

CIOs' innovation capability is regarded as a precondition of successful HIT adoption in hospitals. Based on the data of 142 CIOs, this study aimed at identifying antecedents of perceived innovation capability. Eight features describing the status quo of the hospital IT management (e.g. use of IT governance frameworks), four features of the hospital structure (e.g. functional diversification) and four CIO characteristics (e.g. duration of employment) were tested as potential antecedents in an exploratory stepwise regression approach. Perceived innovation capability in its entirety and its three sub-dimensions served as criterion. The results show that CIOs' perceived innovation capability could be explained significantly (R2=0.34) and exclusively by facts that described the degree of formalism and structure of IT management in a hospital, e.g. intensive and formalised strategic communication, the existence of an IT strategy and the use of IT governance frameworks. Breaking down innovation capability into its constituents revealed that "innovative organisational culture" contributed to a large extent (R2=0.26) to the overall result sharing several predictors. In contrast, "intrapreneurial personality" (R2=0.11) and "openness towards users" (R2=0.18) could be predicted less well. These results hint at the relationship between working in a well-structured, formalised and strategy oriented environment and the overall feeling of being capable to promote IT innovation.

Exploring Innovation Capabilities of Hospital CIOs: An Empirical Assessment.

Hospital CIOs play a central role in the adoption of innovative health IT. Until now, it remained unclear which particular conditions constitute their capability to innovate in terms of intrapersonal as well as organisational factors. An inventory of 20 items was developed to capture these conditions and examined by analysing data obtained from 164 German hospital CIOs. Principal component analysis resulted in three internally consistent components that constitute large portions of the CIOs innovation capability: organisational innovation culture, entrepreneurship personality and openness towards users. Results were used to build composite indicators that allow further evaluations.

Innovative Power of Health Care Organisations Affects IT Adoption: A bi-National Health IT Benchmark Comparing Austria and Germany.

Multinational health IT benchmarks foster cross-country learning and have been employed at various levels, e.g. OECD and Nordic countries. A bi-national benchmark study conducted in 2007 revealed a significantly higher adoption of health IT in Austria compared to Germany, two countries with comparable healthcare systems. We now investigated whether these differences still persisted. We further studied whether these differences were associated with hospital intrinsic factors, i.e. the innovative power of the organisation and hospital demographics. We thus performed a survey to measure the "perceived IT availability" and the "innovative power of the hospital" of 464 German and 70 Austrian hospitals. The survey was based on a questionnaire with 52 items and was given to the directors of nursing in 2013/2014. Our findings confirmed a significantly greater IT availability in Austria than in Germany. This was visible in the aggregated IT adoption composite score "IT function" as well as in the IT adoption for the individual functions "nursing documentation" (OR = 5.98), "intensive care unit (ICU) documentation" (OR = 2.49), "medication administration documentation" (OR = 2.48), "electronic archive" (OR = 2.27) and "medication" (OR = 2.16). "Innovative power" was the strongest factor to explain the variance of the composite score "IT function". It was effective in hospitals of both countries but significantly more effective in Austria than in Germany. "Hospital size" and "hospital system affiliation" were also significantly associated with the composite score "IT function", but they did not differ between the countries. These findings can be partly associated with the national characteristics. Indicators point to a more favourable financial situation in Austrian hospitals; we thus argue that Austrian hospitals may possess a larger degree of financial freedom to be innovative and to act accordingly. This study is the first to empirically demonstrate the effect of "innovative power" in hospitals on health IT adoption in a bi-national health IT benchmark. We recommend directly including the financial situation into future regression models. On a political level, measures to stimulate the "innovative power" of hospitals should be considered to increase the digitalisation of healthcare.

Understanding latent structures of clinical information logistics: A bottom-up approach for model building and validating the workflow composite score.

BACKGROUND AND PURPOSE: Clinical information logistics is a construct that aims to describe and explain various phenomena of information provision to drive clinical processes. It can be measured by the workflow composite score, an aggregated indicator of the degree of IT support in clinical processes. This study primarily aimed to investigate the yet unknown empirical patterns constituting this construct. The second goal was to derive a data-driven weighting scheme for the constituents of the workflow composite score and to contrast this scheme with a literature based, top-down procedure. This approach should finally test the validity and robustness of the workflow composite score. METHODS: Based on secondary data from 183 German hospitals, a tiered factor analytic approach (confirmatory and subsequent exploratory factor analysis) was pursued. A weighting scheme, which was based on factor loadings obtained in the analyses, was put into practice. RESULTS: We were able to identify five statistically significant factors of clinical information logistics that accounted for 63% of the overall variance. These factors were "flow of data and information", "mobility", "clinical decision support and patient safety", "electronic patient record" and "integration and distribution". The system of weights derived from the factor loadings resulted in values for the workflow composite score that differed only slightly from the score values that had been previously published based on a top-down approach. CONCLUSION: Our findings give insight into the internal composition of clinical information logistics both in terms of factors and weights. They also allowed us to propose a coherent model of clinical information logistics from a technical perspective that joins empirical findings with theoretical knowledge. Despite the new scheme of weights applied to the calculation of the workflow composite score, the score behaved robustly, which is yet another hint of its validity and therefore its usefulness.

An Iterative Methodology for Developing National Recommendations for Nursing Informatics Curricula.

The increasing importance of IT in nursing requires educational measures to support its meaningful application. However, many countries do not yet have national recommendations for nursing informatics competencies. We thus developed an iterative triple methodology to yield validated and country specific recommendations for informatics core competencies in nursing. We identified relevant competencies from national sources (step 1), matched and enriched these with input from the international literature (step 2) and fed the resulting 24 core competencies into a survey (120 invited experts from which 87 responded) and two focus group sessions with a total of 48 experts (steps 3a/3b). The subsequent focus group sessions confirmed and expanded the findings. As a result, we were able to define role specific informatics core competencies for three countries.

IT-benchmarking of clinical workflows: concept, implementation, and evaluation.

Due to the emerging evidence of health IT as opportunity and risk for clinical workflows, health IT must undergo a continuous measurement of its efficacy and efficiency. IT-benchmarks are a proven means for providing this information. The aim of this study was to enhance the methodology of an existing benchmarking procedure by including, in particular, new indicators of clinical workflows and by proposing new types of visualisation. Drawing on the concept of information logistics, we propose four workflow descriptors that were applied to four clinical processes. General and specific indicators were derived from these descriptors and processes. 199 chief information officers (CIOs) took part in the benchmarking. These hospitals were assigned to reference groups of a similar size and ownership from a total of 259 hospitals. Stepwise and comprehensive feedback was given to the CIOs. Most participants who evaluated the benchmark rated the procedure as very good, good, or rather good (98.4%). Benchmark information was used by CIOs for getting a general overview, advancing IT, preparing negotiations with board members, and arguing for a new IT project.

Characteristics of German hospitals adopting health IT systems - results from an empirical study.

Hospital characteristics that facilitate IT adoption have been described by the literature extensively, however with controversial results. The aim of this study therefore is to draw a set of the most important variables from previous studies and include them in a combined analysis for testing their contribution as single factors and their interactions. Total number of IT systems installed and number of clinical IT systems in the hospital were used as criterion variables. Data from a national survey of German hospitals served as basis. Based on a stepwise multiple regression analysis four variables were identified to significantly explain the degree of IT adoption (60% explained variance): 1) hospital size, 2) IT department, 3) reference customer and 4) ownership (private vs. public). Our results replicate previous findings with regard to hospital size and ownership. In addition our study emphasizes the importance of a reliable internal structure for IT projects (existence of an IT department) and the culture of testing and installing most recent IT products (being a reference customer). None of the interactions between factors was significant.