Stakeholders' Involvement in Health Technology Assessment in Kazakhstan, Poland and Bulgaria.

Background: In recent years, involvement of healthcare stakeholders in health technology assessment (HTA) has been discussed as helping the inclusion of social values in the decision-making process. The aim of our research was to identify and compare details from Kazakhstan, Poland and Bulgaria on their stakeholders' involvement in the HTA process. Information was sought on their identification, responsibilities, and regulation. Methods: We conducted a survey of seven types of stakeholders in the healthcare systems of Kazakhstan, Poland, and Bulgaria. They included patients and the public, providers, purchasers, payers, policy makers, product makers, and principal investigators. They were questioned on their involvement in the HTA process, and on the objectives of their participation. Results: Levels of involvement of different kinds of stakeholder varied between countries, reflecting political and administrative developments. There was full or partial agreement on the objectives of stakeholder participation. All respondents agreed that representatives of the ministry of health should be involved in selection of stakeholders for HTA. Conclusion: Progress has been made in the involvement of stakeholders, with interest in further development in all three countries.

The contribution of electronic health records to risk management through accreditation of residential aged care homes in Australia.

BACKGROUND: The Australian government has implemented a compulsory aged care accreditation system to guide and monitor the risk management approach in registered residential aged care (RAC) homes. This research assessed the contribution of electronic health records (EHR) to risk management in RAC homes in relation to the extent that aged care accreditation fulfils its role. METHODS: A convenience sample of 5560 aged care accreditation reports published from 2011 to 2018 was manually downloaded from the Accreditation Agency web site. A mixed-method approach of text data mining and manual content analysis was used to identify any significant differences in failure to meet accreditation outcomes among the RAC homes. This took account of whether EHR or paper records were used, year of accreditation, and size and location of the homes. RESULTS: It appears that aged care accreditation was focused on structure and process, with limited attention to outcome. There was a big variation between homes in their use of measurement indicators to assess accreditation outcomes. No difference was found in outcomes between RAC homes using EHR and those using paper records. Only 3% of the RAC homes were found to have failed some accreditation outcomes. Failure in monitoring mechanism was the key factor for failing many accreditation outcomes. The top five failed outcomes were Human Resource Management, Clinical Care, Information Systems, Medication Management and Behavioural Management. CONCLUSIONS: Sub-optimal outcomes have limited the effectiveness of accreditation in driving and monitoring risk management for care recipient safety in RAC homes. Although EHR is an important structure and process component for RAC services, it made a limited contribution to risk management for accreditation in Australian RAC homes. Either EHR was not effective, or the accreditation process was not robust enough to recognize its influence. Aged care accreditation in Australia needs to develop further outcome-based measures that are supported by robust data infrastructure and clear guidance.

Hospital-based Health Technology Assessment in Kazakhstan: 3 years' experience of one unit.

OBJECTIVES: The aim of this study was to describe the development and activities of the Hospital-Based Health Technology Assessment (HB-HTA) Unit in the Hospital of the President's Affairs Administration, one of the first examples of the implementation of HB-HTA into the practice of Kazakhstani hospitals. METHODS: Details of the development of the Unit were obtained from the hospital's administrative records. The Unit's own records were used to describe the reports prepared and the clinical areas that were covered. Responses to recommendations in the Unit's reports were obtained from hospital administration and individual departments. Estimates of savings and payback periods were based on data from the hospital information system, and data submitted by manufacturers and distributors of medical equipment. RESULTS: Fifty-one rapid- and mini-HTA reports were prepared by the Unit from 2015 to 2017. Seventeen health technologies (33 percent) were not recommended for implementation in hospital practice. Refusal to implement sixteen of these technologies saved approximately 1,053,500 USD. Of the thirty-four recommended health technologies, twenty-four were implemented to treat or diagnose 1,376 patients, and eight others were included in plans for 2018-20. Of the twenty-four implemented health technologies, twelve did not require additional investments. The payback period of investments for the other twelve implemented technologies is not more than 3 years for six, less than 5 years for four, and more than 10 years for two technologies. CONCLUSIONS: Establishment of the HB-HTA Unit in the hospital created the basis for making informed managerial decisions; identifying key directions for strategic development; and improving hospital management.

The Impact of Electronic Health Records on Risk Management of Information Systems in Australian Residential Aged Care Homes.

To obtain indications of the influence of electronic health records (EHR) in managing risks and meeting information system accreditation standard in Australian residential aged care (RAC) homes. The hypothesis to be tested is that the RAC homes using EHR have better performance in meeting information system standards in aged care accreditation than their counterparts only using paper records for information management. Content analysis of aged care accreditation reports from the Aged Care Standards and Accreditation Agency produced between April 2011 and December 2013. Items identified included types of information systems, compliance with accreditation standards, and indicators of failure to meet an expected outcome for information systems. The Chi-square test was used to identify difference between the RAC homes that used EHR systems and those that used paper records in not meeting aged care accreditation standards. 1,031 (37.4%) of 2,754 RAC homes had adopted EHR systems. Although the proportion of homes that met all accreditation standards was significantly higher for those with EHR than for homes with paper records, only 13 RAC homes did not meet one or more expected outcomes. 12 used paper records and nine of these failed the expected outcome for information systems. The overall contribution of EHR to meeting aged care accreditation standard in Australia was very small. Risk indicators for not meeting information system standard were no access to accurate and appropriate information, failure in monitoring mechanisms, not reporting clinical incidents, insufficient recording of residents' clinical changes, not providing accurate care plans, and communication processes failure. The study has provided indications that use of EHR provides small, yet significant advantages for RAC homes in Australia in managing risks for information management and in meeting accreditation requirements. The implication of the study for introducing technology innovation in RAC in Australia is discussed.

A PROCESS OF PRIORITIZING TOPICS FOR HEALTH TECHNOLOGY ASSESSMENT IN KAZAKHSTAN.

OBJECTIVES: The aim of this study was to develop criteria for the prioritization of topics for health technology assessment (HTA) in the healthcare system of Kazakhstan. METHODS: Initial proposals for criteria were suggested through consultation with Ministry of Health (MoH) policy areas. These were refined through a workshop attended by HTA department staff, persons from medical universities and research institutes, and MoH policy makers. The workshop included discussion on methods used in international HTA practice. Opinions of participants on selection of criteria from those specified in a review of prioritization processes were used to define a list for inclusion in an instrument for routine use. A scoring system was established in later discussion. RESULTS: Selected criteria for HTA prioritization were burden of disease, availability of alternative technology, clinical effectiveness, economic efficiency, budget impact, and ethical, legal, and/or psychosocial aspects. For each criterion, a health technology under consideration is given a score from 3 (High) to 1 (Low). The total score determines whether the technology is of high to medium priority or of low priority. Determination of priorities for assessment, using the instrument, should be carried out by an expert group appointed by the MoH. The process was applied in 2014 to a selection of topics, and three health technologies were chosen for full assessments. CONCLUSIONS: Criteria for prioritization have evolved with development of the HTA program in Kazakhstan. A method for HTA prioritization has been developed that is easy to apply, requires comparatively few resources, and is compatible with processes required by the MoH.

IMPLEMENTATION OF HEALTH TECHNOLOGY ASSESSMENT WORK IN A HOSPITAL IN KAZAKHSTAN.

OBJECTIVES: The aim of this study was to implement health technology assessment (HA) in the First General City Hospital in Astana, Kazakhstan. METHODS: We organized trainings to familiarize hospital staff with the purpose and details of HTA. An HTA committee was established, with representation from hospital physicians and managers, and criteria for prioritization of health technologies determined. Clinical departments of the hospital were asked to prepare applications for new technologies for their services. RESULTS: The HTA committee reviewed five applications and selected a technology from one of these, on single incision laparoscopic surgery (SILS), for assessment. A short HTA report on SILS was prepared, covering its safety, clinical effectiveness, and cost effectiveness. The report was used to support a request to the Department of Health for additional funding to implement this technology within the hospital. This funding was approved and SILS was established in several hospital departments. CONCLUSIONS: This successful initial experience with HTA has paved the way for its routine use by the hospital for informing decisions on the procurement and use of new health technologies.

INFLUENCE OF HEALTH TECHNOLOGY ASSESSMENT AND ITS MEASUREMENT.

OBJECTIVES: The aim of this study was to obtain information on methods used to measure health technology assessment (HTA) influence, decisions that were influenced, and outcomes linked to HTA. METHODS: Electronic databases were used to locate studies in which HTA influence had been demonstrated. Inclusion criteria were studies that reliably reported consideration by decision makers of HTA findings; comparative studies of technology use before and after HTA; and details of changes in policy, health outcomes, or research that could be credibly linked to an HTA. RESULTS: Fifty-one studies were selected for review. Settings were national (24), regional (12), both national and regional (3) hospitals (9), and multinational (3). The most common approach to appraisal of influence was review of policy or administrative decisions following HTA recommendations (51 percent). Eighteen studies (35 percent) reported interview or survey findings, thirteen (26 percent) reviewed administrative data, and six considered the influence of primary studies. Of 142 decisions informed by HTA, the most common types were on routine clinical practice (67 percent of studies), coverage (63 percent), and program operation (37 percent). The most frequent indications of HTA influence were on decisions related to resource allocation (59 percent), change in practice pattern (31 percent), and incorporation of HTA details in reference material (18 percent). Few publications assessed the contribution of HTA to changing patient outcomes. CONCLUSIONS: The literature on HTA influence remains limited, with little on longer term effects on practice and outcomes. The reviewed publications indicated how HTA is being used in different settings and approaches to measuring its influence that might be more widely applied, such as surveys and monitoring administrative data.

Rapid assessment of bilateral cochlear implantation for children in Kazakhstan.

OBJECTIVES: The aim of this study was to evaluate the effectiveness of bilateral cochlear implantation (CI) compared with unilateral CI for deaf children in the context of the Republic of Kazakhstan health system. Methods. A literature search was conducted, using the PubMed, Cochrane, and Embase data bases for studies that compared the effectiveness of bilateral and unilateral CI in children. The search included English language, publications from 2002-2012. Two reviewers independently evaluated all relevant studies. Administrative data relevant to CI in Kazakhstan were obtained from the Ministry of Health. RESULTS: Three relevant systematic reviews and an health technology assessment report were found. There was evidence of incremental benefits from bilateral CI but the quality of the available studies was poor and there was little information on longer term outcomes. No conclusions could be drawn regarding later incremental improvements to speech perception, learning, and quality of life. To date, in the Republic of Kazakhstan there is not full coverage of audiological screening due to the lack of medical equipment. This leads to late detection of hearing-impaired children and a long rehabilitation period, requiring more resources. Age of implantation in children is late and only a small minority attend general schools. CONCLUSIONS: The clinical effectiveness of bilateral CI, an expensive health technology, requires further study. Given the current situation in Kazakhstan with audiological screening and access to unilateral CI, there appeared to be other priorities for improving services for children with profound hearing impairment.

A review of data quality assessment methods for public health information systems.

High quality data and effective data quality assessment are required for accurately evaluating the impact of public health interventions and measuring public health outcomes. Data, data use, and data collection process, as the three dimensions of data quality, all need to be assessed for overall data quality assessment. We reviewed current data quality assessment methods. The relevant study was identified in major databases and well-known institutional websites. We found the dimension of data was most frequently assessed. Completeness, accuracy, and timeliness were the three most-used attributes among a total of 49 attributes of data quality. The major quantitative assessment methods were descriptive surveys and data audits, whereas the common qualitative assessment methods were interview and documentation review. The limitations of the reviewed studies included inattentiveness to data use and data collection process, inconsistency in the definition of attributes of data quality, failure to address data users' concerns and a lack of systematic procedures in data quality assessment. This review study is limited by the coverage of the databases and the breadth of public health information systems. Further research could develop consistent data quality definitions and attributes. More research efforts should be given to assess the quality of data use and the quality of data collection process.

Mentoring a health technology assessment initiative in Kazakhstan.

OBJECTIVES: The aim of this study was to assist in the development of a health technology assessment (HTA) program for the Ministry of Health (MOH) of the Republic of Kazakhstan METHODS: Mentoring of an initial HTA program in Kazakhstan was provided by the Canadian Society for International Health (CSIH) by means of a partnership with the Kazakhstan MOH. HTA materials, courses, and one-on-one support for the preparation of a series of initial HTA reports by MOH HTA staff were provided by a seven-member CSIH team over a 2.5-year project. RESULTS: Guidance documents on HTA and institutional strengthening were prepared in response to an extensive set of deliverables developed by the MOH and the World Bank. Introductory and train-the-trainer workshops in HTA and economic evaluation were provided for MOH staff members, experts from Kazakhstan research institutes and physicians. Five short HTA reports were successfully developed by staff in the Ministry's HTA Unit with assistance from the CSIH team. Challenges that may be relevant to other emerging HTA programs included lack of familiarity with some essential underlying concepts, organization culture, and limited time for MOH staff to do HTA work. CONCLUSIONS: The project helped to define the need for HTA and mentored MOH staff in taking the first steps to establish a program to support health policy decision making in Kazakhstan. This experience offers practical lessons for other emerging HTA programs, although these should be tailored to the specific context.

An exploration of the effects of introducing a telemonitoring system for continence assessment in a nursing home.

AIMS AND OBJECTIVES: To explore the effects of introducing a telemonitoring and care planning system for urinary continence assessment in a nursing home and adherence by care staff to urinary continence care plans. BACKGROUND: Only a few studies have explored the effect of introducing telemonitoring system on urinary continence care, none for older people in nursing homes. DESIGN: Pre- and postintervention repeated measures design. METHODS: Data for the study were collected from August-October 2011. Care staff were trained in the use of a telemonitoring system for continence assessment. Voiding events for each older person were recorded using the system during a 72-hour urinary continence assessment, and the data were used to prepare an individualised care plan. After two weeks of using the new care plan, a second assessment was carried out for each older person, using the telemonitoring system. RESULTS: The participants were on average 81 years old and assessed as having high care needs. The statistically significant outcomes were as follows: reduced volume of urine voided into continence aids, reduced number of prescribed toileting visits, increased number of actual toilet visits, increased number of successful toileting events and increased adherence to urinary continence care plans by staff. CONCLUSIONS: During a 12-week trial, urinary continence assessment and management of older people were improved. This suggests that the introduction of a suitably designed telemonitoring system combined with staff training can improve urinary continence care. RELEVANCE TO CLINICAL PRACTICE: The results suggest that nursing homes can improve continence assessment and management practices by adopting an appropriately designed mobile, wireless telemonitoring system for continence assessment and providing the associated staff training programmes.

Cost-effectiveness of telehealth in the management of chronic conditions.

Telehealth has the potential to improve the management of chronic health conditions but there are still limited data on its effectiveness and cost-effectiveness in routine practice. A questionnaire study nested within a pragmatic randomized controlled trial estimated costs and outcomes for management of patients in England who had heart failure, chronic obstructive pulmonary disease, or diabetes. Patients received telehealth support and usual care or usual care only. The study perspective was that of the health system. Incremental cost per quality-adjusted life year of telehealth when added to usual care was £79,000. In this setting, telehealth had a low probability of being a cost-effective addition to standard care for patients with the chronic conditions.

Description and comparison of documentation of nursing assessment between paper-based and electronic systems in Australian aged care homes.

PURPOSE: To describe nursing assessment documentation practices in aged care organizations and to evaluate the quality of electronic versus paper-based documentation of nursing assessment. METHODS: This was a retrospective nursing documentation audit study. Study samples were 2299 paper-based and 6997 electronic resident assessment forms contained in 159 paper-based and 249 electronic resident nursing records, respectively, from three aged care organizations. The practice of nursing assessment documentation in participating aged care homes was described. Three attributes of quality of nursing assessment documentation were evaluated: format and structure, process, and content by seven measures: quantity, completeness, timeliness comprehensiveness, frequencies of documentation specific to care domains and data items, and whether assessment forms were signed and dated. RESULTS: Varying practice in documentation of nursing assessment was found among different aged care organizations and homes. Electronic resident records contained higher numbers and more comprehensive resident assessment forms than paper-based records. The frequency of documentation was higher in electronic than in paper-based records in relation to most care domains. There was no difference between the two types of documentation systems on other aspects of nursing assessment documentation (overall completeness and timeliness, variation of frequencies among different care domains, and item completion in personal hygiene assessment forms). CONCLUSIONS: Electronic nursing documentation systems could improve the quality of documentation structure and format, process and content in the aspects of quantity, comprehensiveness and signing and dating of assessment forms. Further studies are needed to understand the factors leading to the variations of practice and the limitations of nursing assessment documentation and to evaluate documentation quality from a clinical perspective.

Involvement of consumers in health technology assessment activities by Inahta agencies.

OBJECTIVES: To obtain further information from members of the International Network of Agencies for Health Technology Assessment (INAHTA) on the involvement of consumers in their programs. METHODS: A questionnaire for a survey was developed and sent to member agencies in November 2010. Survey responses were compared with those from an earlier survey conducted in 2005. RESULTS: Of the thirty-three agencies that provided responses, 67 percent involve consumers in some aspects of their health technology assessment (HTA) programs, compared with 57 percent in 2005. As in the earlier survey, most agencies reporting involvement have contact with consumer or patient organizations and a large minority also involve individual consumers. Summaries of HTA reports that are intended to be easily understood by consumers are prepared by 84 percent of the agencies, and 42 percent involve consumers in dissemination of HTA material. In both areas, there was some increase from the levels previously reported. CONCLUSIONS: The survey results suggest that there is a trend to increased involvement of consumers by the INAHTA agencies in their programs but that the level of involvement remains relatively limited. The manner of consumer participation varies between agencies.

Estimating travel reduction associated with the use of telemedicine by patients and healthcare professionals: proposal for quantitative synthesis in a systematic review.

BACKGROUND: A major benefit offered by telemedicine is the avoidance of travel, by patients, their carers and health care professionals. Unfortunately, there is very little published information about the extent of avoided travel. We propose to undertake a systematic review of literature which reports credible data on the reductions in travel associated with the use of telemedicine. METHOD: The conventional approach to quantitative synthesis of the results from multiple studies is to conduct a meta analysis. However, too much heterogeneity exists between available studies to allow a meaningful meta analysis of the avoided travel when telemedicine is used across all possible settings. We propose instead to consider all credible evidence on avoided travel through telemedicine by fitting a linear model which takes into account the relevant factors in the circumstances of the studies performed. We propose the use of stepwise multiple regression to identify which factors are significant. DISCUSSION: Our proposed approach is illustrated by the example of teledermatology. In a preliminary review of the literature we found 20 studies in which the percentage of avoided travel through telemedicine could be inferred (a total of 5199 patients). The mean percentage avoided travel reported in the 12 store-and-forward studies was 43%. In the 7 real-time studies and in a single study with a hybrid technique, 70% of the patients avoided travel. A simplified model based on the modality of telemedicine employed (i.e. real-time or store and forward) explained 29% of the variance. The use of store and forward teledermatology alone was associated with 43% of avoided travel. The increase in the proportion of patients who avoided travel (25%) when real-time telemedicine was employed was significant (P = 0.014). Service planners can use this information to weigh up the costs and benefits of the two approaches.

A preliminary survey on the influence of rapid health technology assessments.

OBJECTIVES: The aim of this study was to obtain information on rapid health technology assessments (HTAs) prepared by members of the International Network of Agencies for Health Technology Assessment (INAHTA). METHODS: A questionnaire was prepared, drawing on earlier INAHTA documents for recording HTA impact. A request for responses was sent to member agencies, seeking information on rapid HTA reports prepared during 2006. RESULTS: Responses were provided on fifteen rapid HTAs, which covered both new and widely distributed technologies. The most common purpose for the HTAs (n = 8) was to inform coverage decisions, but other reasons included capital funding, formulary decisions, referral for treatment, program operation, guideline formulation, influence on routine practice, and indications for further research. All the rapid HTAs were considered by the agencies to have had some influence. The most common indications of influence were consideration by the decision maker, use of the HTA as reference material (both n = 10), and acceptance of recommendations or conclusions (n = 8). CONCLUSIONS: Rapid HTAs are used for a broad range of technologies, to inform several types of decision, and are effective in informing the decision-making process. Supplementation of their findings by further assessments will be appropriate in some cases.

Development of the International Network of Agencies for Health Technology Assessment.

OBJECTIVES: To describe the development of the International Network of Agencies for Health Technology Assessment (INAHTA) and its activities. METHODS: Review of literature material and other documents produced by or relating to INAHTA. RESULTS: INAHTA includes organizations that provide health technology assessment (HTA) advice to governments and receive most of their funding from public sources. In early 2009, there were forty-six members from twenty-seven countries, including both national and regional agencies. Interaction with other organizations includes links to Health Technology Assessment International (HTAi), the European Union Network for Health Technology Assessment (EUnetHTA), PAHO (the Pan American Health Organization, and the World Health Organization (WHO). An important feature of INAHTA is the routine interaction of its members with public sector decision makers. Output from the network has included guidelines and frameworks on HTA, reports of surveys, and joint projects. CONCLUSIONS: INAHTA has developed as a global point of contact and information for those with interests in HTA.

The history of health technology assessment in Australia.

OBJECTIVES: To describe the development and application of health technology assessment (HTA) in Australia. METHODS: Review of relevant literature and other documents related to HTA in Australia. RESULTS: Most HTA activity in Australia has been associated with provision of advice for the two national subsidy programs, Medicare, and the Pharmaceutical Benefits Scheme (PBS). National advisory bodies established by the federal government have had a prominent role. Assessments from the advisory bodies have had a major influence on decisions related to Medicare and the PBS, and in some other areas. Technologies without links to the national subsidy schemes, and those that are widely distributed, have been less well covered by HTA. To some extent these are addressed by evaluations supported by state governments, but details of approaches taken are not readily available. CONCLUSIONS: HTA in Australia now has a long history and is well established as a source of advice to health decision makers. Challenges remain in extending the scope of assessments, developing more transparent approaches in some areas, and consistently applying appropriate standards.

A randomized controlled trial of telephone-supported care coordination in patients with congestive heart failure.

An evaluation was undertaken on the effectiveness and efficiency of care coordination as a means of delivering health services to Australian veterans with a diagnosis of congestive heart failure. The veterans participated in a randomized controlled trial of care coordination that was supported by the Department of Veterans' Affairs (DVA). Of 490 veterans who were recruited, 409 were surveyed at baseline (214 in the intervention group and 195 controls). At follow-up, 288 were surveyed (155 intervention and 133 controls). Information on cost of care and quality of life (QOL) was collected before the commencement of coordinated care and at follow-up after 12 months. Cost of care data were obtained from DVA records. Information on QOL was obtained from telephone interviews, using the Short Form (SF-12) Health Survey and the EuroQol Group EQ-5D survey. There were no significant differences in costs of care between the intervention (coordinated care) and control groups of veterans. Nor were there significant differences between the intervention and control groups in QOL measurements with either of the evaluation tools that were used. Because evidence of benefit from coordinated care may be slow to emerge in patients with chronic disease, it would be desirable for future work in this area to include long term, good quality comparative studies on selected veteran populations. Such studies should measure QOL and economic outcomes in addition to clinical indicators.

Rapid versus full systematic reviews: validity in clinical practice?

INTRODUCTION: Rapid reviews are being produced with greater frequency by health technology assessment (HTA) agencies in response to increased pressure from end-user clinicians and policy-makers for rapid, evidence-based advice on health-care technologies. This comparative study examines the differences in methodologies and essential conclusions between rapid and full reviews on the same topic, with the aim of determining the validity of rapid reviews in the clinical context and making recommendations for their future application. METHODS: Rapid reviews were located by Internet searching of international HTA agency websites, with any ambiguities resolved by further communication with the agencies. Comparator full systematic reviews were identified using the University of York Centre for Reviews and Dissemination HTA database. Data on a number of review components were extracted using standardized data extraction tables, then analysed and reported narratively. RESULTS: Axiomatic differences between all the rapid and full reviews were identified; however, the essential conclusions of the rapid and full reviews did not differ extensively across the topics. For each of the four topics examined, it was clear that the scope of the rapid reviews was substantially narrower than that of full reviews. The methodology underpinning the rapid reviews was often inadequately described. CONCLUSIONS: Rapid reviews do not adhere to any single validated methodology. They frequently provide adequate advice on which to base clinical and policy decisions; however, their scope is limited, which may compromise their appropriateness for evaluating technologies in certain circumstances.