Ensuring best practice in genomics education: A theory- and empirically informed evaluation framework.

Implementation of genomic medicine into healthcare requires a workforce educated through effective educational approaches. However, ascertaining the impact of genomics education activities or resources is limited by a lack of evaluation and inconsistent descriptions in the literature. We aim to support those developing genomics education to consider how best to capture evaluation data that demonstrate program outcomes and effectiveness within scope. Here, we present an evaluation framework that is adaptable to multiple settings for use by genomics educators with or without education or evaluation backgrounds. The framework was developed as part of a broader program supporting genomic research translation coordinated by the Australian Genomics consortium. We detail our mixed-methods approach involving an expert workshop, literature review and iterative expert input to reach consensus and synthesis of a new evaluation framework for genomics education. The resulting theory-informed and evidence-based framework encompasses evaluation across all stages of education program development, implementation and reporting, and acknowledges the critical role of stakeholders and the effects of external influences.

A national education program for rapid genomics in pediatric acute care: Building workforce confidence, competence, and capability.

PURPOSE: To develop and evaluate a scalable national program to build confidence, competence and capability in the use of rapid genomic testing (rGT) in the acute pediatric setting. METHODS: We used theory-informed approaches to design a modular, adaptive program of blended learning aimed at diverse professional groups involved in acute pediatric care. The program comprised 4 online learning modules and an online workshop and was centered on case-based learning. We evaluated the program using the Kirkpatrick 4-level model of training evaluation and report our findings using the Reporting Item Standards for Education and its Evaluation (RISE2) guidelines for genomics education and evaluation. RESULTS: Two hundred and two participants engaged with at least 1 component of the program. Participants self-reported increased confidence in using rGT, (P < .001), and quiz responses objectively demonstrated increased competence (eg, correct responses to a question on pretest counseling increased from 30% to 64%; P < .001). Additionally, their capability in applying genomic principles to simulated clinical cases increased (P < .001), as did their desire to take on more responsibility for performing rGT. The clinical interpretation of more complex test results (such as negative results or variants of uncertain significance) appeared to be more challenging, indicating a need for targeted education in this area. CONCLUSION: The program format was effective in delivering multidisciplinary and wide-scale genomics education in the acute care context. The modular approach we have developed now lends itself to application in other medical specialties or areas of health care.

Is faster better? An economic evaluation of rapid and ultra-rapid genomic testing in critically ill infants and children.

PURPOSE: To evaluate whether the additional cost of providing increasingly faster genomic results in pediatric critical care is outweighed by reductions in health care costs and increases in personal utility. METHODS: Hospital costs and medical files from a cohort of 40 children were analyzed. The health economic impact of rapid and ultra-rapid genomic testing, with and without early initiation, relative to standard genomic testing was evaluated. RESULTS: Shortening the time to results led to substantial economic and personal benefits. Early initiation of ultra-rapid genomic testing was the most cost-beneficial strategy, leading to a cost saving of AU$26,600 per child tested relative to standard genomic testing and a welfare gain of AU$12,000 per child tested. Implementation of early ultra-rapid testing of critically ill children is expected to lead to an annual cost saving of AU$7.3 million for the Australian health system and an aggregate welfare gain of AU$3.3 million, corresponding to a total net benefit of AU$10.6 million. CONCLUSION: Early initiation of ultra-rapid genomic testing can offer substantial economic and personal benefits. Future implementation of rapid genomic testing programs should focus not only on optimizing the laboratory workflow to achieve a fast turnaround time but also on changing clinical practice to expedite test initiation.

Evaluating the resource implications of different service delivery models for offering additional genomic findings.

PURPOSE: To evaluate the resource implications of different delivery models for the provision of additional findings (AF) in genomics from a health-care purchaser perspective. METHODS: Data from the Additional Findings study were used to develop and validate a discrete event simulation model that represented the pathway of delivering AF. Resource implications were estimated by microcosting the consultations, sample verifications, bioinformatics, curation, and multidisciplinary case review meetings. A proof-of-concept model was used to generate costing, and then the simulation model was varied to assess the impact of an automated analysis pipeline, use of telehealth consultation, full automation with electronic decision support, and prioritizing case review for cases with pathogenic variants. RESULTS: For the proof-of-concept delivery model, the average total cost to report AF was US$430 per patient irrespective of result pathogenicity (95% confidence interval [CI] US$375-US$489). However, the cost of per AF diagnosis was US$4349 (95% CI US$3794-US$4953). Alternative approaches to genetic counseling (telehealth, decision support materials) and to multidisciplinary case review (pathogenic AF cases only) lowered the total per patient cost of AF analysis and reporting by 41-51%. CONCLUSION: Resources required to provide AF can be reduced substantially by implementing alternative approaches to counseling and multidisciplinary case review.

Clinical impact of genomic testing in patients with suspected monogenic kidney disease.

PURPOSE: To determine the diagnostic yield and clinical impact of exome sequencing (ES) in patients with suspected monogenic kidney disease. METHODS: We performed clinically accredited singleton ES in a prospectively ascertained cohort of 204 patients assessed in multidisciplinary renal genetics clinics at four tertiary hospitals in Melbourne, Australia. RESULTS: ES identified a molecular diagnosis in 80 (39%) patients, encompassing 35 distinct genetic disorders. Younger age at presentation was independently associated with an ES diagnosis (p < 0.001). Of those diagnosed, 31/80 (39%) had a change in their clinical diagnosis. ES diagnosis was considered to have contributed to management in 47/80 (59%), including negating the need for diagnostic renal biopsy in 10/80 (13%), changing surveillance in 35/80 (44%), and changing the treatment plan in 16/80 (20%). In cases with no change to management in the proband, the ES result had implications for the management of family members in 26/33 (79%). Cascade testing was subsequently offered to 40/80 families (50%). CONCLUSION: In this pragmatic pediatric and adult cohort with suspected monogenic kidney disease, ES had high diagnostic and clinical utility. Our findings, including predictors of positive diagnosis, can be used to guide clinical practice and health service design.

Ensuring best practice in genomics education and evaluation: reporting item standards for education and its evaluation in genomics (RISE2 Genomics).

PURPOSE: Widespread, quality genomics education for health professionals is required to create a competent genomic workforce. A lack of standards for reporting genomics education and evaluation limits the evidence base for replication and comparison. We therefore undertook a consensus process to develop a recommended minimum set of information to support consistent reporting of design, development, delivery, and evaluation of genomics education interventions. METHODS: Draft standards were derived from literature (25 items from 21 publications). Thirty-six international experts were purposively recruited for three rounds of a modified Delphi process to reach consensus on relevance, clarity, comprehensiveness, utility, and design. RESULTS: The final standards include 18 items relating to development and delivery of genomics education interventions, 12 relating to evaluation, and 1 on stakeholder engagement. CONCLUSION: These Reporting Item Standards for Education and its Evaluation in Genomics (RISE2 Genomics) are intended to be widely applicable across settings and health professions. Their use by those involved in reporting genomics education interventions and evaluation, as well as adoption by journals and policy makers as the expected standard, will support greater transparency, consistency, and comprehensiveness of reporting. Consequently, the genomics education evidence base will be more robust, enabling high-quality education and evaluation across diverse settings.

Utility of clinical comprehensive genomic characterization for diagnostic categorization in patients presenting with hypocellular bone marrow failure syndromes.

Bone marrow failure (BMF) related to hypoplasia of hematopoietic elements in the bone marrow is a heterogeneous clinical entity with a broad differential diagnosis including both inherited and acquired causes. Accurate diagnostic categorization is critical to optimal patient care and detection of genomic variants in these patients may provide this important diagnostic and prognostic information. We performed real-time, accredited (ISO15189) comprehensive genomic characterization including targeted sequencing and whole exome sequencing in 115 patients with BMF syndrome (median age 24 years, range 3 months - 81 years). In patients with clinical diagnoses of inherited BMF syndromes, acquired BMF syndromes or clinically unclassifiable BMF we detected variants in 52% (12/23), 53% (25/47) and 56% (25/45) respectively. Genomic characterization resulted in a change of diagnosis in 30/115 (26%) including the identification of germline causes for 3/47 and 16/45 cases with pre-test diagnoses of acquired and clinically unclassifiable BMF respectively. The observed clinical impact of accurate diagnostic categorization included choice to perform allogeneic stem cell transplantation, disease-specific targeted treatments, identification of at-risk family members and influence of sibling allogeneic stem cell donor choice. Multiple novel pathogenic variants and copy number changes were identified in our cohort including in TERT, FANCA, RPS7 and SAMD9. Whole exome sequence analysis facilitated the identification of variants in two genes not typically associated with a primary clinical manifestation of BMF but also demonstrated reduced sensitivity for detecting low level acquired variants. In conclusion, genomic characterization can improve diagnostic categorization of patients presenting with hypoplastic BMF syndromes and should be routinely performed in this group of patients.

Exome sequencing in newborns with congenital deafness as a model for genomic newborn screening: the Baby Beyond Hearing project.

PURPOSE: Genomic newborn screening raises practical and ethical issues. Evidence is required to build a framework to introduce this technology safely and effectively. We investigated the choices made by a diverse group of parents with newborns when offered tiered genomic information from exome sequencing. METHODS: This population-derived cohort comprised infants with congenital deafness. Parents were offered exome sequencing and choice regarding the scope of analysis. Options were choice A, diagnostic analysis only; choice B, diagnostic analysis plus childhood-onset diseases with medical actionability; or choice C, diagnostic analysis plus childhood-onset diseases with or without medical actionability. RESULTS: Of the 106 participants, 72 (68%) consented to receive additional findings with 29 (27.4%) selecting choice B and 43 (40.6%) opting for choice C. Family size, ethnicity, and age of infant at time of recruitment were the significant predictors of choice. Parents who opted to have additional findings analysis demonstrated less anxiety and decisional conflict. CONCLUSIONS: These data provide evidence from a culturally diverse population that choice around additional findings is important and the age of the infant when this choice is offered impacts on their decision. We found no evidence that offering different levels of genomic information to parents of newborns has a negative psychological impact.

Parental experiences of ultrarapid genomic testing for their critically unwell infants and children.

PURPOSE: To explore parental experiences of ultrarapid genomic testing for their critically unwell infants and children. METHODS: Parents of critically unwell children who participated in a national ultrarapid genomic diagnosis program were surveyed >12 weeks after genomic results return. Surveys consisted of custom questions and validated scales, including the Decision Regret Scale and Genomics Outcome Scale. RESULTS: With 96 survey invitations sent, the response rate was 57% (n = 55). Most parents reported receiving enough information during pretest (n = 50, 94%) and post-test (n = 44, 83%) counseling. Perceptions varied regarding benefits of testing, however most parents reported no or mild decision regret (n = 45, 82%). The majority of parents (31/52, 60%) were extremely concerned about the condition recurring in future children, regardless of actual or perceived recurrence risk. Parents whose child received a diagnostic result reported higher empowerment. CONCLUSION: This study provides valuable insight into parental experiences of ultrarapid genomic testing in critically unwell children, including decision regret, empowerment, and post-test reproductive planning, to inform design and delivery of rapid diagnosis programs. The findings suggest considerations for pre- and post-test counseling that may influence parental experiences during the testing process and beyond, such as the importance of realistically conveying the likelihood for clinical and/or personal utility.

A cost-effectiveness analysis of genomic sequencing in a prospective versus historical cohort of complex pediatric patients.

PURPOSE: Cost-effectiveness evaluations of first-line genomic sequencing (GS) in the diagnosis of children with genetic conditions are limited by the lack of well-defined comparative cohorts. We sought to evaluate the cost-effectiveness of early GS in pediatric patients with complex monogenic conditions compared with a matched historical cohort. METHODS: Data, including investigation costs, were collected in a prospective cohort of 92 pediatric patients undergoing singleton GS over an 18-month period (2016-2017) with two of the following: a condition with high mortality, multisystem disease involving three or more organs, or severe limitation of daily function. Comparative data were collected in a matched historical cohort who underwent traditional investigations in the years 2012-2013. RESULTS: GS yielded a diagnosis in 42% while traditional investigations yielded a diagnosis in 23% (p = 0.003). A change in management was experienced by 74% of patients diagnosed following GS, compared with 32% diagnosed following traditional investigations. Singleton GS at a cost of AU$3100 resulted in a mean saving per person of AU$3602 (95% confidence interval [CI] AU$2520-4685). Cost savings occurred across all investigation subtypes and were only minimally offset by clinical management costs. CONCLUSION: GS in complex pediatric patients saves significant costs and doubles the diagnostic yield of traditional approaches.

Mapping the Minnesota Living with Heart Failure Questionnaire (MLHFQ) onto the Assessment of Quality of Life 8D (AQoL-8D) utility scores.

PURPOSE: The Minnesota Living with Heart Failure Questionnaire (MLHFQ) is a widely used condition-specific measure of quality of life (QoL) in patients with heart failure. To use information from the MLHFQ in an economic evaluation, the MLHFQ must be mapped onto a preference-based measure of QoL. This study aims to develop a mapping algorithm between the MLHFQ and the Assessment of Quality of Life (AQoL) 8D utility instrument in patients with dilated cardiomyopathy (DCM). METHODS: MLHFQ and AQoL-8D data were collected on 61 Australian adults with idiopathic DCM or other non-hypertrophic cardiomyopathies. Three statistical methods were used as follows: ordinary least squares (OLS) regression, the robust MM estimator, and the generalised linear models (GLM). Each included a range of explanatory variables. Model performance was assessed using key goodness-of-fit measures, the mean absolute error (MAE), and the root-mean-square error (RMSE). RESULTS: The MLHFQ summary score and AQoL-8D utility scores were strongly correlated (r = - 0.83, p < 0.0001) and the two subscales of the MLHFQ were correlated with the eight dimensions of the AQoL-8D. Utility scores were predicted with acceptable precision based on responses to the MLHFQ physical, emotional, social, and other subscales. OLS and GLM performed similarly with MAE and RMSE ranging 0.086-0.106 and 0.114-0.130, respectively. CONCLUSION: The mapping algorithm developed in this study allows the derivation of AQoL-8D utilities from MLHFQ scores for use in cost-effectiveness analyses and most importantly, enables the economic evaluation of alternative heart failure therapy options when only the MLHFQ has been collected.

A cost-effectiveness model of genetic testing and periodical clinical screening for the evaluation of families with dilated cardiomyopathy.

PURPOSE: To assess the relative cost-effectiveness of cascade genetic testing in asymptomatic relatives of patients with dilated cardiomyopathy (DCM) compared with periodical clinical surveillance. METHODS: A decision-analytic model, combining a decision tree and a Markov model, was used to determine the lifetime costs and quality-adjusted life years (QALYs) for the two strategies. Deterministic and probabilistic sensitivity analyses were undertaken to assess the robustness of findings and to explore decision uncertainty. RESULTS: The incremental cost per additional QALY of cascade genetic testing prior to periodical clinical surveillance of first-degree relatives compared with periodical clinical surveillance alone was estimated at approximately AUD $6100. At established thresholds of cost-effectiveness, there is a 90% probability that cascade genetic testing is cost-effective. Extensive sensitivity analyses, including the addition of second-degree relatives, did not alter the conclusions drawn from the main analysis. CONCLUSION: Using cascade genetic testing to guide clinical surveillance of asymptomatic relatives of patients with DCM is very likely to be cost-effective. As the DCM pathogenic variant detection rate rises and new evidence for personalized treatment of at-risk individuals becomes available, the cost-effectiveness of cascade testing will further increase.

Does genomic sequencing early in the diagnostic trajectory make a difference? A follow-up study of clinical outcomes and cost-effectiveness.

PURPOSE: To systematically investigate the longer-term clinical and health economic impacts of genomic sequencing for rare-disease diagnoses. METHODS: We collected information on continuing diagnostic investigation, changes in management, cascade testing, and parental reproductive outcomes in 80 infants who underwent singleton whole-exome sequencing (WES). RESULTS: The median duration of follow-up following result disclosure was 473 days. Changes in clinical management due to diagnostic WES results led to a cost saving of AU$1,578 per quality-adjusted life year gained, without increased hospital service use. Uninformative WES results contributed to the diagnosis of non-Mendelian conditions in seven infants. Further usual diagnostic investigations in those with ongoing suspicion of a genetic condition yielded no new diagnoses, while WES data reanalysis yielded four. Reanalysis at 18 months was more cost-effective than every 6 months. The parents of diagnosed children had eight more ongoing pregnancies than those without a diagnosis. Taking the costs and benefits of cascade testing and reproductive service use into account, there was an additional cost of AU$8,118 per quality-adjusted life year gained due to genomic sequencing. CONCLUSION: These data strengthen the case for the early use of genomic testing in the diagnostic trajectory, and can guide laboratory policy on periodic WES data reanalysis.

Correction: Does genomic sequencing early in the diagnostic trajectory make a difference? A follow-up study of clinical outcomes and cost-effectiveness.

The original PDF version of this Article omitted to list Clara L Gaff as a corresponding author and the affiliations were incorrectly labelled as Present Addresses. Furthermore, Tables 1 and 2 have been updated to clarify that the Australian dollar is used for the values. These errors have now been corrected in the PDF and HTML versions of the Article.

Diagnostic and service impact of genomic testing technologies in a neonatal intensive care unit.

AIM: To investigate the diagnostic and service impact of chromosomal microarray and whole exome sequencing (WES) in a neonatal intensive care unit (NICU). METHODS: This was a retrospective medical record review of NICU patients referred for genetics consultation at three time points over a 9-year period at a single centre to determine referral indications, genetic consultation outcomes and time to diagnosis. RESULTS: The number of NICU patients referred for genetics consultation increased from 44 in 2007 to 95 in 2015. The proportion of NICU patients suspected of having a genetic condition following clinical geneticist assessment remained stable, averaging 5.3% of all admissions. The proportion of patients receiving a confirmed diagnosis rose from 21% in 2007 to 53% in 2015, with a shift from primarily chromosomal abnormalities to a broad range of monogenic disorders, increasingly diagnosed by WES as a first-tier test. The average age at diagnosis in 2015 was 19 days (range 12-38 days) for chromosomal abnormalities and 138 days (range 10-309 days) for monogenic conditions. CONCLUSIONS: The adoption of new genetic technologies at our centre has increased the proportion of patients receiving a confirmed genetic diagnosis. This study provides important benchmark data to measure further improvements as turn-around times for genomic testing decrease.

A novel approach to offering additional genomic findings-A protocol to test a two-step approach in the healthcare system.

Internationally, the practice of offering additional findings (AFs) when undertaking a clinically indicated genomic test differs. In the USA, the recommendation is to include analysis for AFs alongside diagnostic analysis, unless a patient opts-out, whereas European and Canadian guidelines recommend opt-in models. These guidelines all consider the offer of AFs as an activity concurrent with the offer of diagnostic testing. This paper describes a novel two-step model for managing AFs within the healthcare system in Victoria, Australia and presents the study protocol for its evaluation. Adults who have received results of diagnostic whole exome sequencing undertaken within the healthcare system are invited to attend a genetic counseling appointment to consider reanalysis of their stored genomic data for AFs. The evaluation protocol addresses uptake, decision-making, understanding, counseling challenges, and explores preferences for future models of care. Recruitment commenced in November 2017 and will cease when 200 participants have been approached. When the study is concluded, the evaluation results will contribute to the evidence base guiding approaches to counseling and models of care for AFs.

Meeting the challenges of implementing rapid genomic testing in acute pediatric care.

PURPOSE: The purpose of the study was to implement and prospectively evaluate the outcomes of a rapid genomic diagnosis program at two pediatric tertiary centers. METHODS: Rapid singleton whole-exome sequencing (rWES) was performed in acutely unwell pediatric patients with suspected monogenic disorders. Laboratory and clinical barriers to implementation were addressed through continuous multidisciplinary review of process parameters. Diagnostic and clinical utility and cost-effectiveness of rWES were assessed. RESULTS: Of 40 enrolled patients, 21 (52.5%) received a diagnosis, with median time to report of 16 days (range 9-109 days). A result was provided during the first hospital admission in 28 of 36 inpatients (78%). Clinical management changed in 12 of the 21 diagnosed patients (57%), including the provision of lifesaving treatment, avoidance of invasive biopsies, and palliative care guidance. The cost per diagnosis was AU$13,388 (US$10,453). Additional cost savings from avoidance of planned tests and procedures and reduced length of stay are estimated to be around AU$543,178 (US$424,101). The clear relative advantage of rWES, joint clinical and laboratory leadership, and the creation of a multidisciplinary "rapid team" were key to successful implementation. CONCLUSION: Rapid genomic testing in acute pediatrics is not only feasible but also cost-effective, and has high diagnostic and clinical utility. It requires a whole-of-system approach for successful implementation.

Informed decision making and psychosocial outcomes in pregnant and nonpregnant women offered population fragile X carrier screening.

PurposePopulation-based carrier screening for fragile X syndrome (FXS) is still not universally endorsed by professional organizations due to concerns around genetic counseling for complex information and potential for psychosocial harms.MethodsWe determined uptake levels, decision making, and psychosocial impact in a prospective study of pregnant and nonpregnant Australian women offered FXS carrier screening in clinical settings. Women received pretest genetic counseling, and completed questionnaires when deciding and one month later.ResultsOf 1,156 women recruited, 83.1% returned the first questionnaire with 70.6% nonpregnant and 58.8% pregnant women choosing testing (χ2=16.98, P<0.001). Overall, informed choice was high in both nonpregnant (77.4%) and pregnant (72.9%) women (χ2=0.21, P=0.644), and more tested (76.0%) than not-tested (66.7%) women (χ2=6.35, P=0.012) made an informed choice. Measures of depression, stress, and anxiety were similar to population norms for ~85% of women. Decisional conflict and regret were generally low; however, decisional uncertainty and regret were greater in pregnant than nonpregnant women, and not-tested than tested women (uncertainty: χ2=18.51, P<0.001 and χ2=43.11, P<0.001, respectively; regret: χ2=6.61, P<0.037 and χ2=35.54, P<0.001, respectively).ConclusionWe provide evidence to inform guidelines that population FXS carrier screening can be implemented with minimal psychosocial harms following appropriate information and prescreening genetic counseling.

Clinical audit of genetic testing and referral patterns for fragile X and associated conditions.

An audit was conducted of laboratory/clinical databases of genetic tests performed between January 2003 and December 2009, and for 2014, as well as referrals to the clinical service and a specialist multidisciplinary clinic, to determine genetic testing request patterns for fragile X syndrome and associated conditions and referrals for genetic counseling/multidisciplinary management in Victoria, Australia. An expanded allele (full mutation, premutation or intermediate) was found in 3.7% of tests. Pediatricians requested ∼70% of test samples, although fewer general practitioners and more obstetricians/gynecologists ordered tests in 2014. Median age at testing for individuals with a full mutation seeking a diagnosis without a fragile X family history was 4.3 years (males) and 9.4 years (females); these ages were lower when pediatricians ordered the tests (2.1 years and 6.1 years, respectively). Individuals with a premutation were generally tested at a later age (median age: males, 33.2 years; females, 36.4 years). Logistic regression showed that a family history of ID (OR 3.28 P = 0.005, CI 1.77-5.98) was the only indication to independently increase the likelihood of a test-positive (FM or PM) result. Following testing, ∼25% of full mutation or premutation individuals may not have attended clinical services providing genetic counseling or multidisciplinary management for these families. The apparent delay in fragile X syndrome diagnosis and lack of appropriate referrals for some may result in less than optimal management for these families. These findings suggest continued need for awareness and education of health professionals around diagnosis and familial implications of fragile X syndrome and associated conditions. © 2016 Wiley Periodicals, Inc.