Understanding the landscape of web-based medical misinformation about vaccination.

Given the high rates of vaccine hesitancy, web-based medical misinformation about vaccination is a serious issue. We sought to understand the nature of Google searches leading to medical misinformation about vaccination, and guided by fuzzy-trace theory, the characteristics of misinformation pages related to comprehension, inference-making, and medical decision-making. We collected data from web pages presenting vaccination information. We assessed whether web pages presented medical misinformation, had an overarching gist, used narrative, and employed emotional appeals. We used Search Engine Optimization tools to determine the number of backlinks from other web pages, monthly Google traffic, and Google Keywords. We used Coh-Metrix to measure readability and Gist Inference Scores (GIS). For medical misinformation web pages, Google traffic and backlinks were heavily skewed with means of 138.8 visitors/month and 805 backlinks per page. Medical misinformation pages were significantly more likely than other vaccine pages to have backlinks from other pages, and significantly less likely to receive at least one visitor from Google searches per month. The top Google searches leading to medical misinformation were "the truth about vaccinations," "dangers of vaccination," and "pro con vaccines." Most frequently, pages challenged vaccine safety, with 32.7% having an overarching gist, 7.7% presenting narratives, and 17.3% making emotional appeals. Emotional appeals were significantly more common with medical misinformation than other high-traffic vaccination pages. Misinformation pages had a mean readability grade level of 11.5, and a mean GIS of - 0.234. Low GIS scores are a likely barrier to understanding gist, and are the "Achilles' heel" of misinformation pages.

Gist Inference Scores Predict Cloze Comprehension "In Your Own Words" for Native, Not ESL Readers.

Three patient education texts from the National Cancer Institute (NCI) were subjected to a Coh-Metrix analysis, then further analyzed to obtain Gist Inference Scores (GIS), a new measure of the likelihood that readers will make appropriate inferences about a text's bottom-line meaning. In the GIS formula, the Coh-Metrix psycholinguistic variables referential cohesion, deep cohesion, and latent semantic analysis (LSA) verb overlap increase GIS because cohesive texts that describe related actions are likely to induce gist representations. The Coh-Metrix variables word concreteness, imagability for content words, and hypernymy for nouns and verbs are negatively weighted because they tend to promote verbatim mental representations. NCI texts were modified for a cloze procedure with every tenth word replaced by a blank starting with the second sentence. Participants in two studies received all three cloze-modified texts. Fuzzy-Trace Theory suggests that people are more likely to comprehend high GIS texts "in their own words," and thus fill-the-blanks with multiple words that differ from those omitted by the cloze procedure expressing comparable meaning. In Study One, non-native English speakers appropriately filled blanks with different words at the same rate for all three texts of low-, medium-, and high-GIS. In Study Two, replicating previous findings, for high GIS texts, native English speakers filled blanks appropriately with words other than those removed significantly more often than for medium- or low-GIS texts. High GIS texts apparently afford readers more semantic and lexical flexibility, but non-native English speakers may be ill-equipped to capitalize on this characteristic of high GIS texts.

A theoretically motivated method for automatically evaluating texts for gist inferences.

We developed a method to automatically assess texts for features that help readers produce gist inferences. Following fuzzy-trace theory, we used a procedure in which participants recalled events under gist or verbatim instructions. Applying Coh-Metrix, we analyzed written responses in order to create gist inference scores (GISs), or seven variables converted to Z scores and averaged, which assess the potential for readers to form gist inferences from observable text characteristics. Coh-Metrix measures reflect referential cohesion and deep cohesion, which increase GIS because they facilitate coherent mental representations. Conversely, word concreteness, hypernymy for nouns and verbs (specificity), and imageability decrease GIS, because they promote verbatim representations. Also, the difference between abstract verb overlap among sentences (using latent semantic analysis) and more concrete verb overlap (using WordNet) should enhance coherent gist inferences, rather than verbatim memory for specific verbs. In the first study, gist condition responses scored nearly two standard deviations higher on GIS than did the verbatim condition responses. Predictions based on GIS were confirmed in two text analysis studies of 50 scientific journal article texts and 50 news articles and editorials. Texts from the Discussion sections of psychology journal articles scored significantly higher on GIS than did texts from the Method sections of the same journal articles. News reports also scored significantly lower than editorials on the same topics from the same news outlets. GIS proved better at discriminating among texts than did alternative formulae. In a behavioral experiment with closely matched text pairs, people randomly assigned to high-GIS versions scored significantly higher on knowledge and comprehension.

Twenty years of Internet-based research at SCiP: A discussion of surviving concepts and new methodologies.

This discussion of the symposium 20 Years of Internet-Based Research at SCiP: Surviving Concepts, New Methodologies compares the issues faced by the pioneering Internet-based psychology researchers who presented at the first symposia on the topic, at the 1996 annual meeting of the Society for Computers in Psychology, to the issues facing researchers today. New methodologies unavailable in the early days of Web-based psychological research are discussed, with an emphasis on mobile computing with smartphones that is capitalizing on capabilities such as touch screens and gyro sensors. A persistent issue spanning the decades has been the challenge of conducting scientific research with consumer-grade electronics. In the 1996 symposia on Internet-based research, four advantages were identified: easy access to a geographically unlimited subject population, including subjects from very specific and previously inaccessible target populations; bringing the experiment to the subject; high statistical power through large sample size; and reduced cost. In retrospect, it appears that Internet-based research has largely lived up to this early promise-with the possible exception of sample size, since the public demand for controlled psychology experiments has not always been greater than the supply offered by researchers. There are many reasons for optimism about the future of Internet-based research. However, unless courses and textbooks on psychological research methods begin to give Web-based research the attention it deserves, the future of Internet-based psychological research will remain in doubt.

Active engagement in a web-based tutorial to prevent obesity grounded in Fuzzy-Trace Theory predicts higher knowledge and gist comprehension.

We used Sharable Knowledge Objects (SKOs) to create an Intelligent Tutoring System (ITS) grounded in Fuzzy-Trace Theory to teach women about obesity prevention: GistFit, getting the gist of healthy eating and exercise. The theory predicts that reliance on gist mental representations (as opposed to verbatim) is more effective in reducing health risks and improving decision making. Technical information was translated into decision-relevant gist representations and gist principles (i.e., healthy values). The SKO was hypothesized to facilitate extracting these gist representations and principles by engaging women in dialogue, "understanding" their responses, and replying appropriately to prompt additional engagement. Participants were randomly assigned to either the obesity prevention tutorial (GistFit) or a control tutorial containing different content using the same technology. Participants were administered assessments of knowledge about nutrition and exercise, gist comprehension, gist principles, behavioral intentions and self-reported behavior. An analysis of engagement in tutorial dialogues and responses to multiple-choice questions to check understanding throughout the tutorial revealed significant correlations between these conversations and scores on subsequent knowledge tests and gist comprehension. Knowledge and comprehension measures correlated with healthier behavior and greater intentions to perform healthy behavior. Differences between GistFit and control tutorials were greater for participants who engaged more fully. Thus, results are consistent with the hypothesis that active engagement with a new gist-based ITS, rather than a passive memorization of verbatim details, was associated with an array of known psychosocial mediators of preventive health decisions, such as knowledge acquisition, and gist comprehension.

The effectiveness of argumentation in tutorial dialogues with an Intelligent Tutoring System for genetic risk of breast cancer.

BRCA Gist is an Intelligent Tutoring System that helps women understand issues related to genetic testing and breast cancer risk. In two laboratory experiments and a field experiment with community and web-based samples, an avatar asked 120 participants to produce arguments for and against genetic testing for breast cancer risk. Two raters assessed the number of argumentation elements (claim, reason, backing, etc.) found in response to prompts soliciting arguments for and against genetic testing for breast cancer risk (IRR=.85). When asked to argue for genetic testing, 53.3 % failed to meet the minimum operational definition of making an argument, a claim supported by one or more reasons. When asked to argue against genetic testing, 59.3 % failed to do so. Of those who failed to generate arguments most simply listed disconnected reasons. However, participants who provided arguments against testing (40.7 %) performed significantly higher on a posttest of declarative knowledge. In each study we found positive correlations between the quality of arguments against genetic testing (i.e., number of argumentation elements) and genetic risk categorization scores. Although most interactions did not contain two or more argument elements, when more elements of arguments were included in the argument against genetic testing interaction, participants had greater learning outcomes. Apparently, many participants lack skills in making coherent arguments. These results suggest an association between argumentation ability (knowing how to make complex arguments) and subsequent learning. Better education in developing arguments may be necessary for people to learn from generating arguments within Intelligent Tutoring Systems and other settings.

Understanding Genetic Breast Cancer Risk: Processing Loci of the BRCA Gist Intelligent Tutoring System.

The BRCA Gist Intelligent Tutoring System helps women understand and make decisions about genetic testing for breast cancer risk. BRCA Gist is guided by Fuzzy-Trace Theory, (FTT) and built using AutoTutor Lite. It responds differently to participants depending on what they say. Seven tutorial dialogues requiring explanation and argumentation are guided by three FTT concepts: forming gist explanations in one's own words, emphasizing decision-relevant information, and deliberating the consequences of decision alternatives. Participants were randomly assigned to BRCA Gist, a control, or impoverished BRCA Gist conditions removing gist explanation dialogues, argumentation dialogues, or FTT images. All BRCA Gist conditions performed significantly better than controls on knowledge, comprehension, and risk assessment. Significant differences in knowledge, comprehension, and fine-grained dialogue analyses demonstrate the efficacy of gist explanation dialogues. FTT images significantly increased knowledge. Providing more elements in arguments against testing correlated with increased knowledge and comprehension.

Tutorial dialogues and gist explanations of genetic breast cancer risk.

The intelligent tutoring system (ITS) BRCA Gist is a Web-based tutor developed using the Shareable Knowledge Objects (SKO) platform that uses latent semantic analysis to engage women in natural-language dialogues to teach about breast cancer risk. BRCA Gist appears to be the first ITS designed to assist patients' health decision making. Two studies provide fine-grained analyses of the verbal interactions between BRCA Gist and women responding to five questions pertaining to breast cancer and genetic risk. We examined how "gist explanations" generated by participants during natural-language dialogues related to outcomes. Using reliable rubrics, scripts of the participants' verbal interactions with BRCA Gist were rated for content and for the appropriateness of the tutor's responses. Human researchers' scores for the content covered by the participants were strongly correlated with the coverage scores generated by BRCA Gist, indicating that BRCA Gist accurately assesses the extent to which people respond appropriately. In Study 1, participants' performance during the dialogues was consistently associated with learning outcomes about breast cancer risk. Study 2 was a field study with a more diverse population. Participants with an undergraduate degree or less education who were randomly assigned to BRCA Gist scored higher on tests of knowledge than those assigned to the National Cancer Institute website or than a control group. We replicated findings that the more expected content that participants included in their gist explanations, the better they performed on outcome measures. As fuzzy-trace theory suggests, encouraging people to develop and elaborate upon gist explanations appears to improve learning, comprehension, and decision making.

A signal detection analysis of gist-based discrimination of genetic breast cancer risk.

Pervasive biases in probability judgment render the probability scale a poor response mode for assessing risk judgments. By applying fuzzy trace theory, we used ordinal gist categories as a response mode, coupled with a signal detection model to assess risk judgments. The signal detection model is an extension of the familiar model used in binary choice paradigms. It provides three measures of discriminability-low versus medium risk, medium versus high risk, and low versus high risk-and two measures of response bias. We used the model to assess the effectiveness of BRCA Gist, an intelligent tutoring system designed to improve women's judgments and understanding of genetic risk for breast cancer. Participants were randomly assigned to the BRCA Gist intelligent tutoring system, the National Cancer Institute (NCI) Web pages, or a control group, and then they rated cases that had been developed using the Pedigree Assessment Tool and also vetted by medical experts. BRCA Gist participants demonstrated increased discriminability for all three risk categories, relative to the control group; the NCI group showed increased discriminability for two of the three levels. This result suggests that BRCA Gist best improved discriminability among genetic risk categories, and both BRCA Gist and the NCI website improved participants' ability to discriminate, rather than simply shifting their decision criterion. A spreadsheet that fits the model and compares parameters across the conditions can be downloaded from the Behavior Research Methods website and used in any research involving categorical responses.

The development and analysis of tutorial dialogues in AutoTutor Lite.

The goal of intelligent tutoring systems (ITS) that interact in natural language is to emulate the benefits that a well-trained human tutor provides to students, by interpreting student answers and appropriately responding in order to encourage elaboration. BRCA Gist is an ITS developed using AutoTutor Lite, a Web-based version of AutoTutor. Fuzzy-trace theory theoretically motivated the development of BRCA Gist, which engages people in tutorial dialogues to teach them about genetic breast cancer risk. We describe an empirical method to create tutorial dialogues and fine-tune the calibration of BRCA Gist's semantic processing engine without a team of computer scientists. We created five interactive dialogues centered on pedagogic questions such as "What should someone do if she receives a positive result for genetic risk of breast cancer?" This method involved an iterative refinement process of repeated testing with different texts and successively making adjustments to the tutor's expectations and settings in order to improve performance. The goal of this method was to enable BRCA Gist to interpret and respond to answers in a manner that best facilitated learning. We developed a method to analyze the efficacy of the tutor's dialogues. We found that BRCA Gist's assessment of participants' answers was highly correlated with the quality of the answers found by trained human judges using a reliable rubric. The dialogue quality between users and BRCA Gist predicted performance on a breast cancer risk knowledge test completed after exposure to the tutor. The appropriateness of BRCA Gist's feedback also predicted the quality of answers and breast cancer risk knowledge test scores.

Individual Differences in Base Rate Neglect: A Fuzzy Processing Preference Index.

Little is known about individual differences in integrating numeric base-rates and qualitative text in making probability judgments. Fuzzy-Trace Theory predicts a preference for fuzzy processing. We conducted six studies to develop the FPPI, a reliable and valid instrument assessing individual differences in this fuzzy processing preference. It consists of 19 probability estimation items plus 4 "M-Scale" items that distinguish simple pattern matching from "base rate respect." Cronbach's Alpha was consistently above 0.90. Validity is suggested by significant correlations between FPPI scores and three other measurers: "Rule Based" Process Dissociation Procedure scores; the number of conjunction fallacies in joint probability estimation; and logic index scores on syllogistic reasoning. Replicating norms collected in a university study with a web-based study produced negligible differences in FPPI scores, indicating robustness. The predicted relationships between individual differences in base rate respect and both conjunction fallacies and syllogistic reasoning were partially replicated in two web-based studies.

Facial Appearance as Core Expression Scale: Benchmarks and Properties.

Objective: Patients presenting for corrective facial surgery may have ideals that are not congruent with their surgeon's expectations for surgical outcomes. To identify and reduce disparities in expectations, the Facial Appearance as Core Expression Scale (FACES) was developed to assess the extent to which individuals identify their own faces as representing their ideal self. Method: In Study 1, 504 healthy young adult participants answered online questions about their own faces. In Study 2, 165 participants rated their own faces, digitally manipulated images of four patients before and after surgery, and two digitally averaged benchmark images. Results: In Study 1, the final FACES instrument had seven items and was highly reliable across genders and races. Study 2 replicated reliability findings. The before surgery and after surgery pictures yielded significant improvements in ratings, suggesting scale validity. Conclusions: The FACES consists of 14 items including a benchmark image to detect unusual responding. Results indicate the measure is reliable and sensitive to perceptions of surgical changes to faces. While the scale needs to be validated in a clinical sample, the measure may help identify patients with atypical ideal expectations for their face and may be used to quantify surgical outcomes.

Individual differences in risk perception and misperception of COVID-19 in the context of political ideology.

The COVID-19 pandemic has been characterized by misinformation, politicization of public health, and extreme differences in risk assessment. In two studies, we sought to understand factors that contribute to differences in people's understanding of the virus and associated risks. We found that conservative participants reported higher levels of acceptable risk, have lower risk estimates of activities, and endorsed more misinformation. Participants with personal health risk factors rated COVID-19 risks as higher, more reflective participants had lower acceptable risk levels, and impulsive participants endorsed more misinformation. In our second study, we also found that reflective participants were more likely to wear a mask, get vaccinated, and maintain social distancing, and that participants judged arguments about COVID-19 measures largely based on the claim rather than supporting reasons. By clarifying these individual differences, public health experts can more effectively create targeted interventions for at risk populations, and be better prepared for future outbreaks.

Adapting a Theoretically-Based intervention for underserved clinical populations at increased risk for hereditary Cancer: Lessons learned from the BRCA-Gist experience.

Background: Minorities at increased risk for Hereditary Breast and Ovarian Cancer (HBOC) frequently have low awareness and use of genetic counseling and testing (GCT). Making sure that evidence-based interventions (EBIs) reach minorities is key to reduce disparities. BRCA-Gist is a theory-informed EBI that has been proven to be efficacious in mostly non-Hispanic White non-clinical populations. We conducted formative work to inform adaptations of BRCA-Gist for use in clinical settings with at-risk diverse women. Methods: Genetic counselors (n = 20) were recruited nationally; at-risk Latinas and Blacks (n = 21) were recruited in Washington DC and Virginia. They completed the BRCA-Gist EBI between April 2018 - September 2019. Participants completed an acceptability scale and an interview to provide suggestions about implementation adaptations. T-tests for independent samples compared acceptability between at-risk women and genetic counselors. The Consensual Qualitative Research Framework was used to code adaptation suggestions. Suggested adaptations were discussed by a multidisciplinary team to integrate fidelity and adaptation considerations. Results: At-risk women had a significantly higher acceptability (M = 4.17, SD = 0.47 vs. M = 3.24, SD = 0.64; p = 0.000; scale 1-5) and satisfaction scores (M = 8.3, SD = 1.3 vs. M = 4.2, SD = 2.0; p = 0.000; scale 1-10) than genetic counselors. Genetic counselors and at-risk women suggested contextual (e.g. format) and content (e.g. shortening) adaptations to enhance the fit of BRCA-Gist for diverse clinical populations. Conclusions: Findings illustrate the process of integrating fidelity and adaptation considerations to ensure that EBIs retain their core components while enhancing the fit to minoritized clinical populations. Future studies will test the efficacy of the adapted BRCA-Gist in a Randomized Controlled Trial.

Assessing semantic coherence in conditional probability estimates.

Semantic coherence is a higher-order coherence benchmark that assesses whether a constellation of estimates--P(A), P(B), P(B | A), and P(A | B)--maps onto the relationship between sets implied by the description of a given problem. We present an automated method for evaluating semantic coherence in conditional probability estimates that efficiently reduces a large problem space into five meaningful patterns: identical sets, subsets, mutually exclusive sets, overlapping sets, and independent sets. It also identifies three theoretically interesting nonfallacious errors. We discuss unique issues in evaluating semantic coherence in conditional probabilities that are not present in joint probability judgments, such as errors resulting from dividing by zero and the use of a tolerance parameter to manage rounding errors. A spreadsheet implementing the methods described above can be downloaded as a supplement from www.springerlink.com .

Assessing semantic coherence and logical fallacies in joint probability estimates.

A constellation of joint probability estimates is semantically coherent when the quantitative relationship among estimates of P(A), P(B), P(A and B), and P(A or B) is consistent with the relationship among the sets described in the problem statement. The possible probability estimates can form an extremely large number of permutations. However, this entire problem space can be reduced to six theoretically meaningful patterns: logically fallacious (conjunction or disjunction fallacies), identical sets (e.g., water and H(2)O), mutually exclusive sets (e.g., horses and zebras), subsets (e.g., robins and birds), overlapping sets (e.g., accountants and musicians), and inconsistent overlapping sets. Determining which of these patterns describes any set of probability estimates has been automated using Excel spreadsheet formulae. Researchers may use the semantic coherence technique to examine the consequences of differently worded problems, individual differences, or experimental manipulations. The spreadsheet described above can be downloaded as a supplement from http://brm.psychonomic-journals.org/content/supplemental.

Gist Inference Scores predict gist memory for authentic patient education cancer texts.

OBJECTIVES: Develop a tool to evaluate and improve written medical communication to patients. Determine how effectively Gist Inference Scores (GIS) predict comprehension of patient education texts independently of health literacy. Explicate the text characteristics and psychological mechanism underlying GIS. METHODS: For study 1, a nationally representative sample of older women (N = 61) completed a fill-in-the-blank comprehension task on authentic National Cancer Institute (NCI) texts of varying GIS levels. In study 2, participants (N = 198) read NCI texts (high or low GIS) then recalled what they read. RESULTS: Study 1 showed that a higher percentage of different words yielding semantically similar sentence meaning were used to correctly fill-the-blanks on high GIS texts and there was no significant interaction with health literacy. In study 2, a greater proportion of decision-relevant information was recalled for high GIS texts. CONCLUSIONS: GIS predicts the likelihood that readers will form gist representations of medical texts on free recall and fill-in-the-blank tasks. High GIS texts allow for more semantic flexibility to mentally represent the same meaning, and more strongly emphasizes gist rather than verbatim representations. PRACTICAL IMPLICATIONS: GIS provides medical communicators with an automated and user-friendly method to evaluate medical texts for their ability to convey the bottom-line meaning.

Automatic Evaluation of Cancer Treatment Texts for Gist Inferences and Comprehension.

Background. It is difficult to write about cancer for laypeople such that everyone understands. One common approach to readability is the Flesch-Kincaid Grade Level (FKGL). However, FKGL has been shown to be less effective than emerging discourse technologies in predicting readability. Objective. Guided by fuzzy-trace theory, we used the discourse technology Coh-Metrix to create a Gist Inference Score (GIS) and applied it to texts from the National Cancer Institute website written for patients and health care providers. We tested the prediction that patient cancer texts with higher GIS scores are likely to be better understood than others. Design. In study 1, all 244 cancer texts were systematically subjected to an automated Coh-Metrix analysis. In study 2, 9 of those patient texts (3 each at high, medium, and low GIS) were systematically converted to fill-the-blanks (Cloze) tests in which readers had to supply the missing words. Participants (162) received 3 texts, 1 at each GIS level. Measures. GIS was measured as the mean of 7 Coh-Metrix variables, and comprehension was measured through a Cloze procedure. Results. Although texts for patients scored lower on FKGL than those for providers, they also scored lower on GIS, suggesting difficulties for readers. In study 2, participants scored higher on the Cloze task for high GIS texts than for low- or medium-GIS texts. High-GIS texts seemed to better lend themselves to correct responses using different words. Limitations. GIS is limited to text and cannot assess inferences made from images. The systematic Cloze procedure worked well in aggregate but does not make fine-grained distinctions. Conclusions. GIS appears to be a useful, theoretically motivated supplement to FKGL for use in research and clinical practice.

Lessons learned: the effect of prior technology use on Web-based interventions.

This study examined the role of regular prior technology use in treatment response to an online family problem-solving (OFPS) intervention and an Internet resource intervention (IRI) for pediatric traumatic brain injury (TBI). Participants were 150 individuals in 40 families of children with TBI randomly assigned to OFPS intervention or an IRI. All families received free computers and Internet access to TBI resources. OFPS families received Web-based sessions and therapist-guided synchronous videoconferences focusing on problem solving, communication skills, and behavior management. All participants completed measures of depression, anxiety, and computer usage. OFPS participants rated treatment satisfaction, therapeutic alliance, and Web site and technology comfort. With the OFPS intervention, depression and anxiety improved significantly more among technology using parents (n = 14) than nontechnology users (n = 6). Technology users reported increasing comfort with technology over time, and this change was predictive of depression at followup. Satisfaction and ease-of-use ratings did not differ by technology usage. Lack of regular prior home computer usage and nonadherence were predictive of anxiety at followup. The IRI was not globally effective. However, controlling for prior depression, age, and technology at work, there was a significant effect of technology at home for depression. Families with technology experience at home (n = 11) reported significantly greater improvements in depression than families without prior technology experience at home (n = 8). Although Web-based OFPS was effective in improving caregiver functioning, individuals with limited computer experience may benefit less from an online intervention due to increased nonadherence.

An online family intervention to reduce parental distress following pediatric brain injury.

This study examined whether an online problem-solving intervention could improve parental adjustment following pediatric traumatic brain injury (TBI). Families of children with moderate-to-severe TBI were recruited from the trauma registry of a large children's hospital and randomly assigned to receive online family problem solving therapy (FPS; n = 20) or Internet resources (IRC; n = 20) in addition to usual care. The FPS group reported significantly less global distress, depressive symptoms, and anxiety at follow-up than did the IRC group after controlling for baseline symptoms. The FPS group also reported significant improvements in problem-solving skills, although the groups did not differ significantly at follow-up. Findings suggest that an online, skill-building approach can be effective in facilitating parental adaptation after TBI.