Assessing the reliability of automatic sentiment analysis tools on rating the sentiment of reviews of NHS dental practices in England.

BACKGROUND: Online reviews may act as a rich source of data to assess the quality of dental practices. Assessing the content and sentiment of reviews on a large scale is time consuming and expensive. Automation of the process of assigning sentiment to big data samples of reviews may allow for reviews to be used as Patient Reported Experience Measures for primary care dentistry. AIM: To assess the reliability of three different online sentiment analysis tools (Amazon Comprehend DetectSentiment API (ACDAPI), Google and Monkeylearn) at assessing the sentiment of reviews of dental practices working on National Health Service contracts in the United Kingdom. METHODS: A Python 3 script was used to mine 15800 reviews from 4803 unique dental practices on the NHS.uk websites between April 2018 -March 2019. A random sample of 270 reviews were rated by the three sentiment analysis tools. These reviews were rated by 3 blinded independent human reviewers and a pooled sentiment score was assigned. Kappa statistics and polychoric evalutaiton were used to assess the level of agreement. Disagreements between the automated and human reviewers were qualitatively assessed. RESULTS: There was good agreement between the sentiment assigned to reviews by the human reviews and ACDAPI (k = 0.660). The Google (k = 0.706) and Monkeylearn (k = 0.728) showed slightly better agreement at the expense of usability on a massive dataset. There were 33 disagreements in rating between ACDAPI and human reviewers, of which n = 16 were due to syntax errors, n = 10 were due to misappropriation of the strength of conflicting emotions and n = 7 were due to a lack of overtly emotive language in the text. CONCLUSIONS: There is good agreement between the sentiment of an online review assigned by a group of humans and by cloud-based sentiment analysis. This may allow the use of automated sentiment analysis for quality assessment of dental service provision in the NHS.

Systemic interventions for recurrent aphthous stomatitis (mouth ulcers).

BACKGROUND: Recurrent aphthous stomatitis (RAS) is the most frequent form of oral ulceration, characterised by recurrent oral mucosal ulceration in an otherwise healthy individual. At its worst RAS can cause significant difficulties in eating and drinking. Treatment is primarily aimed at pain relief and the promotion of healing to reduce the duration of the disease or reduce the rate of recurrence. A variety of topical and systemic therapies have been utilised. OBJECTIVES: To determine the clinical effect of systemic interventions in the reduction of pain associated with RAS, a reduction in episode duration or frequency. SEARCH METHODS: We undertook electronic searches of: Cochrane Oral Health Group and PaPaS Trials Registers (to 6 June 2012); CENTRAL via The Cochrane Library (to Issue 4, 2012); MEDLINE via OVID (1950 to 6 June 2012); EMBASE via OVID (1980 to 6 June 2012); CINAHL via EBSCO (1980 to 6 June 2012); and AMED via PubMed (1950 to 6 June 2012). We searched reference lists from relevant articles and contacted the authors of eligible trials to identify further trials and obtain additional information. SELECTION CRITERIA: We included randomised controlled trials (RCTs) in which the primary outcome measures assess a reduction of pain associated with RAS, a reduction in episode duration or a reduction in episode frequency. Trials were not restricted by outcome alone. We also included RCTs of a cross-over design. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted data in duplicate. We contacted trial authors for details of randomisation, blindness and withdrawals. We carried out risk of bias assessment on six domains. We followed The Cochrane Collaboration statistical guidelines and risk ratio (RR) values were to be calculated using fixed-effect models (if two or three trials in each meta-analysis) or random-effects models (if four or more trials in each meta-analysis). MAIN RESULTS: A total of 25 trials were included, 22 of which were placebo controlled and eight made head-to-head comparisons (five trials had more than two treatment arms). Twenty-one different interventions were assessed. The interventions were grouped into two categories: immunomodulatory/anti-inflammatory and uncertain. Only one study was assessed as being at low risk of bias. There was insufficient evidence to support or refute the use of any intervention. AUTHORS' CONCLUSIONS: No single treatment was found to be effective and therefore the results remain inconclusive in regard to the best systemic intervention for RAS. This is likely to reflect the poor methodological rigour of trials, and lack of studies for certain drugs, rather than the true effect of the intervention. It is also recognised that in clinical practice, individual drugs appear to work for individual patients and so the interventions are likely to be complex in nature. In addition, it is acknowledged that systemic interventions are often reserved for those patients who have been unresponsive to topical treatments, and therefore may represent a select group of patients.