RESUMO
This article examines the basis of Natural Language Understanding of transformer based language models, such as BERT. It does this through a case study on idiom token classification. We use idiom token identification as a basis for our analysis because of the variety of information types that have previously been explored in the literature for this task, including: topic, lexical, and syntactic features. This variety of relevant information types means that the task of idiom token identification enables us to explore the forms of linguistic information that a BERT language model captures and encodes in its representations. The core of this article presents three experiments. The first experiment analyzes the effectiveness of BERT sentence embeddings for creating a general idiom token identification model and the results indicate that the BERT sentence embeddings outperform Skip-Thought. In the second and third experiment we use the game theory concept of Shapley Values to rank the usefulness of individual idiomatic expressions for model training and use this ranking to analyse the type of information that the model finds useful. We find that a combination of idiom-intrinsic and topic-based properties contribute to an expression's usefulness in idiom token identification. Overall our results indicate that BERT efficiently encodes a variety of information from topic, through lexical and syntactic information. Based on these results we argue that notwithstanding recent criticisms of language model based semantics, the ability of BERT to efficiently encode a variety of linguistic information types does represent a significant step forward in natural language understanding.
RESUMO
BACKGROUND: Stroke is the most relevant cause of acquired persistent disability in adulthood. The relationship between patient's weight during rehabilitation and stroke functional outcome is controversial, previous research reported positive, negative and no effects, with scarce studies specifically addressing working-age patients. AIM: To evaluate the association between Body Mass Index (BMI) and the functional progress of adult (<65 years) patients with stroke admitted to a rehabilitation hospital. DESIGN: Retrospective observational cohort study. SETTING: Inpatient rehabilitation center. POPULATION: 178 stroke patients (ischemic or hemorrhagic). METHODS: Point-biserial and Spearman's correlations, multivariate linear regressions and analysis of covariance were used to describe differences in functional outcomes after adjusting for age, sex, severity, dysphagia, depression and BMI category. Functional Independence Measure (FIM), FIM gain, efficiency and effectiveness were assessed. RESULTS: Participants were separated in 3 BMI categories: normal weight (47%), overweight (33%) and obese (20%). There were no significant differences between BMI categories in any functional outcome (total FIM [T-FIM], cognitive [C-FIM]), motor [M-FIM]) at discharge, admission, gain, efficiency or effectiveness. In regression models BMI (as continuous variable) was not significant predictor of T-FIM at discharge after adjusting for age, sex, severity, dysphagia, depression and ataxia (R2=0.4813), significant predictors were T-FIM at admission (ß=0.528) and NIHSS (ß=-0.208). M-FIM efficiency did not significantly differ by BMI subgroups, neither did C-FIM efficiency. Length of stay (LOS) and T-FIM effectiveness were associated for normal (r=0.33) and overweight (r=0.43), but not for obese. LOS and T-FIM efficiency were strongly negatively associated only for obese (r=-0.50). CONCLUSIONS: FIM outcomes were not associated to BMI, nevertheless each BMI category when individually considered (normal weight, overweight or obese) was characterized by different associations involving FIM outcomes and clinical factors. CLINICAL REHABILITATION IMPACT: In subacute post-stroke working-age patients undergoing rehabilitation, BMI was not associated to FIM outcomes (no obesity paradox was reported in this sample). Distinctive significant associations emerged within each BMI category, (supporting their characterization) such as length of stay and T-FIM effectiveness were associated for normal weight and overweight, but not for obese. Length of stay and T-FIM efficiency were strongly negatively associated only for obese.