Revisiting the video deficit in technology-saturated environments: Successful imitation from people, screens, and social robots.

The "video deficit" is a well-documented effect whereby children learn less well about information delivered via a screen than the same information delivered in person. Research suggests that increasing social contingency may ameliorate this video deficit. The current study instantiated social contingency to screen-based information by embodying the screen within a socially interactive robot presented to urban Australian children with frequent exposure to screen-based communication. We failed to document differences between 22- to 26-month-old children's (N = 80) imitation of screen-based information embedded in a social robot and in-person humans. Furthermore, we did not replicate the video deficit with children imitating at similar levels regardless of the presentation medium. This failure to replicate supports the findings of a recent meta-analysis of video deficit research whereby there appears to be a steady decrease over time in the magnitude of the video deficit effect. We postulate that, should the video deficit effect be truly dwindling in effect size, the video deficit may soon be a historical artifact as children begin perceiving technology as relevant and meaningful in everyday life more and more. This research finds that observational-based learning material can be successfully delivered in person, via a screen, or via a screen embedded in a social robot.

Putting "the broken bits together": A qualitative exploration of the impact of communication changes in dementia.

INTRODUCTION: Communication is an area of health and functioning that is profoundly affected by dementia. While it is known that people living with dementia and their care partners experience disruptions to daily activities and social engagement, detailed knowledge about the lived impact of dementia-related communication changes is lacking. This study sought an in-depth understanding of the lived experience of dementia-related communication changes and the associated impact, needs, and strategies. METHODS: As part of an overarching participatory design study, a qualitative (interpretive description) exploration was undertaken with people living with dementia and their care partners. Data from semi-structured interviews were analysed using thematic analysis. RESULTS: We interviewed 13 people living with dementia and 21 care partners and drew three themes and 10 subthemes from the interviews. The first theme illustrates how dementia changes communication which in turn changes life; the second captures the impact of changes on people living with dementia and care partners emotionally and in the context of relationships; and the third describes some positive and constructive ways of moving forward with dementia-related communication changes. CONCLUSIONS: Interventions to enhance function, participation, and wellbeing for people living with dementia and their care partners need to encompass support for communication changes. There is a need to ensure that people living with dementia feel dignified and respected during communication, and that care partners and inclusive communities are educated, trained, and supported to facilitate communication.

Interactive neurorobotics: Behavioral and neural dynamics of agent interactions.

Interactive neurorobotics is a subfield which characterizes brain responses evoked during interaction with a robot, and their relationship with the behavioral responses. Gathering rich neural and behavioral data from humans or animals responding to agents can act as a scaffold for the design process of future social robots. This research seeks to study how organisms respond to artificial agents in contrast to biological or inanimate ones. This experiment uses the novel affordances of the robotic platforms to investigate complex dynamics during minimally structured interactions that would be difficult to capture with classical experimental setups. We then propose a general framework for such experiments that emphasizes naturalistic interactions combined with multimodal observations and complementary analysis pipelines that are necessary to render a holistic picture of the data for the purpose of informing robotic design principles. Finally, we demonstrate this approach with an exemplar rat-robot social interaction task which included simultaneous multi-agent tracking and neural recordings.

Estimating Sentence-like Structure in Synthetic Languages Using Information Topology.

Estimating sentence-like units and sentence boundaries in human language is an important task in the context of natural language understanding. While this topic has been considered using a range of techniques, including rule-based approaches and supervised and unsupervised algorithms, a common aspect of these methods is that they inherently rely on a priori knowledge of human language in one form or another. Recently we have been exploring synthetic languages based on the concept of modeling behaviors using emergent languages. These synthetic languages are characterized by a small alphabet and limited vocabulary and grammatical structure. A particular challenge for synthetic languages is that there is generally no a priori language model available, which limits the use of many natural language processing methods. In this paper, we are interested in exploring how it may be possible to discover natural 'chunks' in synthetic language sequences in terms of sentence-like units. The problem is how to do this with no linguistic or semantic language model. Our approach is to consider the problem from the perspective of information theory. We extend the basis of information geometry and propose a new concept, which we term information topology, to model the incremental flow of information in natural sequences. We introduce an information topology view of the incremental information and incremental tangent angle of the Wasserstein-1 distance of the probabilistic symbolic language input. It is not suggested as a fully viable alternative for sentence boundary detection per se but provides a new conceptual method for estimating the structure and natural limits of information flow in language sequences but without any semantic knowledge. We consider relevant existing performance metrics such as the F-measure and indicate limitations, leading to the introduction of a new information-theoretic global performance based on modeled distributions. Although the methodology is not proposed for human language sentence detection, we provide some examples using human language corpora where potentially useful results are shown. The proposed model shows potential advantages for overcoming difficulties due to the disambiguation of complex language and potential improvements for human language methods.

Group? What group? A computational model of the group needs a psychology of "us" (not "them").

Groups are only real, and only serve as a basis for collective action, when their members perceive them to be real. For a computational model to have analytic fidelity and predictive validity it, therefore, needs to engage with the psychological reality of groups, their internal structure, and their structuring by (and of) the social context in which they function.

Personal and complex: The needs and experiences related to technology use for people living with dementia.

BACKGROUND: With increasing focus on living well with dementia, technology has been identified as having potential benefits for safety, independence and wellbeing. Despite a large growth in specialised technology development, there has been limited uptake. There is a need to understand the current use and needs related to technology for people living with dementia and their care partners (informal carers). METHODS: As part of a participatory design study, a qualitative inquiry into technology experiences and needs was undertaken within an interpretive description approach. A cross-disciplinary team including living experience experts (people living with dementia, unpaid care partners) was involved. Semi-structured interviews, including sharing technology locations and supports, were conducted, audio-recorded and transcribed. Key aspects of the technology use experience were constructed. FINDINGS: Thirteen people living with dementia and 21 care partners participated in the study. Two core aspects of technology use were identified: Lots of moving parts: the complex context, and A technology cycle: the use and non-use. The findings indicated that the context involved an individual configuration of multiple factors including the user, their technology identity, their supporters, the technology and the environment. The experience is underpinned by shifting foundations created by constantly changing technologies and user characteristics. The cycle of use and non-use identified the processes required to maintain technology engagement. CONCLUSION: The use of technology for people living with dementia is complex and personal. Future technology development, policies and practices need to consider this complexity and the effort required to keep using technology to realise the benefits.

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages.

An important aspect of using entropy-based models and proposed "synthetic languages", is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf-Mandelbrot-Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent.

Convergent cross sorting for estimating dynamic coupling.

Natural systems exhibit diverse behavior generated by complex interactions between their constituent parts. To characterize these interactions, we introduce Convergent Cross Sorting (CCS), a novel algorithm based on convergent cross mapping (CCM) for estimating dynamic coupling from time series data. CCS extends CCM by using the relative ranking of distances within state-space reconstructions to improve the prior methods' performance at identifying the existence, relative strength, and directionality of coupling across a wide range of signal and noise characteristics. In particular, relative to CCM, CCS has a large performance advantage when analyzing very short time series data and data from continuous dynamical systems with synchronous behavior. This advantage allows CCS to better uncover the temporal and directional relationships within systems that undergo frequent and short-lived switches in dynamics, such as neural systems. In this paper, we validate CCS on simulated data and demonstrate its applicability to electrophysiological recordings from interacting brain regions.

Entropy Estimation Using a Linguistic Zipf-Mandelbrot-Li Model for Natural Sequences.

Entropy estimation faces numerous challenges when applied to various real-world problems. Our interest is in divergence and entropy estimation algorithms which are capable of rapid estimation for natural sequence data such as human and synthetic languages. This typically requires a large amount of data; however, we propose a new approach which is based on a new rank-based analytic Zipf-Mandelbrot-Li probabilistic model. Unlike previous approaches, which do not consider the nature of the probability distribution in relation to language; here, we introduce a novel analytic Zipfian model which includes linguistic constraints. This provides more accurate distributions for natural sequences such as natural or synthetic emergent languages. Results are given which indicates the performance of the proposed ZML model. We derive an entropy estimation method which incorporates the linguistic constraint-based Zipf-Mandelbrot-Li into a new non-equiprobable coincidence counting algorithm which is shown to be effective for tasks such as entropy rate estimation with limited data.

The early ontogeny of infants' imitation of on screen humans and robots.

Traditionally, infants have learned how to interact with objects in their environment through direct observations of adults and peers. In recent decades these models have been available over different media, and this has introduced non-human agents to infants' learning environments. Humanoid robots are increasingly portrayed as social agents in on screen, but the degree to which infants are capable of observational learning from screen-based robots is unknown. The current study thus investigated how well 1- to 3-year-olds (N = 230) could imitate on-screen robots relative to on-screen and live humans. Participants exhibited an imitation deficit for robots that varied with age. Furthermore, the well-known video deficit did not replicate as expected, and was weak and transient relative to past research. Together, the findings documented here suggest that infants are learning from media in ways that differ from past generations, but that this new learning is nuanced when novel technologies are involved.

"Building the Threads of Connection that We Already Have": The Nature of Connections via Technology for Older People.

Objectives: The social connectedness of older people is of increasing concern. Technology has been suggested for enhancing social inclusion. This study aimed to explore the nature and quality of connections via technology.Methods: Qualitative exploration of experiences, stories, and needs was undertaken through semi-structured interviews with older (7) and middle-aged (3) adults with rich experience of connections via technology in Australia and England. Core aspects of connections through technology were constructed through interpretive description analysis.Results: Four key aspects were: 1. The caliber of connections: descriptions of a range of subjective quality of connections and characteristics of good connections; 2. Experiences of poor connection (mis- and dis-connection) including descriptions of experiences creating isolation; 3. Reasons to connect described the purposes of technology-based connections including connecting with others, themselves and places important to them; 4. Making connections work described active strategies to enhance connection.Conclusions: Using technology is part of the social engagement of many people. Considering the related feelings of connection and support strategies and needs could enhance future research and practice with older people.Clinical implications: The different characteristics and potential positive and negative experiences of connection via technology need consideration in measuring social isolation and supporting older adults.

Can a robot teach me that? Children's ability to imitate robots.

Commensurate with constant technological advances, social robots are increasingly anticipated to enter homes and classrooms; however, little is known about the efficacy of social robots as teaching tools. To investigate children's learning from robots, 1- to 3-year-olds observed either a human or a robot demonstrate two goal-directed object manipulation tasks and were then given the opportunity to act on the objects. Children exhibited less imitation from robotic models that varied with task complexity and age, a phenomenon we term the "robot deficit." In addition, the more children engaged with the robot prior to administration of the imitation task, the more likely they were to replicate the robot's actions. These findings document how children are able to learn from robots but that ongoing design of robotic platforms needs to be oriented to developing more socially engaging means of interacting.

An Automated Approach to Examining Pausing in the Speech of People With Dementia.

Dementia is a common neurodegenerative condition involving the deterioration of cognitive and communication skills. Pausing in the speech of people with dementia is a dysfluency that may be used to signal conversational trouble in social interaction. This study aimed to examine the speech-pausing profile within picture description samples from people with dementia and healthy controls (HCs) within the DementiaBank database using the Calpy computational speech processing toolkit. Sixty English-speaking participants between the ages of 53 and 88 years (Mage = 67.43, SD = 8.33; 42 females) were included in the study: 20 participants with mild cognitive impairment, 20 participants with moderate cognitive impairment, and 20 HCs. Quantitative analysis shows a progressive increase in the duration of pausing between HCs, the mild dementia group, and the moderate dementia group, respectively.

Preschool children overimitate robots, but do so less than they overimitate humans.

Past research has indicated that young children have a propensity to adopt the causally unnecessary actions of an adult, a phenomenon known as overimitation. Among competing perspectives, social accounts suggest that overimitation satisfies social motivations, be they affiliative or normative, whereas the "copy-all/refine-later" account proposes that overimitation serves a functional purpose by giving children the greatest opportunity to acquire knowledge with little error. Until recently, these two accounts have been difficult to extricate experimentally, but the development of humanoid robots provides a novel test. Here we document that children overimitate robots, but to a lesser degree than humans and regardless of whether the redundant actions are seen to be ritualistic or functional. These results are best explained with a combined account of overimitation, whereby children approach a learning task with a copy-all/refine-later motivation, but the fidelity of the reproduction of novel behaviors is modulated by the social availability of the demonstrator.

Toward the Development of SMART Communication Technology: Automating the Analysis of Communicative Trouble and Repair in Dementia.

BACKGROUND AND OBJECTIVES: Communication difficulties have been reported as one of the most stress-inducing aspects of caring for people with dementia. Notably, with disease progression comes an increase in the frequency of communication difficulty and a reduction in the effectiveness of attempts to remedy breakdowns in communication. The aim of the current research was to evaluate the utility of an automated discourse analysis tool (i.e., Discursis) in distinguishing between different types of trouble and repair signaling behaviors, demonstrated within conversations between people with dementia and their professional care staff. RESEARCH DESIGN AND METHODS: Twenty conversations between people with dementia and their professional care staff were human-coded for instances of interactive/noninteractive trouble and typical/facilitative repair behaviors. Associations were then examined between these behaviors and recurrence metrics generated by Discursis. RESULTS: Significant associations were identified between Discursis metrics, trouble-indicating, and repair behaviors. DISCUSSION AND IMPLICATIONS: These results suggest that discourse analysis software is capable of discriminating between different types of trouble and repair signaling behavior, on the basis of term recurrence calculated across speaker turns. The subsequent recurrence metrics generated by Discursis offer a means of automating the analysis of episodes of conversational trouble and repair. This achievement represents the first step toward the future development of an intelligent assistant that can analyze conversations in real time and offers support to people with dementia and their carers during periods of communicative trouble.

Social semantic networks: Measuring topic management in discourse using a pyramid of conceptual recurrence metrics.

Studies of conversational encounters typically employ manual discourse analysis methods to reveal participants' topic management patterns, usually focusing on turn-by-turn interactions within a specific social context. These analyses, while powerful, are time-consuming to apply and can prove difficult to generalize. Recurrence analysis has recently been applied to discourse datasets to model how individual terms and concepts recur over whole conversation time scales and relate patterns of recurrence to topic management practices by individuals. In this paper, we propose a new multi-level quantitative method for modelling the topical interaction dynamics in conversation based on conceptual recurrence quantification analysis. The new protocol develops a hierarchy of speakers and their interactions, and partitions recurrence based on these groups. The new protocol is evaluated against expert human coding of television broadcast interviews. Our analysis reveals topic use patterns and networks of conceptual engagement (person-person and group-group) that show experts preferentially engaging with other experts rather than with laypeople, findings that are consistent with prior expectations for this discourse, although never before expressed as metrics. The studies provide a starting point for new computational protocols to provide fast, semi-automated methods for measuring the degree of conceptual interaction between individuals and groups.

Hello Harlie: Enabling Speech Monitoring Through Chat-Bot Conversations.

People with neurological conditions such as Parkinson's disease and dementia are known to have difficulties in language and communication. This paper presents initial testing of an artificial conversational agent, called Harlie. Harlie runs on a smartphone and is able to converse with the user on a variety of topics. A description of the application and a sample dialog are provided to illustrate the various roles chat-bots can play in the management of neurological conditions. Harlie can be used for measuring voice and communication outcomes during the daily life of the user, and for gaining information about challenges encountered. Moreover, it is anticipated that she may also have an educational and support role.

Acquired Codes of Meaning in Data Visualization and Infographics: Beyond Perceptual Primitives.

While information visualization frameworks and heuristics have traditionally been reluctant to include acquired codes of meaning, designers are making use of them in a wide variety of ways. Acquired codes leverage a user's experience to understand the meaning of a visualization. They range from figurative visualizations which rely on the reader's recognition of shapes, to conventional arrangements of graphic elements which represent particular subjects. In this study, we used content analysis to codify acquired meaning in visualization. We applied the content analysis to a set of infographics and data visualizations which are exemplars of innovative and effective design. 88% of the infographics and 71% of data visualizations in the sample contain at least one use of figurative visualization. Conventions on the arrangement of graphics are also widespread in the sample. In particular, a comparison of representations of time and other quantitative data showed that conventions can be specific to a subject. These results suggest that there is a need for information visualization research to expand its scope beyond perceptual channels, to include social and culturally constructed meaning. Our paper demonstrates a viable method for identifying figurative techniques and graphic conventions and integrating them into heuristics for visualization design.

An Automated Approach to Examining Conversational Dynamics between People with Dementia and Their Carers.

The progressive neuropathology involved in dementia frequently causes a gradual decline in communication skills. Communication partners who are unaware of the specific communication problems faced by people with dementia (PWD) can inadvertently challenge their conversation partner, leading to distress and a reduced flow of information between speakers. Previous research has produced an extensive literature base recommending strategies to facilitate conversational engagement in dementia. However, empirical evidence for the beneficial effects of these strategies on conversational dynamics is sparse. This study uses a time-efficient computational discourse analysis tool called Discursis to examine the link between specific communication behaviours and content-based conversational engagement in 20 conversations between PWD living in residential aged-care facilities and care staff members. Conversations analysed here were baseline conversations recorded before staff members underwent communication training. Care staff members spontaneously exhibited a wide range of facilitative and non-facilitative communication behaviours, which were coded for analysis of conversation dynamics within these baseline conversations. A hybrid approach combining manual coding and automated Discursis metric analysis provides two sets of novel insights. Firstly, this study revealed nine communication behaviours that, if used by the care staff member in a given turn, significantly increased the appearance of subsequent content-based engagement in the conversation by PWD. Secondly, the current findings reveal alignment between human- and computer-generated labelling of communication behaviour for 8 out of the total 22 behaviours under investigation. The approach demonstrated in this study provides an empirical procedure for the detailed evaluation of content-based conversational engagement associated with specific communication behaviours.

Global segregation of cortical activity and metastable dynamics.

Cortical activity exhibits persistent metastable dynamics. Assemblies of neurons transiently couple (integrate) and decouple (segregate) at multiple spatiotemporal scales; both integration and segregation are required to support metastability. Integration of distant brain regions can be achieved through long range excitatory projections, but the mechanism supporting long range segregation is not clear. We argue that the thalamocortical matrix connections, which project diffusely from the thalamus to the cortex and have long been thought to support cortical gain control, play an equally-important role in cortical segregation. We present a computational model of the diffuse thalamocortical loop, called the competitive cross-coupling (CXC) spiking network. Simulations of the model show how different levels of tonic input from the brainstem to the thalamus could control dynamical complexity in the cortex, directing transitions between sleep, wakefulness and high attention or vigilance. The model also explains how mutually-exclusive activity could arise across large portions of the cortex, such as between the default-mode and task-positive networks. It is robust to noise but does not require noise to autonomously generate metastability. We conclude that the long range segregation observed in brain activity and required for global metastable dynamics could be provided by the thalamocortical matrix, and is strongly modulated by brainstem input to the thalamus.