UNav: An Infrastructure-Independent Vision-Based Navigation System for People with Blindness and Low Vision.

Vision-based localization approaches now underpin newly emerging navigation pipelines for myriad use cases, from robotics to assistive technologies. Compared to sensor-based solutions, vision-based localization does not require pre-installed sensor infrastructure, which is costly, time-consuming, and/or often infeasible at scale. Herein, we propose a novel vision-based localization pipeline for a specific use case: navigation support for end users with blindness and low vision. Given a query image taken by an end user on a mobile application, the pipeline leverages a visual place recognition (VPR) algorithm to find similar images in a reference image database of the target space. The geolocations of these similar images are utilized in a downstream task that employs a weighted-average method to estimate the end user's location. Another downstream task utilizes the perspective-n-point (PnP) algorithm to estimate the end user's direction by exploiting the 2D-3D point correspondences between the query image and the 3D environment, as extracted from matched images in the database. Additionally, this system implements Dijkstra's algorithm to calculate a shortest path based on a navigable map that includes the trip origin and destination. The topometric map used for localization and navigation is built using a customized graphical user interface that projects a 3D reconstructed sparse map, built from a sequence of images, to the corresponding a priori 2D floor plan. Sequential images used for map construction can be collected in a pre-mapping step or scavenged through public databases/citizen science. The end-to-end system can be installed on any internet-accessible device with a camera that hosts a custom mobile application. For evaluation purposes, mapping and localization were tested in a complex hospital environment. The evaluation results demonstrate that our system can achieve localization with an average error of less than 1 m without knowledge of the camera's intrinsic parameters, such as focal length.

Dysfunctional mode switching between fixation and saccades: collaborative insights into two unusual clinical disorders.

Voluntary rapid eye movements (saccades) redirect the fovea toward objects of visual interest. The saccadic system can be considered as a dual-mode system: in one mode the eye is fixating, in the other it is making a saccade. In this review, we consider two examples of dysfunctional saccades, interrupted saccades in late-onset Tay-Sachs disease and gaze-position dependent opsoclonus after concussion, which fail to properly shift between fixation and saccade modes. Insights and benefits gained from bi-directional collaborative exchange between clinical and basic scientists are emphasized. In the case of interrupted saccades, existing mathematical models were sufficiently detailed to provide support for the cause of interrupted saccades. In the case of gaze-position dependent opsoclonus, existing models could not explain the behavior, but further development provided a reasonable hypothesis for the mechanism underlying the behavior. Collaboration between clinical and basic science is a rich source of progress for developing biologically plausible models and understanding neurological disease. Approaching a clinical problem with a specific hypothesis (model) in mind often prompts new experimental tests and provides insights into basic mechanisms.

How sandbag-able are concussion sideline assessments? A close look at eye movements to uncover strategies.

Background: Sideline diagnostic tests for concussion are vulnerable to volitional poor performance ("sandbagging") on baseline assessments, motivated by desire to subvert concussion detection and potential removal from play. We investigated eye movements during sandbagging versus best effort on the King-Devick (KD) test, a rapid automatized naming (RAN) task.Methods: Participants performed KD testing during oculography following instructions to sandbag or give best effort.Results: Twenty healthy participants without concussion history were included (mean age 27 ± 8 years). Sandbagging resulted in longer test times (89.6 ± 39.2 s vs 48.2 ± 8.5 s, p < .001), longer inter-saccadic intervals (459.5 ± 125.4 ms vs 311.2 ± 79.1 ms, p < .001) and greater numbers of saccades (171.4 ± 47 vs 138 ± 24.2, p < .001) and reverse saccades (wrong direction for reading) (21.2% vs 11.3%, p < .001). Sandbagging was detectable using a logistic model with KD times as the only predictor, though more robustly detectable using eye movement metrics.Conclusions: KD sandbagging results in eye movement differences that are detectable by eye movement recordings and suggest an invalid test score. Objective eye movement recording during the KD test shows promise for distinguishing between best effort and post-injury performance, as well as for identifying sandbagging red flags.

Did I do that? Detecting a perturbation to visual feedback in a reaching task.

The motor system executes actions in a highly stereotyped manner despite the high number of degrees of freedom available. Studies of motor adaptation leverage this fact by disrupting, or perturbing, visual feedback to measure how the motor system compensates. To elicit detectable effects, perturbations are often large compared to trial-to-trial reach endpoint variability. However, awareness of large perturbations can elicit qualitatively different compensation processes than unnoticeable ones can. The current experiment measures the perturbation detection threshold, and investigates how humans combine proprioception and vision to decide whether displayed reach endpoint errors are self-generated only, or are due to experimenter-imposed perturbation. We scaled or rotated the position of the visual feedback of center-out reaches to targets and asked subjects to indicate whether visual feedback was perturbed. Subjects detected perturbations when they were at least 1.5 times the standard deviation of trial-to-trial endpoint variability. In contrast to previous studies, subjects suboptimally combined vision and proprioception. Instead of using proprioceptive input, they responded based on the final (possibly perturbed) visual feedback. These results inform methodology in motor system experimentation, and more broadly highlight the ability to attribute errors to one's own motor output and combine visual and proprioceptive feedback to make decisions.

Rapid Number Naming and Quantitative Eye Movements May Reflect Contact Sport Exposure in a Collegiate Ice Hockey Cohort.

BACKGROUND: The King-Devick (K-D) test of rapid number naming is a reliable visual performance measure that is a sensitive sideline indicator of concussion when time scores worsen (lengthen) from preseason baseline. Within cohorts of youth athletes <18 years old, baseline K-D times become faster with increasing age. We determined the relation of rapid number-naming time scores on the K-D test to electronic measurements of saccade performance during preseason baseline assessments in a collegiate ice hockey team cohort. Within this group of young adult athletes, we also sought to examine the potential role for player age in determining baseline scores. METHODS: Athletes from a collegiate ice hockey team received preseason baseline testing as part of an ongoing study of rapid rink-side performance measures for concussion. These included the K-D test (spiral-bound cards and tablet computer versions). Participants also performed a laboratory-based version of the K-D test with simultaneous infrared-based video-oculographic recordings using an EyeLink 1000+. This allowed measurement of the temporal and spatial characteristics of eye movements, including saccadic velocity, duration, and intersaccadic interval (ISI). RESULTS: Among 13 male athletes, aged 18-23 years (mean 20.5 ± 1.6 years), prolongation of the ISI (a combined measure of saccade latency and fixation duration) was the measure most associated with slower baseline time scores for the EyeLink-paired K-D (mean 38.2 ± 6.2 seconds, r = 0.88 [95% CI 0.63-0.96], P = 0.0001), the K-D spiral-bound cards (36.6 ± 5.9 seconds, r = 0.60 [95% CI 0.08-0.87], P = 0.03), and K-D computerized tablet version (39.1 ± 5.4 seconds, r = 0.79 [95% CI 0.42-0.93], P = 0.001). In this cohort, older age was a predictor of longer (worse) K-D baseline time performance (age vs EyeLink-paired K-D: r = 0.70 [95% CI 0.24-0.90], P = 0.008; age vs K-D spiral-bound cards: r = 0.57 [95% CI 0.03-0.85], P = 0.04; age vs K-D tablet version: r = 0.59 [95% CI 0.06-0.86], P = 0.03) as well as prolonged ISI (r = 0.62 [95% CI 0.11-0.87], P = 0.02). Slower baseline K-D times were not associated with greater numbers of reported prior concussions. CONCLUSIONS: Rapid number-naming performance using the K-D at preseason baseline in this small cohort of collegiate ice hockey players is best correlated with ISI among eye movement-recording measures. Baseline K-D scores notably worsened with increasing age, but not with numbers of prior concussions in this small cohort. While these findings require further investigation by larger studies of contact and noncontact sports athletes, they suggest that duration of contact sports exposure may influence preseason test performance.

The effect of linguistic background on rapid number naming: implications for native versus non-native English speakers on sideline-focused concussion assessments.

OBJECTIVE: To determine if native English speakers (NES) perform differently compared to non-native English speakers (NNES) on a sideline-focused rapid number naming task. A secondary aim was to characterize objective differences in eye movement behaviour between cohorts. BACKGROUND: The King-Devick (KD) test is a rapid number-naming task in which numbers are read from left-to-right. This performance measure adds vision-based assessment to sideline concussion testing. Reading strategies differ by language. Concussion may also impact language and attention. Both factors may affect test performance. METHODS: Twenty-seven healthy  NNES and healthy NES performed a computerized KD test under high-resolution video-oculography.  NNES also performed a Bilingual Dominance Scale (BDS) questionnaire to weight linguistic preferences (i.e., reliance on non-English language(s)). RESULTS: Inter-saccadic intervals were significantly longer in  NNES (346.3 ± 78.3 ms vs. 286.1 ± 49.7 ms, p = 0.001), as were KD test times (54.4 ± 15.1 s vs. 43.8 ± 8.6 s, p = 0.002). Higher BDS scores, reflecting higher native language dominance, were associated with longer inter-saccadic intervals in  NNES. CONCLUSION: These findings have direct implications for the assessment of athlete performance on vision-based and other verbal sideline concussion tests; these results are particularly important given the international scope of sport. Pre-season baseline scores are essential to evaluation in the event of concussion, and performance of sideline tests in the athlete's native language should be considered to optimize both baseline and post-injury test accuracy.

Wearables for persons with blindness and low vision: form factor matters.

Based on statistics from the WHO and the International Agency for the Prevention of Blindness, an estimated 43.3 million people have blindness and 295 million have moderate and severe vision impairment globally as of 2020, statistics expected to increase to 61 million and 474 million respectively by 2050, staggering numbers. Blindness and low vision (BLV) stultify many activities of daily living, as sight is beneficial to most functional tasks. Assistive technologies for persons with blindness and low vision (pBLV) consist of a wide range of aids that work in some way to enhance one's functioning and support independence. Although handheld and head-mounted approaches have been primary foci when building new platforms or devices to support function and mobility, this perspective reviews potential shortcomings of these form factors or embodiments and posits that a body-centered approach may overcome many of these limitations.

Accuracy and Usability of Smartphone-Based Distance Estimation Approaches for Visual Assistive Technology Development.

Goal: Distance information is highly requested in assistive smartphone Apps by people who are blind or low vision (PBLV). However, current techniques have not been evaluated systematically for accuracy and usability. Methods: We tested five smartphone-based distance-estimation approaches in the image center and periphery at 1-3 meters, including machine learning (CoreML), infrared grid distortion (IR_self), light detection and ranging (LiDAR_back), and augmented reality room-tracking on the front (ARKit_self) and back-facing cameras (ARKit_back). Results: For accuracy in the image center, all approaches had <±2.5 cm average error, except CoreML which had ±5.2-6.2 cm average error at 2-3 meters. In the periphery, all approaches were more inaccurate, with CoreML and IR_self having the highest average errors at ±41 cm and ±32 cm respectively. For usability, CoreML fared favorably with the lowest central processing unit usage, second lowest battery usage, highest field-of-view, and no specialized sensor requirements. Conclusions: We provide key information that helps design reliable smartphone-based visual assistive technologies to enhance the functionality of PBLV.

Speeded reaching movements around invisible obstacles.

We analyze the problem of obstacle avoidance from a Bayesian decision-theoretic perspective using an experimental task in which reaches around a virtual obstacle were made toward targets on an upright monitor. Subjects received monetary rewards for touching the target and incurred losses for accidentally touching the intervening obstacle. The locations of target-obstacle pairs within the workspace were varied from trial to trial. We compared human performance to that of a Bayesian ideal movement planner (who chooses motor strategies maximizing expected gain) using the Dominance Test employed in Hudson et al. (2007). The ideal movement planner suffers from the same sources of noise as the human, but selects movement plans that maximize expected gain in the presence of that noise. We find good agreement between the predictions of the model and actual performance in most but not all experimental conditions.

A Smart Service System for Spatial Intelligence and Onboard Navigation for Individuals with Visual Impairment (VIS4ION Thailand): study protocol of a randomized controlled trial of visually impaired students at the Ratchasuda College, Thailand.

BACKGROUND: Blind/low vision (BLV) severely limits information about our three-dimensional world, leading to poor spatial cognition and impaired navigation. BLV engenders mobility losses, debility, illness, and premature mortality. These mobility losses have been associated with unemployment and severe compromises in quality of life. VI not only eviscerates mobility and safety but also, creates barriers to inclusive higher education. Although true in almost every high-income country, these startling facts are even more severe in low- and middle-income countries, such as Thailand. We aim to use VIS4ION (Visually Impaired Smart Service System for Spatial Intelligence and Onboard Navigation), an advanced wearable technology, to enable real-time access to microservices, providing a potential solution to close this gap and deliver consistent and reliable access to critical spatial information needed for mobility and orientation during navigation. METHODS: We are leveraging 3D reconstruction and semantic segmentation techniques to create a digital twin of the campus that houses Mahidol University's disability college. We will do cross-over randomization, and two groups of randomized VI students will deploy this augmented platform in two phases: a passive phase, during which the wearable will only record location, and an active phase, in which end users receive orientation cueing during location recording. A group will perform the active phase first, then the passive, and the other group will experiment reciprocally. We will assess for acceptability, appropriateness, and feasibility, focusing on experiences with VIS4ION. In addition, we will test another cohort of students for navigational, health, and well-being improvements, comparing weeks 1 to 4. We will also conduct a process evaluation according to the Saunders Framework. Finally, we will extend our computer vision and digital twinning technique to a 12-block spatial grid in Bangkok, providing aid in a more complex environment. DISCUSSION: Although electronic navigation aids seem like an attractive solution, there are several barriers to their use; chief among them is their dependence on either environmental (sensor-based) infrastructure or WiFi/cell "connectivity" infrastructure or both. These barriers limit their widespread adoption, particularly in low-and-middle-income countries. Here we propose a navigation solution that operates independently of both environmental and Wi-Fi/cell infrastructure. We predict the proposed platform supports spatial cognition in BLV populations, augmenting personal freedom and agency, and promoting health and well-being. TRIAL REGISTRATION: ClinicalTrials.gov under the identifier: NCT03174314, Registered 2017.06.02.

Training AI to Recognize Objects of Interest to the Blind and Low Vision Community.

Recent object detection models show promising advances in their architecture and performance, expanding potential applications for the benefit of persons with blindness or low vision (pBLV). However, object detection models are usually trained on generic data rather than datasets that focus on the needs of pBLV. Hence, for applications that locate objects of interest to pBLV, object detection models need to be trained specifically for this purpose. Informed by prior interviews, questionnaires, and Microsoft's ORBIT research, we identified thirty-five objects pertinent to pBLV. We employed this user-centric feedback to gather images of these objects from the Google Open Images V6 dataset. We subsequently trained a YOLOv5x model with this dataset to recognize these objects of interest. We demonstrate that the model can identify objects that previous generic models could not, such as those related to tasks of daily functioning - e.g., coffee mug, knife, fork, and glass. Crucially, we show that careful pruning of a dataset with severe class imbalances leads to a rapid, noticeable improvement in the overall performance of the model by two-fold, as measured using the mean average precision at the intersection over union thresholds from 0.5 to 0.95 (mAP50-95). Specifically, mAP50-95 improved from 0.14 to 0.36 on the seven least prevalent classes in the training dataset. Overall, we show that careful curation of training data can improve training speed and object detection outcomes. We show clear directions on effectively customizing training data to create models that focus on the desires and needs of pBLV.Clinical Relevance- This work demonstrated the benefits of developing assistive AI technology customized to individual users or the wider BLV community.

Motor learning reveals the existence of multiple codes for movement planning.

Coordinate systems for movement planning are comprised of an anchor point (e.g., retinocentric coordinates) and a representation (encoding) of the desired movement. One of two representations is often assumed: a final-position code describing desired limb endpoint position and a vector code describing movement direction and extent. The existence of movement-planning systems using both representations is controversial. In our experiments, participants completed reaches grouped by target location (providing practice for a final-position code) and the same reaches grouped by movement vector (providing vector-code practice). Target-grouped reaches resulted in the isotropic (circular) distribution of errors predicted for position-coded reaches. The identical reaches grouped by vector resulted in error ellipses aligned with the reach direction, as predicted for vector-coded reaches. Manipulating only recent movement history to provide better learning for one or the other movement code, we provide definitive evidence that both movement representations are used in the identical task.

Adaptation to sensory-motor reflex perturbations is blind to the source of errors.

In the study of visual-motor control, perhaps the most familiar findings involve adaptation to externally imposed movement errors. Theories of visual-motor adaptation based on optimal information processing suppose that the nervous system identifies the sources of errors to effect the most efficient adaptive response. We report two experiments using a novel perturbation based on stimulating a visually induced reflex in the reaching arm. Unlike adaptation to an external force, our method induces a perturbing reflex within the motor system itself, i.e., perturbing forces are self-generated. This novel method allows a test of the theory that error source information is used to generate an optimal adaptive response. If the self-generated source of the visually induced reflex perturbation is identified, the optimal response will be via reflex gain control. If the source is not identified, a compensatory force should be generated to counteract the reflex. Gain control is the optimal response to reflex perturbation, both because energy cost and movement errors are minimized. Energy is conserved because neither reflex-induced nor compensatory forces are generated. Precision is maximized because endpoint variance is proportional to force production. We find evidence against source-identified adaptation in both experiments, suggesting that sensory-motor information processing is not always optimal.

Compensation for changing motor uncertainty.

When movement outcome differs consistently from the intended movement, errors are used to correct subsequent movements (e.g., adaptation to displacing prisms or force fields) by updating an internal model of motor and/or sensory systems. Here, we examine changes to an internal model of the motor system under changes in the variance structure of movement errors lacking an overall bias. We introduced a horizontal visuomotor perturbation to change the statistical distribution of movement errors anisotropically, while monetary gains/losses were awarded based on movement outcomes. We derive predictions for simulated movement planners, each differing in its internal model of the motor system. We find that humans optimally respond to the overall change in error magnitude, but ignore the anisotropy of the error distribution. Through comparison with simulated movement planners, we found that aimpoints corresponded quantitatively to an ideal movement planner that updates a strictly isotropic (circular) internal model of the error distribution. Aimpoints were planned in a manner that ignored the direction-dependence of error magnitudes, despite the continuous availability of unambiguous information regarding the anisotropic distribution of actual motor errors.

Binocular spatial induction for the perception of depth does not cross the midline.

Although horizontal binocular retinal disparity between images in the two eyes resulting from their different views of the world has long been the centerpiece for understanding the unique characteristics of stereovision, it does not suffice to explain many binocular phenomena. Binocular depth contrast (BDC), the induction of an appearance of visual pitch in a centrally located line by pitched-from-vertical flanking lines, has particularly been the subject of a good deal of attention in this regard. In the present article, we show that BDC does not cross the median plane but is restricted to the side of the visual field containing a unilateral inducer. These results cannot be explained by the use of retinal disparity alone or in combination with any additional factors or processes previously suggested to account for stereovision. We present a two-channel three-stage neuromathematical model that accounts quantitatively for present and previous BDC results and also accounts for a large number of the most prominent features of binocular pitch perception: Stage 1 of the differencing channel obtains the difference between the retinal orientations of the images in the two eyes separately for the inducer and the test line; stage 1 of the summing channel obtains the corresponding sums. Signals from inducer and test stimuli are combined linearly in each channel in stage 2, and in stage 3 the outputs from the two channels are combined along with a bias signal from the body-referenced mechanism to yield ', the model's prediction for the perception of pitch.

Looking "Cherry Red Spot Myoclonus" in the Eyes: Clinical Phenotype, Treatment Response, and Eye Movements in Sialidosis Type 1.

Sialidosis type 1 is a rare lysosomal storage disorder caused by mutations of the neuraminidase gene. Specific features suggesting this condition include myoclonus, ataxia and macular cherry-red spots. However, phenotypic variability exists. Here, we present detailed clinical and video description of three patients with this rare condition. We also provide an in-depth characterization of eye movement abnormalities, as an additional tool to investigate pathophysiological mechanisms and to facilitate diagnosis. In our patients, despite phenotypic differences, eye movement deficits largely localized to the cerebellum.

King-Devick Test Performance and Cognitive Dysfunction after Concussion: A Pilot Eye Movement Study.

(1) Background: The King-Devick (KD) rapid number naming test is sensitive for concussion diagnosis, with increased test time from baseline as the outcome measure. Eye tracking during KD performance in concussed individuals shows an association between inter-saccadic interval (ISI) (the time between saccades) prolongation and prolonged testing time. This pilot study retrospectively assesses the relation between ISI prolongation during KD testing and cognitive performance in persistently-symptomatic individuals post-concussion. (2) Results: Fourteen participants (median age 34 years; 6 women) with prior neuropsychological assessment and KD testing with eye tracking were included. KD test times (72.6 ± 20.7 s) and median ISI (379.1 ± 199.1 msec) were prolonged compared to published normative values. Greater ISI prolongation was associated with lower scores for processing speed (WAIS-IV Coding, r = 0.72, p = 0.0017), attention/working memory (Trails Making A, r = -0.65, p = 0.006) (Digit Span Forward, r = 0.57, p = -0.017) (Digit Span Backward, r= -0.55, p = 0.021) (Digit Span Total, r = -0.74, p = 0.001), and executive function (Stroop Color Word Interference, r = -0.8, p = 0.0003). (3) Conclusions: This pilot study provides preliminary evidence suggesting that cognitive dysfunction may be associated with prolonged ISI and KD test times in concussion.

Art therapy for Parkinson's disease.

OBJECTIVE: To explore the potential rehabilitative effect of art therapy and its underlying mechanisms in Parkinson's disease (PD). METHODS: Observational study of eighteen patients with PD, followed in a prospective, open-label, exploratory trial. Before and after twenty sessions of art therapy, PD patients were assessed with the UPDRS, Pegboard Test, Timed Up and Go Test (TUG), Beck Depression Inventory (BDI), Modified Fatigue Impact Scale and PROMIS-Self-Efficacy, Montreal Cognitive Assessment, Rey-Osterrieth Complex Figure Test (RCFT), Benton Visual Recognition Test (BVRT), Navon Test, Visual Search, and Stop Signal Task. Eye movements were recorded during the BVRT. Resting-state functional MRI (rs-fMRI) was also performed to assess functional connectivity (FC) changes within the dorsal attention (DAN), executive control (ECN), fronto-occipital (FOC), salience (SAL), primary and secondary visual (V1, V2) brain networks. We also tested fourteen age-matched healthy controls at baseline. RESULTS: At baseline, PD patients showed abnormal visual-cognitive functions and eye movements. Analyses of rs-fMRI showed increased functional connectivity within DAN and ECN in patients compared to controls. Following art therapy, performance improved on Navon test, eye tracking, and UPDRS scores. Rs-fMRI analysis revealed significantly increased FC levels in brain regions within V1 and V2 networks. INTERPRETATION: Art therapy improves overall visual-cognitive skills and visual exploration strategies as well as general motor function in patients with PD. The changes in brain connectivity highlight a functional reorganization of visual networks.

The complexity of eye-hand coordination: a perspective on cortico-cerebellar cooperation.

BACKGROUND: Eye-hand coordination (EHC) is a sophisticated act that requires interconnected processes governing synchronization of ocular and manual motor systems. Precise, timely and skillful movements such as reaching for and grasping small objects depend on the acquisition of high-quality visual information about the environment and simultaneous eye and hand control. Multiple areas in the brainstem and cerebellum, as well as some frontal and parietal structures, have critical roles in the control of eye movements and their coordination with the head. Although both cortex and cerebellum contribute critical elements to normal eye-hand function, differences in these contributions suggest that there may be separable deficits following injury. METHOD: As a preliminary assessment for this perspective, we compared eye and hand-movement control in a patient with cortical stroke relative to a patient with cerebellar stroke. RESULT: We found the onset of eye and hand movements to be temporally decoupled, with significant decoupling variance in the patient with cerebellar stroke. In contrast, the patient with cortical stroke displayed increased hand spatial errors and less significant temporal decoupling variance. Increased decoupling variance in the patient with cerebellar stroke was primarily due to unstable timing of rapid eye movements, saccades. CONCLUSION: These findings highlight a perspective in which facets of eye-hand dyscoordination are dependent on lesion location and may or may not cooperate to varying degrees. Broadly speaking, the results corroborate the general notion that the cerebellum is instrumental to the process of temporal prediction for eye and hand movements, while the cortex is instrumental to the process of spatial prediction, both of which are critical aspects of functional movement control.