Kinesthetic Feedback for Understanding Program Execution.

To better prepare future generations, knowledge about computers and programming are one of the many skills that are part of almost all Science, Technology, Engineering, and Mathematic programs; however, teaching and learning programming is a complex task that is generally considered difficult by students and teachers alike. One approach to engage and inspire students from a variety of backgrounds is the use of educational robots. Unfortunately, previous research presents mixed results on the effectiveness of educational robots on student learning. One possibility for this lack of clarity may be because students have a wide variety of styles of learning. It is possible that the use of kinesthetic feedback, in addition to the normally used visual feedback, may improve learning with educational robots by providing a richer, multi-modal experience that may appeal to a larger number of students with different learning styles. It is also possible, however, that the addition of kinesthetic feedback, and how it may interfere with the visual feedback, may decrease a student's ability to interpret the program commands being executed by a robot, which is critical for program debugging. In this work, we investigated whether human participants were able to accurately determine a sequence of program commands performed by a robot when both kinesthetic and visual feedback were being used together. Command recall and end point location determination were compared to the typically used visual-only method, as well as a narrative description. Results from 10 sighted participants indicated that individuals were able to accurately determine a sequence of movement commands and their magnitude when using combined kinesthetic + visual feedback. Participants' recall accuracy of program commands was actually better with kinesthetic + visual feedback than just visual feedback. Although the recall accuracy was even better with the narrative description, this was primarily due to participants confusing an absolute rotation command with a relative rotation command with the kinesthetic + visual feedback. Participants' zone location accuracy of the end point after a command was executed was significantly better for both the kinesthetic + visual feedback and narrative methods compared to the visual-only method. Together, these results suggest that the use of both kinesthetic + visual feedback improves an individual's ability to interpret program commands, rather than decreases it.

Design of a "Cobot Tactile Display" for Accessing Virtual Diagrams by Blind and Visually Impaired Users.

Access to graphical information plays a very significant role in today's world. Access to this information can be particularly limiting for individuals who are blind or visually impaired (BVIs). In this work, we present the design of a low-cost, mobile tactile display that also provides robotic assistance/guidance using haptic virtual fixtures in a shared control paradigm to aid in tactile diagram exploration. This work is part of a larger project intended to improve the ability of BVI users to explore tactile graphics on refreshable displays (particularly exploration time and cognitive load) through the use of robotic assistance/guidance. The particular focus of this paper is to share information related to the design and development of an affordable and compact device that may serve as a solution towards this overall goal. The proposed system uses a small omni-wheeled robot base to allow for smooth and unlimited movements in the 2D plane. Sufficient position and orientation accuracy is obtained by using a low-cost dead reckoning approach that combines data from an optical mouse sensor and inertial measurement unit. A low-cost force-sensing system and an admittance control model are used to allow shared control between the Cobot and the user, with the addition of guidance/virtual fixtures to aid in diagram exploration. Preliminary semi-structured interviews, with four blind or visually impaired participants who were allowed to use the Cobot, found that the system was easy to use and potentially useful for exploring virtual diagrams tactually.

Toward an improved haptic zooming algorithm for graphical information accessed by individuals who are blind and visually impaired.

An increasing amount of information content used in school, work, and everyday living is presented in graphical form. Unfortunately, it is difficult for people who are blind or visually impaired to access this information, especially when many diagrams are needed. One problem is that details, even in relatively simple visual diagrams, can be very difficult to perceive using touch. With manually created tactile diagrams, these details are often presented in separate diagrams which must be selected from among others. Being able to actively zoom in on an area of a single diagram so that the details can be presented at a reasonable size for exploration purposes seems a simpler approach for the user. However, directly using visual zooming methods have some limitations when used haptically. Therefore, a new zooming method is proposed to avoid these pitfalls. A preliminary experiment was performed to examine the usefulness of the algorithm compared to not using zooming. The results showed that the number of correct responses improved with the developed zooming algorithm and participants found it to be more usable than not using zooming for exploration of a floor map.

Dynamic tactile diagram simplification on refreshable displays.

The increasing use of visual diagrams in educational and work environments, and even our daily lives, has created obstacles for individuals who are blind or visually impaired to independently access the information they represent. Although physical tactile pictures can be created to convey the visual information, it is typically a slow, cumbersome, and costly process. Refreshable haptic displays, which interact with computers, promise to make this access quicker, easier, and cheaper. One important aspect in converting visual to tactile diagrams is to simplify the diagram as otherwise it can be too difficult to interpret with touch. Enabling this to be under user control in an interactive environment, such as with refreshable displays, could allow users to avoid being overwhelmed by the diagrams at any instant in time while still retaining access to all information in "storage". Through this article the authors investigate whether two types of diagram simplification--boundary simplification and contextual simplification--showed potential utility in an interactive environment. Boundary simplification was found to be significantly helpful in answering general questions about borders on a geographic map, and contextual simplification was helpful in answering relational questions, as compared to using the original map unchanged.

Designing Haptic Assistive Technology for Individuals Who Are Blind or Visually Impaired.

This paper considers issues relevant for the design and use of haptic technology for assistive devices for individuals who are blind or visually impaired in some of the major areas of importance: Braille reading, tactile graphics, orientation and mobility. We show that there is a wealth of behavioral research that is highly applicable to assistive technology design. In a few cases, conclusions from behavioral experiments have been directly applied to design with positive results. Differences in brain organization and performance capabilities between individuals who are "early blind" and "late blind" from using the same tactile/haptic accommodations, such as the use of Braille, suggest the importance of training and assessing these groups individually. Practical restrictions on device design, such as performance limitations of the technology and cost, raise questions as to which aspects of these restrictions are truly important to overcome to achieve high performance. In general, this raises the question of what it means to provide functional equivalence as opposed to sensory equivalence.