A Virtual Environment for People Who Are Blind - A Usability Study.

For most people who are blind, exploring an unknown environment can be unpleasant, uncomfortable, and unsafe. Over the past years, the use of virtual reality as a learning and rehabilitation tool for people with disabilities has been on the rise. This research is based on the hypothesis that the supply of appropriate perceptual and conceptual information through compensatory sensorial channels may assist people who are blind with anticipatory exploration. In this research we developed and tested the BlindAid system, which allows the user to explore a virtual environment. The two main goals of the research were: (a) evaluation of different modalities (haptic and audio) and navigation tools, and (b) evaluation of spatial cognitive mapping employed by people who are blind. Our research included four participants who are totally blind. The preliminary findings confirm that the system enabled participants to develop comprehensive cognitive maps by exploring the virtual environment.

Physically based hybrid approach in real time surgical simulation with force feedback.

This paper describes a novel hybrid-modeling paradigm for the simulation of surgical tool-soft tissue interactions in real time medical simulations using force feedback. A local point collocation-based method of finite spheres is coupled with a global boundary element technique to capture local features of the interaction (e.g., nonlinearities of the soft tissue) without sacrificing global accuracy. The technique is demonstrated using realistic examples.

Multimodal simulation of laparoscopic Heller myotomy using a meshless technique.

In this work we focus our attention on developing a surgical simulator for performing laparoscopic Heller myotomy using force feedback. A meshless numerical technique, the method of finite spheres, is used for the purpose of physically based, real time haptic and graphical rendering of soft tissues. Localized discretization allows display of deformations in the vicinity of the tool tip as well as interaction forces at high update rates (kHz). Novel cutting algorithms are implemented using point-based representation of anatomical models. Graphical rendering is accomplished by using a recently developed volumetric rendering technique known as splatting.

A new approach for the synthesis of glistening effect in deformable anatomical objects displayed with haptic feedback.

An environment mapping approach for texture mapping of anatomical objects with glistening was studied and implemented in surgical simulation. A classifier based on the generalized goal of achieving glistening effect, which depends on the underlying physics and the mapped data, was used as a visual metric and shown to be realistic in the presence of achieving the desired quality in real time in the presence of haptic feedback. The proposed approach was compared with the standard texture mapping approaches used in existing surgical simulation methods that generally suffer from texture stretching and with little or no glistening effect. In our work, we have successfully applied environment mapping techniques for realistic real-time rendering of anatomical objects. We also show several examples of anatomical objects with glistening effects that mimic real anatomical objects.

Measurement of in-vivo force response of intra-abdominal soft tissues for surgical simulation.

The lack of data on in-vivo material properties of soft tissues has been a significant impediment in the development of virtual reality based surgical simulators that can provide the user with realistic visual and haptic feedback. As a first step towards characterizing the mechanical behavior of organs, this work presents in-vivo force response of the liver and lower esophagus of pigs when subjected to ramp and hold, and sinusoidal indentations delivered using a haptic feedback device, Phantom, employed as a mechanical stimulator. The results show that pulse significantly affects the reaction forces and that the lower esophagus is 2 to 2.5 times stiffer than the liver.

High-frequency ultrasonic attenuation and backscatter coefficients of in vivo normal human dermis and subcutaneous fat.

In vivo attenuation and backscatter coefficients of normal human forearm dermis and subcutaneous fat were determined in the ranges 14 to 50 MHz and 14 to 34 MHz, respectively. Data were collected using three different transducers to ensure that results were independent of the measurement system. Attenuation coefficient was obtained by computing spectral slopes vs. depth, with the transducers axially translated to minimize diffraction effects. Backscatter coefficient was obtained by compensating recorded backscatter spectra for system-dependent effects and, additionally, for one transducer using the reference phantom technique. Good agreement was seen between the computed attenuation and backscatter results from the different transducers/methods. The attenuation coefficient of the forearm dermis was well described by a linear dependence with a slope that ranged between 0.08 to 0.39 (median = 0.21) dB mm(-1) MHz(-1). The backscatter coefficient of the dermis was generally in the range 10(-3) to 10(-1) Sr(-1) mm(-1) and showed an increasing trend with frequency. No significant differences in attenuation coefficient slope between the forearm dermis and fat were noted. Within the range of 14 to 34 MHz, the ratio of integrated (average) backscatter of dermis to that of fat ranged from 1.03 to 87.1 (median = 6.45), indicating significantly higher backscatter for dermis than for fat. Data were also recorded at the fingertip where the attenuation coefficient slope of the dermis was seen to be higher than that at the forearm.

A meshless numerical technique for physically based real time medical simulations.

This work introduces, for the first time, a meshless modeling technique, the method of finite spheres, for physically based, real time rendering of soft tissues in medical simulations. The technique is conceptually similar to the traditional finite element techniques. However, while the finite element techniques requires a slow mesh generation process, this new technique has significant potential for multimodal medical simulations of the future since it does not use a mesh. Several examples are presented showing the effectiveness of the scheme.

Development and evaluation of an epidural injection simulator with force feedback for medical training.

Performing epidural injections is a complex task that demands a high level of skill and precision from the physician, since an improperly performed procedure can result in serious complications for the patient. The objective of our project is to create an epidural injection simulator for medical training and education that provides the user with realistic feel encountered during an actual procedure. We have used a Phantom haptic interface by SensAble Technologies, which is capable of three-dimensional force feedback, to simulate interactions between the needle and bones or tissues. An additional degree-of-freedom through an actual syringe was incorporated to simulate the "loss of resistance" effect, commonly considered to be the most reliable method for identifying the epidural space during an injection procedure. The simulator also includes a new training feature called "Haptic Guidance" that allows the user to follow a previously recorded expert procedure and feel the encountered forces. Evaluations of the simulator by experienced professionals indicate that the simulation system has considerable potential to become a useful aid in medical training.

Real-time prediction of hand trajectory by ensembles of cortical neurons in primates.

Signals derived from the rat motor cortex can be used for controlling one-dimensional movements of a robot arm. It remains unknown, however, whether real-time processing of cortical signals can be employed to reproduce, in a robotic device, the kind of complex arm movements used by primates to reach objects in space. Here we recorded the simultaneous activity of large populations of neurons, distributed in the premotor, primary motor and posterior parietal cortical areas, as non-human primates performed two distinct motor tasks. Accurate real-time predictions of one- and three-dimensional arm movement trajectories were obtained by applying both linear and nonlinear algorithms to cortical neuronal ensemble activity recorded from each animal. In addition, cortically derived signals were successfully used for real-time control of robotic devices, both locally and through the Internet. These results suggest that long-term control of complex prosthetic robot arm movements can be achieved by simple real-time transformations of neuronal population signals derived from multiple cortical areas in primates.

Simulation of tissue cutting and bleeding for laparoscopic surgery using auxiliary surfaces.

Realistic simulation of tissue cutting and bleeding is important components of a surgical simulator that are addressed in this study. Surgeons use a number of instruments to perform incision and dissection of tissues during minimally invasive surgery. For example, a coagulating hook is used to tear and spread the tissue that surrounds organs and scissors are used to dissect the cystic duct during laparoscopic cholecystectomy. During the execution of these procedures, bleeding may occur and blood flows over the tissue surfaces. We have developed computationally fast algorithms to display (1) tissue cutting and (2) bleeding in virtual environments with applications to laparoscopic surgery. Cutting through soft tissue generates an infinitesimally thin slit until the sides of the surface are separated from each other. Simulation of an incision through tissue surface is modeled in three steps: first, the collisions between the instrument and the tissue surface are detected as the simulated cutting tool passes through. Then, the vertices along the cutting path are duplicated. Finally, a simple elastic tissue model is used to separate the vertices from each other to reveal the cut. Accurate simulation of bleeding is a challenging problem because of the complexities of the circulatory system and the physics of viscous fluid flow. There are several fluid flow models described in the literature, but most of them are computationally slow and do not specifically address the problem of blood flowing over soft tissues. We have reviewed the existing models, and have adapted them to our specific task. The key characteristics of our blood flow model are a visually realistic display and real-time computational performance. To display bleeding in virtual environments, we developed a surface flow algorithm. This method is based on a simplified form of the Navier-Stokes equations governing viscous fluid flow. The simplification of these partial differential equations results in a wave equation that can be solved efficiently, in real-time, with finite difference techniques. The solution describes the flow of blood over the polyhedral surfaces representing the anatomical structures and is displayed as a continuous polyhedral surface drawn over the anatomy.

Raised object on a planar surface stroked across the fingerpad: responses of cutaneous mechanoreceptors to shape and orientation.

The representations of orientation and shape were studied in the responses of cutaneous mechanoreceptors to an isolated, raised object on a planar surface stroked across the fingerpad. The objects were the top portions of a sphere with a 5-mm radius, and two toroids each with a radius of 5 mm along one axis and differing radii of 1 or 3 mm along the orthogonal axis. The velocity and direction of stroking were fixed while the orientation of the object in the horizontal plane was varied. Each object was stroked along a series of laterally shifted, parallel, linear trajectories over the receptive fields of slowly adapting, type I (SA), and rapidly adapting, type I (RA) mechanoreceptive afferents innervating the fingerpad of the monkey. "Spatial event plots" (SEPs) of the occurrence of action potentials, as a function of the location of each object on the receptive field, were interpreted as the responses of a spatially distributed population of fibers. That portion of the plot evoked by the curved object (the SEPc) provided a representation of the shape and orientation of the two-dimensional outline of the object in the horizontal plane in contact with the skin. For both SAs and RAs, the major vector of the SEPc, obtained by a principal components analysis, was linearly related to the physical orientation of the major axis of each toroid. The spatial distribution of discharge rates [spatial rate surface profiles (SRSs), after plotting mean instantaneous frequency versus spatial locus within the SEPc] represented object shape in a third dimension, normal to the skin surface. The shape of the SA SRSs, well fitted by Gaussian equations, better represented object shape than that of the RA SRSs. A cross-sectional profile along the minor axis [spatial rate profile (SRP)] was approximately triangular for SAs. After normalization for differences in peak height, the falling slopes of the SA SRPs increased, and the base widths decreased with curvature of the object's minor axis. These curvature-related differences in slopes and widths were invariant with changes in object orientation. It is hypothesized that circularity in object shape is coded by the constancy of slopes of SA SRPs between peak and base and that the constancy of differences in the widths and falling slopes evoked by different raised objects encodes, respectively, the differences in their sizes and shapes regardless of differences in their orientation on the skin.

Encoding of shape and orientation of objects indented into the monkey fingerpad by populations of slowly and rapidly adapting mechanoreceptors.

The peripheral neural representation of object shape and orientation was studied by recording the responses of a spatially distributed population of rapidly and slowly adapting type I mechanoreceptors (RAs and SAs, respectively) to objects of different shapes and orientations indented at a fixed location on the fingerpad of the anesthetized monkey. The toroidal objects had a radius of 5 mm on the major axis, and 1, 3, or 5 mm on the minor axis. Each object was indented into the fingerpad for 4 s at orientations of 0, 45, 90, and 135 degrees using a contact force of 15 gwt. Estimations of the population responses (PRs) were constructed by combining the responses of 91 SA and 97 RA single afferents at discrete times during the indentation. The PR was composed of the neural discharge rates (z coordinate) plotted at x and y coordinates of the most sensitive spot of the receptive field. The shapes of the PRs were related to the shapes of the objects by fitting the PRs with Gaussian surfaces. The orientations of the PRs were determined from weighted principal component analyses. The SA PR encoded both the orientation and shape of the objects, whereas the RA PR did neither. The SA PR orientation was biased toward the long axis of the finger. The RA PR encoded orientation only for the object with the highest curvature but did so ambiguously. Only the SA PR was well fit by a Gaussian surface. The shape of the object was discriminated by the SA PR within the first 500 ms of contact, and the form of the SA PR remained constant during the subsequent 3.5 s. This was manifested by constant widths of the PR along the major and minor axes despite a peak response that decreased from its maximum at 200 ms to an asymptotic value starting at 1 s. Thus the shape and orientation of each object were coded by the shape and orientation of the SA PR.

Force interactions in laparoscopic simulations: haptic rendering of soft tissues.

Research in the area of computer assisted surgery and surgical simulation has mainly focused on developing 3D geometrical models of the human body from 2D medical images, visualization of internal structures for educational and preoperative surgical planning purposes, and graphical display of soft tissue behavior in real time. Conveying to the surgeon the touch and force sensations with the use of haptic interfaces has not been investigated in detail. We have developed a set of haptic rendering algorithms for simulating "surgical instrument--soft tissue" interactions. Although the focus of the study is the development of algorithms for simulation of laparoscopic procedures, the developed techniques are also useful in simulating other medical procedures involving touch and feel of soft tissues. The proposed force-reflecting soft tissue models are in various fidelities and have been developed to simulate the behavior of elastically deformable objects in virtual environments. The developed algorithms deal directly with geometry of anatomical organs, surface and compliance characteristics of tissues, and the estimation of appropriate reaction forces to convey to the user a feeling of touch and force sensations.

Neural encoding of shape: responses of cutaneous mechanoreceptors to a wavy surface stroked across the monkey fingerpad.

1. The role of cutaneous mechanoreceptors in the tactile perception of shape was investigated. Objects whose surfaces were shaped as a pattern of smooth, alternating convex and concave cylindrical surfaces of differing radii of curvature were constructed such that there were no discontinuities in the slope of the surface. These "wavy surfaces" were stroked across the fingerpad of the anesthetized monkey and electrophysiological responses of slowly adapting type I mechanoreceptive afferents (SAs) and rapidly adapting type I mechanoreceptive afferents (RAs) were recorded. 2. For both SAs and RAs, each convexity indenting the skin evoked a burst of impulses and each concavity of the same curvature that followed elicited a pause in response. "Spatial event plots" (SEPs) of the occurrence of action potentials as a function of the location of the object on the receptive field were obtained and interpreted as the responses of a spatially distributed population of fibers. With increasing magnitude of curvature (equivalently, decreasing radius of curvature) of convexity, the mean width of the burst in the SEPs for each fiber type (representing the width of a region of skin containing active fibers) decreased and the mean discharge rate during the burst increased. Over a range of velocities of stroking from 1 to 40 mm/s, the number of RAs activated increased with velocity, whereas SAs were active at all velocities. For both SAs and RAs, the burst rates increased with velocity, whereas the widths of the bursts and pauses remained approximately invariant. Thus the spatial measures of burst or pause width provide a robust representation of the size of a feature on the object surface. 3. For a given velocity of stroking, the spatially distributed pattern of averaged discharge rates (spatial rate profile, SRP) provided a representation of the shape of the wavy surface. The distance between neighboring peaks in the SRP for individual RAs and SAs was approximately the same as the distance between the peaks of the wavy surface. The averaged SRP for a population of SAs provided a better representation of shape than that for RAs. Whereas active regions in the SEP can be isomorphic to the two dimensional form of the stimulus "footprint" in contact with the skin surface, the SRP, which in addition encodes the features of the stimulus in the third dimension normal to the skin surface, is not isomorphic to the stimulus shape. 4. When the sizes as well as the shapes of objects are varied, it is hypothesized that a central processing mechanism extracts the invariant property of shape from the slopes of the rising and falling phases of an SRP that has been normalized for overall differences in discharge rates. These differences would be expected to occur with variations in the parameters of stimulation such as compressional force, stroke trajectory, and stroke velocity. It was shown that a common feature of the mean SRP for SAs evoked by each wavy surface convexity, regardless of its radius, was the constancy of the slope from the base to the peak and from the peak to the base. Thus a possible code for the constant curvature of a cylinder is the constancy of the slopes along the rising and declining phases of the triangular-shaped spatial response profile evoked in the SA population by the cylindrical convexity.

An investigation of the mechanics of tactile sense using two-dimensional models of the primate fingertip.

Tactile information about an object in contact with the skin surface is contained in the spatio-temporal load distribution on the skin, the corresponding stresses and strains at mechanosensitive receptor locations within the skin, and the associated pattern of electrical impulses produced by the receptor population. At present, although the responses of the receptors to known stimuli can be recorded, no experimental techniques exist to observe either the load distribution on the skin or the corresponding stress-state at the receptor locations. In this paper, the role of mechanics in the neural coding of tactile information is investigated using simple models of the primate fingertip. Four models that range in geometry from a semi-infinite medium to a cylindrical finger with a rigid bone, and composed of linear elastic media, are analyzed under plane strain conditions using the finite element method. The results show that the model geometry has a significant influence on the surface load distribution as well as the subsurface stress and strain fields for a given mechanical stimulus. The elastic medium acts like a spatial low pass filter with the property that deeper the receptor location, the more blurred the tactile information. None of the models predicted the experimentally observed surface deflection profiles under line loads as closely as a simple heterogeneous waterbed model that treated the fingerpad as a membrane enclosing an incompressible fluid (Srinivasan, 1989). This waterbed model, however, predicted a uniform state of stress inside the fingertip and thus failed to explain the spatial variations observed in the neural response. For the cylindrical model indented by rectangular gratings, the maximum compressive strain and strain energy density at typical receptor locations emerged as the two strain measures that were directly related to the electrophysiologically recorded response rate of slowly adapting type I (SAI) mechanoreceptors. Strain energy density is a better candidate to be the relevant stimulus for SAIs, since it is a scalar that is invariant with respect to receptor orientations and is a direct measure of the distortion of the receptor caused by the loads imposed on the skin.

Manual discrimination of compliance using active pinch grasp: the roles of force and work cues.

In these experiments, two plates were grasped between the thumb and the index finger and squeezed together along a linear track. The force resisting the squeeze, produced by an electromechanical system under computer control, was programmed to be either constant (in the case of the force discrimination experiments) or linearly increasing (in the case of the compliance discrimination experiments) over the squeezing displacement. After completing a set of basic psychophysical experiments on compliance resolution (Experiment 1), we performed further experiments to investigate whether work and/or terminal-force cues played a role in compliance discrimination. In Experiment 2, compliance and force discrimination experiments were conducted with a roving-displacement paradigm to dissociate work cues (and terminal-force cues for the compliance experiments) from compliance and force cues, respectively. The effect of trial-by-trial feedback on response strategy was also investigated. In Experiment 3, compliance discrimination experiments were conducted with work cues totally eliminated and terminal-force cues greatly reduced. Our results suggest that people tend to use mechanical work and force cues for compliance discrimination. When work and terminal-force cues were dissociated from compliance cues, compliance resolution was poor (22%) relative to force and length resolution. When work cues were totally eliminated, performance could be predicted from terminal-force cues. A parsimonious description of all data from the compliance experiments is that subjects discriminated compliance on the basis of terminal force.

Tactual discrimination of softness.

1. We investigated the ability of humans to tactually discriminate the softness of objects, using novel elastic objects with deformable and rigid surfaces. For objects with deformable surfaces, we cast transparent rubber specimens with variable compliances. For objects with rigid surfaces ("spring cells") we fabricated telescoping hollow cylinders with the inner cylinder supported by several springs. To measure the human discriminability and to isolate the associated information-processing mechanisms, we performed psychophysical experiments under three conditions: 1) active touch with the normal finger, where both tactile and kinesthetic information was available to the subject: 2) active touch with local cutaneous anesthesia, so that only kinesthetic information was available; and 3) passive touch, where a computer-controlled mechanical stimulator brought down the compliant specimens onto the passive fingerpad of the subject, who therefore had only tactile information. 2. We first characterized the mechanical behavior of the human fingerpad and the test objects by determining the relationship between the depth and force of indentation during constant-velocity indentations by a rigid probe. The fingerpad exhibited a pronounced nonlinear behavior in the indentation depth versus force trace such that compliance, as indicated by the local slope of the trace, decreased with increases in indentation depth. The traces for all the rubber specimens were approximately linear, indicating a constant but distinct value of compliance for each specimen. The fingerpad was more compliant than each of the rubber specimens. 3. All the human subjects showed excellent softness discriminability in ranking the rubber specimens by active touch, and the subjective perception of softness correlated one-to-one with the objectively measured compliance. The ability of subjects to discriminate the compliance of spring cells was consistently poorer compared with that of the rubber specimens. 4. For pairwise discrimination of a selected set of rubber specimens, kinesthetic information alone was insufficient. However, tactile information alone was sufficient, even when the velocities and forces of specimen application were randomized. In contrast, for discriminating pairs of spring cells, tactile information alone was insufficient, and both tactile and kinesthetic information were found to be necessary. 5. The differences in the sufficiency of tactile information for the discrimination of the two types of objects can be explained by the mechanics of contact of the fingerpad and its effect on tactile information. For objects with deformable surfaces, the spatial pressure distribution within the contact region depends on both the force applied and the specimen compliance.(ABSTRACT TRUNCATED AT 250 WORDS)

Cutaneous neural codes for shape.

In the pursuit of peripheral neural representations of shape for the sense of touch, a series of two- and three-dimensional objects were stroked across the fingerpad of the anesthetized monkey and responses evoked in cutaneous mechanoreceptive primary afferent nerve fibers recorded. Responses of slowly adapting fibers (SAs) and rapidly adapting fibers (RAs) were recorded to the stroking of a cylinder, a sphere, several ellipsoids, and a pattern of alternating convex and concave cylindrical bars. The compressional force was maintained constant during a stroke, and the stroke velocities as well as orientations of the objects and stroke trajectories were varied between separate sets of trials. The major geometrical properties of the shapes were well represented in the spatiotemporal responses of the afferent fiber populations, particularly those of the SAs. Intensive parameters of shapes, such as the magnitude of change in skin curvature produced as a result of contact with the object surface, were encoded in the discharge rates of SAs and RAs, but this neural code was also influenced by changes in stroke velocity. Spatial parameters of shapes such as the extent of contact and the changes in contour that characterize a shape as belonging to a particular category (such as a sphere as opposed to a cylinder) are encoded in the spatially distributed discharge rates of the SA population. This spatial response profile provides a neural code that is probably invariant with moderate changes in the way the object comes in contact with the skin, such as the contact force or the orientation of the object.

Cultivation of buffalo green monkey kidney cells persistently infected with hepatitis A virus.

Studies were carried out to determine the effect of prolongation of incubation periods, cocultivation with normal buffalo green monkey kidney (BGMK) cells and different concentrations of foetal calf serum (FCS) on the production of hepatitis A virus (HAV) by BGMK cell line persistently infected with HAV strain HM175. HAV could be detected from week 1 onwards. However, maintenance of cultures beyond this period was found to yield substantially higher quantities of virus. Cocultivation of persistently infected cells with normal BGMK cells also improved the antigen yields. Different concentrations of FCS did not show any effect on the amount of virus produced. The cell line was maintained up to 46 passages during which there was continuous production of HAV in the cells and release of small amounts of virus in the culture supernatants. Cell associated and cell free viral particles were found to be infectious. Supernatant derived virus was a highly suitable inoculum for infecting other susceptible cell lines. Persistently infected BGMK cell line appears to be a reliable and economical source to derive HAV in adequate amounts for diagnostic and research purposes.