Design of Haptic and Tactile Interfaces for Blind Users

Abstract

Since computer use became more widespread in the 1980's and 1990's, considerable effort has been put into ensuring that the blind have equal access to state of the art techology. However, the dominance of graphical user interfaces and direct manipulation has reduced the effectiveness of old speech-based systems. This article discusses aspects of tactile and haptic interfaces, reviews current research on the topic, and provides design principles for practitioners culled from recent research.

Introduction

The advent of graphical user interfaces (GUIs) in the late 1980's and early 1990's was a significant cause of concern to the community of blind computer users. GUIs and the related concept of direct manipulation were, by their very nature, inaccessible to blind people. The blind were in danger of being left out of the resulting rise in computer usability. Indeed, where screen reader technology once provided nearly equal access to computers (although this was due more to the primitive nature of early interfaces available to sighted users than to the performance of screen reader software), these tools had to be designed to convey objects such as icons, images, tables, graphs, and windows.

Recently, an increasing body of research has focused on tactile and haptic devices. Tactile devices communicate with users through their sense of touch. Meanwhile, a haptic device combines tactile perception with kinesthetic (i.e., the position, placement, and orientation) sensing. Braille displays and braille readers, tactile diagram and graph interpreters, and tactile mice for use in direct-manipulation environments are just some examples of such devices. This paper surveys some of the modelling and design issues that arise in the construction of such devices.

Background

The majority of research into affording the blind access to computers has focused on speech input or output. There are two main reasons for this. First, in the days before GUIs, when command languages and text-based displays were prevalent, the sequential text contents on the screen accurately conveyed details of the interface. Simply reading the contents of the screen gave blind users the same level of access to the interface available to sighted users. Old displays placed one line of text after another onto the screen, which was easy for software to read. Second, as GUIs became the industry standard, concepts like drag-and-drop, resizable windows, and icons became dominant as well. Blind users wanted interfaces that allowed them to manipulate interface objects in the same way their sighted counterparts did. It seemed reasonable to try to extend the previously effective speech-based systems to accommodate direct manipulation.

However, tactile and haptic interfaces offer significant advantages over auditory-based interfaces. Tactile based systems offer the possibility of interacting more closely with the interface than auditory systems do. This will be discussed in more detail later. Furthermore, tactile systems will work for any blind user, regardless of nationality. By their very nature, speech-based systems must be tuned to the language, dialect, and in some cases accent of the individual user. There is also evidence that tactile displays are faster than auditory displays [9]. Lastly, at this point, effective tactile and haptic devices seem closer to reality than effective auditory-based devices. After almost 40 years of research, state of the art speech understanding and screen reading technology is at best only adequate. With much less effort put into research, tactile and haptic devices can achieve similar performance [27].

Analysis

The majority of research for the blind has been directed towards auditory interpretations of existing interfaces. Two papers dealing with audial rendering of visual information are available at this website: Media Conversion from Visual to Audio: Voice Browsers and The Universal Usability Analysis of Network Conversation System from Video to Audio. Also, there has been a considerable amount of research on tactile and haptic interfaces in general. For a good survey of recent research concerning haptic perception with sighted users, see [17]. Meanwhile, Revesz [24] writes about the nature of haptic perception, noting that haptic and tactile perception is less accurate and much slower than visual perception.

There are many papers that describe existing devices and provide results of experiments with the devices. A Braille display, HyperBraille, is described in [8]. DBNet, a Braille internet access system for users who are both deaf and blind, is detailed in [14]. Petrie, Morley, and Weber describe GUIB, a tactile mouse for use in direct manipulation systems, in [22]. Martin Kurze describes TDRAW, a haptic method for drawing pictures, in [12] and TGUIDE, a haptic method for exploring images, in [13]. In [27] and [28], Gerhard Weber describes an innovative system whereby a blind user gives commands to a terminal via hand and finger gestures. A device to accept this kind of input is described in [31]. Finally, multimodal devices (i.e., devices that incorporate tactile, haptic, and auditory information) are described in [1,19,26].

There has been some experimental work specifically focusing on blind users. In [2], the authors investigate haptic exploration of virtual environments. They compare blind and sighted users' haptic perception of objects with respect to the objects' texture, size and shape. More results concerning the development of tactile mice are contained in [30]. Results regarding both sighted and blind users' ability to identify objects using a tactile display are in [15,16]. In [9], King et al. show that Braille displays can access Videotex information twice as fast as speech displays.

There has been very little work on design principles or on the modeling issues involved in designing tactile and haptic devices. Martin Kurze describes a high-level process for designing interfaces for blind people in [10]. In [18], Miller and Zeleznik develop a tactile mouse designed to enchance sighted users' experiences with GUIs; they give three principles (which are summarized below) that guided their design. These principles are valid for the design of tactile devices in general, so they are included here. Gunderson describes a task-based model for designing interfaces for blind people in [5]. In [3], the authors discuss the difference between the user interface and the screen contents, and argue that any assistive technology should convey the former rather than the latter. Friedlander and Schlueter in [4] show that Fitts' Law does not necessarily apply to tactile pointing devices. They propose an alternative linear model that more accurately predicts performance times in their study. Lastly, [6] considers the issues that partially sighted users have and why devices designed for totally blind people are useless for the partially sighted. A wearable headset that makes use of partially sighted users' existing vision is described in [21].

Guidelines and Recommendations

In this section we present several guidelines for use when designing tactile and haptic devices. It is important to note that principles that guide the design of traditional interfaces, such as Shneiderman's "Eight Golden Rules", still apply [25]. These principles should be used in addition to such principles, not in place of them.

Use Tactile Force To Guide, not Control

Actually the three principles from [18] mentioned above, they are combined into a single rule. In [18], Miller and Zeleznik describe a tactile mouse designed for use with UNIX's X Window system. This device is intended to enhance the windowing system, not to replace it. Therefore, the mouse was not designed to be used by the blind. Nonetheless, their guiding principles are valid.

Certain types of motor-control problems, like drawing a straight line with a mouse, are common even for sighted users. Other, less common problems, such as clicking on the wrong icon or menu item, are going to be more common for blind users. Tactile forces can help reduce these errors. For instance, a slight counter-force can be applied as the user moves from one menu item to another, giving the effect of "ridges" separating the items. Also, as a mouse cursor nears a radio button, slight force could be applied in the direction of the button, giving the effect of "gravity" pulling towards the button.

Furthermore, users should be able to override a device's tactile force if necessary. The notion of overriding a guiding force by "fighting through" the constraint has been implemented in previous systems. Miller and Zeleznik introduce the idea of "sidestepping" a constraint by moving around it in a different direction (one might think of fighting through a constraint as going over a wall, and as sidestepping as going around it).

Lastly, the authors claim that the tactile force applied by a device should be user-inspired in two senses. First, the amount of force that tactile devices apply to users should vary from user to user so as not to overpower or "be too strong" for a user. Second, the direction of force should vary depending on the direction that the user moves the device.

Consider Alternative Predictor Models

The standard predictive model of performance times in visual interfaces is Fitts' Law. Fitts' Law states that the time required for a user to move a mouse from a starting point to a target increases as the logarithm of the distance to the target and decreases as the logarithm of the size of the target. In their design of Bullseye, an alternative menu representation, Friedlander, et al. [4] note that Fitts' Law is not always appropriate for predicting performance times for tactile and auditory devices. First, in nonvisual systems, users rarely know the exact location of the target, so this information is not going to be as helpful in locating screen objects. Second, Fitts' Law does not account for the existence of intermediate targets along a path. In the Bullseye Menu, intermediate targets trigger tactile signals that can be used as navigational aids by counting the number of signals that occur on the path to the target. The authors propose a linear model, which predicts that the time to hit a target is a linear function of the distance to the target and the size of the target, and found that this model more accurately predicted performance times for tasks involving the Bullseye menu.

Be Aware of Diversity Within the Blind Community

While accommodating users' diversity is always a goal of interface designers, there is considerable diversity just within the blind community. The ability to read Braille is not as common as one might think, for example. Furthermore, there is much variation in the reading speeds among users who do read Braille.[23]. When designing interfaces and devices, one should keep this in mind.

Convey the Interface, not the Screen

This is a recommendation from [3]. At a high level, an interface is a collection of objects and operations one can perform on those objects. The visual representation of an interface on the monitor is only one interpretation. The idea is that when affording the blind access to an interface, one should not convey the visual representation, but rather the interface itself. In [3], the authors argue that by translating the semantic level of the interface, one can convey the same constructs that are available to sighted users.

Encourage Collaboration with Sighted Community

Simply giving the blind efficient access to computers should not be the goal of these types of interfaces. These interfaces should enhance the ability of blind users to integrate into the larger community of users. This would enable, for example, allowing blind workers to collaborate with sighted coworkers on projects at the office. The interface should give the blind user a clear "picture" of what a sighted partner is doing with the system. To this end, the interface should attempt to convey to the blind users a mental model of the system similar to that of sighted users.

Conclusion

Tactile and haptic interfaces have great potential in making computers more accessible to the blind. What is particularly attractive is that they address specific shortcomings of the traditional audio-based approach. They do not rely on undependable speech rendering or recognition software, and they can do more than simply echo the screen contents. This paper summarizes recent research in the field, and offers effective guidelines for the design of such devices.

References