The Effect of Screen Size on Readability Using Three Different Portable Devices

Experiment

Introduction and Hypothesis

Participants were asked to perform six tasks, three for each of two different experiments. The tasks involved reading two different types of documents presented on each of three different portable devices, and answering a question based on each document. We recorded the time it took to read each document and recorded the answers to the questions. We also recorded whether the answers to the questions were correct or not. After answering the questions, the subjects were asked to complete a user satisfaction survey.

Our hypothesis is that a reader using a smaller device will process small amounts of information, one page or less, quickly and easily, independent of the screen size. As the need for scrolling or paging increases, because of the small screen size, frustration levels will rise and performance times will increase. In both the one page experiment and the multiple pages experiment error rates will increase as screen size decreases.

The independent variable was a portable computing device. This independent variable had three treatments, the Palm Pilot, Rocket Book, and laptop computer. For each independent variable we conducted two separate experiments. One experiment was based on a single page of text, the other used enough text so that paging or scrolling was required. Our dependent variables were the time to complete each task, the error rates for each task, and subjective satisfaction.

Pilot Study Results

We ran the experiment with two pilot subjects and found that the experiment worked as designed. The task difficulty was checked during the pilot study and no problems were identified. Two problem areas were identified at this time and changes were made to correct them. Both involved practical issues of running the experiment. The first problem occurred when the laptop infrared port tried to beam information to the other two devices. We solved the problem by covering the infrared port and only presenting one device at a time. We also determined that it was necessary to be careful to present only one task at a time to a subject in order to prevent confusion.

Equipment

The three portable devices were a Palm V^TM model of a Palm Pilot^TM , a Rocket Book^TM , and a Toshiba Satellite 405CDT^TM laptop computer. The three devices were set up on a table with one device in front of the subject and the other devices on the side waiting to be presented. Since we only used the page-up and page-down keys, the styluses from the Palm Pilot and Rocket Book were removed. The cover was also removed from the Palm Pilot. Based on the findings of the pilot study the laptop infrared port was covered with a piece of heavy tape during the experiment.

Material

Two types of documents for the experiment were prepared in advance. We prepared one version of each type of document for each device. The length of each type of document remained the same, only the content changed. The non-scrolling/paging document, was a list of names and telephone numbers. This list was the same length on all three devices, and was designed to be the maximum number of lines that would fit on the Palm Pilot without requiring scrolling or paging. The scrolling/paging document, was a list of references to journal articles from scientific journals in a standard reference format. Again, this list was the same length on all three devices, but in this case it was designed to cover two full pages on the laptop, and was allowed to grow to the size needed on the other two devices.

In both cases the documents were single spaced, and in all six cases, we attempted to normalize the font size and typeface, basing our choices on the typeface of the Palm Pilot which was unchangeable. We were not however able to compensate for the graininess of the text presentation on the Palm Pilot. The appearance of the characters is dependent on the number of pixels contained in the letters. The number of pixels per character is built into the design of smaller devices. We also tried to normalize the contrast among the three devices, using the backlighting feature where available. Our goal was to control the appearance of the text as carefully as possible and remove it as a factor in the outcome of the experiment.

We developed a visual search task for each type of document in order to measure readability. For the telephone number list, subjects were asked to find the area code for a name beginning with a specific letter. For the reference lists, we asked the subjects to count the number of articles from a particular journal.

The following are links to the printed materials used during the experiment

Document presentation

For the non-paging experiment the document was pre-positioned in the center of the screen surrounded by white space, except for the Palm Pilot where the document filled the screen. For the paging experiment, the document filled each of two pages on the laptop, and all pages but the last on the Rocket Book, 4 pages and the Palm Pilot, 10 pages. The text was presented as black letters on a white screen. All text was presented in 10 point type, the standard type for the Palm Pilot and the Rocket Book, and at a resolution of 800 x 600. The font design for the Laptop and the Rocket Book was chosen to be as close as possible to that of the pre-set Palm Pilot font.

Subjects

Eighteen subjects were drawn from the University of Maryland community, family members, and a small private boy's high school. Each subject was asked to complete a preliminary questionnaire that collected basic non-identifying information about age, sex, visual acuity, whether left or right handed, education, computer use, and previous experience with the devices. 78% of participants were between the ages of 10 and 30, although the range of ages varied from 10 to over 50. (In fact, no subjects under 16 were tested as the school did not allow Freshmen and Sophomores to participate.) However, we do not feel that the outliers in the age groups affected our statistics, except for one case discussed later in Discussion of Results. 78% were male (although there was some overlap in these groups.) All but one subject was right-handed, and 72% were native English speakers. 78% wore some form of corrective lenses. Because of the source of our subjects, about half were in high school or had completed high school, the rest had at least some college education. None of the subjects had ever used a Rocket Book before, about half had some experience with a Palm Pilot, and 89% had at least some experience with a laptop computer.

Pre-experiment procedures

Before the experiment began all three devices were turned on and the correct document was loaded. The use of the devices and the nature of the tasks were explained to each subject before the experiments began. The modified instructions were repeated before the second set of experiments. In order to avoid bias caused by the order of the devices with each experiment six different order arrangements were used. Once the subjects completed the preliminary questionnaire they were presented with each device in turn. They were also given a sheet of paper containing the task for that device. Subjects were timed and observed as they performed each task and wrote their answers on the task sheet. When each subject had completed the six tasks they were asked to fill out a final subjective evaluation, thanked for their participation, and allowed to "play" with the Palm Pilot or the Rocket Book for a brief while. (In some cases the "playtime" was the incentive used to entice subjects into participating in the experiment.)

Problems

The one major problem that became clear as subjects began to perform the experiment was that the scrolling/paging task was worded ambiguously. The following question and hint made up the scrolling/paging task. "How many papers were presented at an 'ACM CHI Conference'? (Hint: How many times does ACM CHI appear in the reference list)." It wasn't apparent that this question was ambiguous until after many subjects participated in the experiment. We began to see results with high error rates. The words "ACM CHI" and "ACM" were both in the text. It is possible that some subjects interpreted the question to mean that they should write how many times "ACM CHI" appeared, while other subjects might have thought that only the number of times "ACM" appeared was worth counting. Therefore, that could possibly be an explanation for why there were high error rates. We considered discarding the first twelve runs and starting again, but felt that while participants might not be getting the answers we expected, they were scanning the entire text, and therefore our timing results would remain valid. Since the same question was used for the scrolling/paging task for each of the three devices, timing results would be affected consistently despite the ambiguity.