Table 3-1 displays the mean time to complete each task and the corresponding standard deviations. Times were measured in seconds. Figure 3-1 displays this information in a bar chart. The thin solid lines represent one standard deviation centered about the mean. Single factor ANOVAs were run for each task. The full ANOVA tables are contained in the Appendix. The raw data used to compute these statistics can also be found in the Appendix.
Task one, a simple search task, was Is Moosehead Lake included in the subject index? For this simple search task, there was no statistical significance (F(2, 21) = 1.9, p > 0.05). The mean times of the tasks indicate that Netscape was the fastest, followed by WebTOC and then Textual TOC.
Task two was Which index contains more entries, the author index or the subject index? This simple sibling comparison task also showed no statistical significance (F(2, 21) = 1.5, p > 0.05). The means time for WebTOC was the fastest, followed by Netscape and then by Textual TOC.
Task three was There is one picture of the Potomac River in this collection. Where exactly was this picture taken? This complex search task was more significant than the previous two, but was still not statistically significant (F(2, 21) = 2.9, p > 0.05). The mean time for Netscape was much less than that for WebTOC or Textual TOC.
Task four was Which of these two photographers have more works included in the collection: William Henry Jackson or Carleton E. Watkins? This complex sibling comparison task showed the least significance of all of the tasks (F(2, 21) = 0.7, p > 0.05). The mean times for Netscape and Textual TOC were very close while the mean time for WebTOC was somewhat higher.
Task five was Between 1872 and 1889, a document entitled "The
Extermination of the American Bison" was published. Who was the author
of this document? This embedded link task showed no statistical
significance (F(2,
21) = 1.0, p > 0.05). The mean time for Textual TOC was the least,
and the mean times for Netscape and WebTOC were about the same.
The results for the subjective satisfaction survey show that Task 1 was considered the easiest by all users, with users of Netscape giving it the lowest difficulty rating. Task 2 was considered harder by subjects in the latter two groups, especially by the Textual TOC group, which gave it a rating of over 2.5 on a scale of 1-5. However, the standard deviation was fairly high on this question. Netscape users found Task 3 very easy, while users of the two versions of WebTOC indicated that they had problems with this question. For Task 4, the results were especially favorable for WebTOC, as Netscape users found this to be the hardest task of all. WebTOC users in the other two groups gave this task a significantly lower rating. Finally, for Task 5, Netscape users again found this task to be hard, giving it a mean rating of 3.25, with a standard deviation of 1.50. WebTOC users, on the other hand, rated this question around the 2.0 mark, implying that they completed this task with some ease.
Question 2 asked about how easy it was to learn the the tools. Netscape users gave a 1.20 rating. Users in the other two groups gave a rating of around the 2.0. Question 3 asked about the organization of the web site, WebTOC users gave the most positive response, with a mean rating of 3.75. Netscape users were second giving 3.40, and Textual TOC users 3.29.
Question 4 asked whether users think a table of contents for a website is useful. All 21 subjects thought that an on-screen table of contents is a good idea for a web site. Question 5 was for users of WebTOC and Textual TOC only, asking users whether the table of contents helped them in answering the questions. Twelve of the fourteen subjects in those two treatments thought that the table of contents helped them and only two of the subjects said that it did not help them complete the tasks quicker.
Question 6 was asked about the size bars so only WebTOC users answered it. The users gave a rating of 3.25. The standard deviation was high, at 1.28.
Question 7 ask about the general satisfaction for the the tools they were using. WebTOC rated it 3.5, Netscape 3.40 and Textual TOC 3.14.