Effects of Link Arrangements on Search Efficiency
4.1 Result Interpretation:
Overall, objective results of these experiments show that there is an interaction effect between link arrangement and column treatment. In other words, statistical analysis supports the suggestion that changing the column layout of a small number of links does not have as profound an impact on search time as changing the column layout of a large number of links. Further statistics indicate that the link arrangement, in general, has an effect on search time. This simply means that a user forced to scroll through a lengthy list will take significantly more time to find a given web link than a user with a short, non-scrolling list.
The data also implies that, in general, column treatment does not have a significant effect on search time.
4.2 Statistics Explanation:
This battery of tests produced three major objectively based statistics from a two-way ANOVA. The two-way analysis is an analysis of two factors at three levels with seven subjects per treatment and 42 subjects overall.
Two-Way ANOVA: (Link Treatment) x (Column Arrangement) - Figure 3
This two-way ANOVA suggests that there is a significant
interaction effect between columns and links ( F(2,36) = 4.14, p <.02 ). From this effect it can be concluded that relative to a two-column layout, a three-column layout takes less time to search when there are 155 links than it does when there are 30 links (along with other analagous statements about the mean). This implies that changes in column treatments may become more pronounced as as more links are added. This is a worthy discovery and could be considered grounds for further investigation. Additionally, the F-statistic for the links suggests that the number of links has an effect on search time ( F(1, 36)="14.68," p < .01 ). Although the F-value for links is very high, this effect was anticipated and is not of major interest.
Also, the two-way ANOVA implies that, in general, column
treatment does not have a significant effect on search time.
4.3 Anomalies:
Although the interaction effect in the two-way ANOVA is statistically significant, it is difficult to attribute the trends in the associated data to any physical or psychological phenomena. Instances of outlying data exist in almost all subsets of data. There is, perhaps, no obvious reason for this other than chance. Reasons can also be found in analysis of subjective satisfaction.
4.3.1 155-link Treatment Anomalies:
It can be noted from visual analysis that this particular data set indicates that having a three-column layout when dealing with a large number of links can lead to quicker search times than when dealing with one- or two-column layouts with a two-column layout being the slowest. This finding comes concurrently with subjective satisfaction results which show subjects prefer a three-column layout. This confounding data could be considered an anomaly accounted for by . . .
. . . too many links.
Although no statistics support the claim, on visual investigation of the subject, it was noticed that most tended to flounder through the link-data. Many subjects reported minor irritation at having to wade through a sea of "random links." These "random links" could lead to "random" (i.e.: higher) times. This is a problem recognized by other similar studies.. . . preconceived notion.
Most of the subjects have seen the lists of items before. Although they may not have seen them in the particular order presented by the experiment, it is possible that having seen the data before could hasten or delay the subject's think time. If for instance, a subject immediately identifies the list as "different from what is usual" the subject may proceed in a systematic way, to search the data set. On the other hand, if the subject does not identify the list as "different than what is usual" they may search a predetermined location and come up short- leading to confusion and reevaluation which could cost time.
Although the 155-link treatment does seem to affect search times, it is difficult to tell why since the graph of mean times does not follow any simple curve. One could argue that further investigation of a possible trend could yield something more succinct with more testing. It is also possible that an increase in the data set size could reduce the variance among users and lead to something more manageable. In any case, there is reason to perhaps re-enact the 155-link portion of this experiment.
4.3.2 30-link Treatment Anomalies:
Statistically, the 30-link arrangement data shows that different column layouts have no effect on search times. Visual assessment of the 30-link data set shows that, at least in this case, there is a slight trend: "more columns means slower search time." This is evidenced by the task search times bar graph. Again, this is contrary to the original hypothesis and is considered an anomaly. This could be due to . . .
. . . time lost shifting gaze:
It is not hard to envision a situation where a small number of items falls within a relatively small search area. It makes sense that splitting this area into separate areas will lead to longer search times since the eye has to readjust numerous times.. . . need for developed strategy:
It is also possible that the more complex a list (i.e.: the more columns it has) the longer it takes a user to develop an initial plan of attack for the search. In other words, a user stumbling upon a single-column of output has no decisions to make regarding search technique (aside from the top-bottom, bottom-top issue). However, a user stumbling on a list of multiple columns may have to initially decide whether to go across or down. Even worse, it is possible that given two such strategies, the user will change strategies half-way through leading to a longer search time.
It is unlikely that further recreations of the 30-link portion of the experiment would yield results to support the hypothesis.
4.4 Participant Comments:
Although there was no analysis of variance of subjective data, the data shows that participants prefer a three-column layout to a one- or two- column layout. Subjects also seemed to favor fewer links to more links.
One-third of the subjects commented on the link arrangements not being alphabetized or categorized. Although such an implementation was not appropriate for this experiment, it hints at the user's desire for web-link organization and raises questions about further experiments. . .
Calculations by Marc's hack
Forward to Conclusion
Back to Results
Back to Title Page