|
Full vs. Limited Context Hierarchical Menu Structures for Web Site Navigation with Frames |
|
2. EXPERIMENT 2.1 Hypothesis Zaphiris and others before him have proven that broad-shallow menu structures are more efficient than narrow-deep ones. The hypothesis of this experiment is that within a frame structure, the full context menu structure is more effective for web site navigation than the limited context menu. This means that given attention to past research on hierarchical menu performance and design, the full context menu structure will perform more efficiently than the limited context menu with respect to user completion time of task and number of user errors. Moreover, it is hypothesized that users will prefer the full context menu to the limited context menu because its full view of the menu hierarchy at current and previous levels gives the user a better sense of orientation. In addition, the full context menu allows users to traverse other visible levels of the menu hierarchy without backtracking. The independent variable for the experiment is the method of navigating a web site using a framed hierarchical menu structure. For this experiment, the independent variable has two treatments. One is the utilization of a full context menu structure. The other is the utilization of a limited context menu structure. There are three dependent variables that are measured for statistical significance in this experiment. The first is subject completion time, which is the amount of time it takes for a subject to complete one task list. The second is subject accuracy of retrieval, which is the difference in the number of unique and total links visited in completing one task list and the number of links on the "optimal" path of that same task list. The third and last dependent variable measured is the subject's ranking of the ease of navigation for each treatment of the menu structure. 2.2 Pilot Study After a three subject pilot test was conducted, a few concerns were noted regarding the layout of the screen and experiment tasks. Among other things, these concerns included problems with the task lists' questions. The subjects found questions like "What is the price of the autograph 8x10 of John Elway?" too confusing. This may be attributed to the fact that some people did not know that John Elway was a football star. Since the experiment's main focus is ease of navigation, it was decided that the questions should have a very low level of difficulty. After all, the experiment is testing the efficiency of the interface and not the intelligence of the subjects. So the questions were rephrased to give them a better idea of how to search for the answers. The previous question became "What is the price of the autograph 8x10 of football star John Elway?" Subjects also complained that the titles of the categories like "Collectibles" and "Coins" where ambiguous. Originally the tasks questions were rephrased further. It was then realized that task related grouping guidelines (Shniederman 1998) were being violated. So it was decided that "Collectibles" category be broken up and placed into separate categories like "Sports" and "Toys". The last problem to be corrected dealt with the navigation of the limited context menu. The subjects found it irritating to use the "back" button on the Netscape browser for backtracking to higher levels in the menu hierarchy. To correct this problem, links were added to all levels to help in navigation. As for time, on average, it took about 12 - 17 minutes to run the whole experiment on one subject. As a result, it was decided to institute a time of 20 minutes as the maximum for completing the whole experiment. 2.3 Subjects There were ten subjects used in the experiment. Seven of them were undergraduate students with at least one year of experience on the Web. The remaining three were information systems professionals. The majority of them had encountered frames for web site navigation, and within that grouping less than half actually preferred frames for navigation. The rest either had no specific preference for site navigation or did not prefer frames at all. Unfortunately, the reason for the low turnout was that the 15-20 minutes it took to complete the experiment discouraged many potential subjects. 2.4 Materials Unlike Zaphiris' experiment, which utilized a large web site based on Census Bureau information, the web site of this experiment is a small-scale imitation of the online auction house Ebay (see www.ebay.com). Because of this choice, it was not difficult to design a menu structure that is based on existing design research mentioned in the introduction. The hierarchical menu of this experiment's interface has brief but intuitive item names; a broad-shallow 8^3 asymmetrical design (a maximum of three levels with a maximum of eight items per level). Furthermore, an emphasis on task-related grouping and presentation sequence (Shniederman 1998) was incorporated into the design. Other materials used for experiment, other than the web site, included, for one, subject background surveys. These surveys were given to each subject at the beginning of the experiment to collect information on how much experience each has had with the Web. The users were also given two distinct task lists (one for each treatment) with ten short answer questions each. These tasks were to be completed during each phase of the experiment by navigating the site in search of the answers. The actual purpose of the task lists was to give users motivation to navigate the web site so that their accuracy of information retrieval data could be gathered and analyzed. Also, two questionnaires (again, one for each treatment) were given to the users at the end of each phase of the experiment to gather information on the user's evaluation of the each menu structure. And lastly, since the web site was optimized for the software, Netscape's Navigator web browser was used to conduct the experiment. 2.5 Procedures The experiment began with subjects reading over an introduction sheet that explained and described the nature of the experiment. Once they completed the introduction, they were given time to ask any questions they might have before agreeing to do the experiment. Once they agreed, they were asked to sign a consent form before the experiment got under way. After signing the consent form and taking care of the particulars, subjects were then ready to begin the first of the two phases of the experiment. In phase one, with Netscape Navigator, a subject would search the web site using one menu structure in an attempt to complete the corresponding task list. In the second phase, they would do the same just with the other menu structure. It should be noted that each subject was started on a different menu structure than the last subject. This way the data would not be biased because of the subject's familiarity with the site by the second phase. For each phase, the subjects were first given a sheet of instructions on how to complete the task at hand. They were then given a training session on the how to correctly operate menu structure they were to use. Following that, they were given the task list and then instructed to click "start" on the current phase's start page when they were ready to begin. As soon as subjects clicked on "start", their start time was recorded from a Javascript activated timer. The subjects were given no help as soon as they started on the current task list. The instructions given to them stated that they were to complete the task list fully and then click on "done" when they were finished with the last task. After they clicked on "done", their completion time was recorded from another timer that was activated. At the end of each phase, the subjects were to complete an evaluation of that phase and its menu structure. The scale used to rate the menu structure with respect to ease of navigation was 1 to 10; with "1" being the worst and "10" the best/highest rating. While they were completing their evaluation of the current phase, the history of the browser was saved because it contained the subject's path as they navigated the site. This information was later analyzed for the subject's accuracy of information retrieval. |
 |
Department of Computer Sciences Direct questions and comments to the student editorial team |