3.1 Method
Through the server log data gathering it was able to follow the path each individual subject followed in performing each specific task.
The mean access time and mean number of errors per model were caluclated by first getting the average for the two tasks each subject performed on each model. Then the ANOVA analysis on these means was performed.
Values for wrongly performed tasks (i.e. wrong final answer) were excluded from the calsulations.
What follows is a presentation of the final results. All data are presented as appendices to this report.
3.2 Subjective Ranking: Difficulty of Use
The mean difficulty of use rankings are presented in Table 1 and plotted in Graph 1.
| Table 1: Mean difficulty of use Rankings | |||||
|---|---|---|---|---|---|
| Tree structure | Model 1 | Model 2 | Model 3 | Model 4 | Model 5 |
| Ranking | 4.06 (1.14) | 2.94 (1.25) | 3.06 (1.30) | 2.65 (1.53) | 2.30 (1.36) |
The ANOVA for the Ranking was significant [F(4,16)=3.40, p<0.05]
Graph 1: Ranking of Difficulty
3.3 Performance Measures: Speed and Accuracy of Information retrieval
The average total task time for each model was determined using a one-way analysis of variance. By total task time, we refer to the the total time it takes for a subject to locate the specific information asked against the corresponding models. Since each of our subjects performed two tasks per model, the average time per subject per model was calculated before doing an ANOVA analysis on these averages.
Table 2 shows a summary of the results, and graph 2 plots these data.
| Table 2: Mean total task times | |||||
|---|---|---|---|---|---|
| Tree structure | Model 1 | Model 2 | Model 3 | Model 4 | Model 5 |
| Mean Total task time | 36.03 (14.87) | 32.21 (27.92) | 19.78 (11.85) | 20.97 (11.27) | 32.41 (19.80) |
Graph 2: Average access time per Model
The ANOVA for task time was significant [F(4,16)=3.20, p<0.05]
The accuracy of information retrieval was analyzed using the total number of errors for each model (tree structure). The analysis of variance for mean number of errors was not significant [F(4,16)=1.29, p>0.05]
The mean number of error is presented in Table 3, the same data is plotted on graph 3
| Table 3: Mean number of errors | |||||
|---|---|---|---|---|---|
| Tree structure | Model 1 | Model 2 | Model 3 | Model 4 | Model 5 |
| Average | 0.94 (1.43) | 1.53 (1.97) | 0.71 (1.16) | 0.41 (0.94) | 1.06 (1.56) |
Graph 3: Mean Number of Errors per Model