Comparison of Photo Library Programs

Discussion

 Our goal in this experiment was to compare three photo library software packages in two separate categories: browsing time and subjective satisfaction. We hypothesized that among the three packages, PhotoFinder would produce the fastest browsing times and provide the highest level of subjective user satisfaction. Our test results matched our initial hypothesis by producing a statistically significant difference between searching for photos using PhotoFinder, as compared to searching for photos using ACDSee of PhotoSuiteIII.

Task Times

We believe that PhotoFinder's three-frame layout made it easier for users to search for photos. When using PhotoFinder to view a collection of photos, the user can simultaneously view the entire collection in the top-right corner, a collection of thumbnails among the larger collection to be viewed in the top-left, as well as one full-size photo in the bottom-right corner of the screen. ACDSee and PhotoSuite III allow no such luxury of being able to view a collection, group, and full-size photo all on the screen at once. For instance, in ACDSee, if the user wished to view the images in full-size in "slideshow mode", the thumbnails disappeared. In fact, during my ACDSee subject testing, the subjects would frequently click out of "slideshow mode" after viewing two or three dozen images; my only guess would be that the subject forgot their place among the list of thumbnails and would have to go back to the thumbnail viewer. PhotoSuite III has a similar problem, to a lesser degree, in that only 10-15 thumbnails would appear on the screen at only one time. When traversing through large photo sets, it is very possible that the user could lose their place when only a small number of images appear on the screen. Furthermore, in large photo sets, it is necessary for the user to scroll through the thumbnails when using PhotoSuite III. The extra scrolling and clicking that ACDSee and PhotoSuite III require all take time. However, I believe that the greatest factor in the time difference between PhotoFinder and its two competitors lies in the extra short-term memory load that ACDSee and PhotoFinder require, by requiring the user to remember their place among large (over 30) collections of photos.

Tasks III, IV, and V all required our subjects to search through large collections of photos, and indeed, PhotoFinder produced faster search times in all three of these tests. Task III required out subjects to "find a photo containing a band of musicians" among a collection of over 200 photos. In this task, the subjects using PhotoFinder averaged 143 seconds, whereas the subjects using the ACDSee and PhotoSuite III averaged 179 and 207 seconds, respectively. We believe that PhotoFinder's increased performance in Task III was due to its ability to display a large number of thumbnails alongside a full-sized image. Unlike ACDSee's linear-search slideshow mode, this allowed users to go through a large list of thumbnails and click on the images with more than one person that "resembled" a group of musicians -- without the use of scrolling.

For Task IV, we gave our subjects a photo of a man's face and told the subject to find an image with the same man, among a collection of over 50 images. This time, the subjects that used PhotoFinder averaged 32 seconds, whereas the subjects using ACDSee and PhotoSuite III averaged 54 and 59 seconds, respectively. We believe that the results of Task IV further substantiate our claim that using PhotoFinder is less demanding on one's short-term memory load, by virtue of its multi-frame presentation. After all, in order to complete Task IV, the subject had to keep the facial features of the man in the photo in their memory, and compare the "picture" stored in one's memory to the images on the screen. When using PhotoFinder to complete this task, short-term memory not spent keeping track of one's place in ACDSee's slideshow mode or PhotoSuite III's thumbnail viewer allows the user to retain a clearer recollection of the man in the photo.

For our fifth and final task, our subjects were given a photo, and had to find an image that matched the photo among five different directories. Again, subjects using PhotoFinder had faster search times, with an average search time of 33 seconds as compared to ACDSee's 38 seconds and PhotoSuite III's whopping 103 second average search time. PhotoSuite III's poor performance in Task V was undoubtedly due to the lack of an elegant point-and-click "change directory" feature, which ACDSee had by virtue of its Windows-Explorer-like user interface and PhotoFinder had in its searching option located in the lower-left corner of the PhotoFinder interface.

However, for our two simple tasks (Tasks I and II), it is important to note that PhotoFinder didn't have the fastest search times. In Task I, where the user had to find an image among a small collection (25-30 images), PhotoFinder produced the slowest search times! The average search time of subjects using PhotoFinder was 14 seconds as compared to an average search time of 10 seconds with the other two photo library packages! We believe that PhotoFinder's smaller thumbnails contributed to its relatively poor performance in this simple task. In Task II, where the subjects had to compare a photograph against an entire a collection to determine whether the photograph matched any of the pictures in the photo set, PhotoFinder was bested by PhotoSuite, but still managed to beat out ACDSee. Once again, we believe that PhotoFinder's smaller thumbnails contributed to its poor performance.

PhotoFinder's complex interface may have contributed to its poor initial performance as well. The complex, multi-framed interface may have intimidated our subjects at the start of this experiment, as the subjects take the first few minutes to figure out "what frame does what", whereas ACDSee and PhotoSuite III have more familiar designs to Windows-experienced. ACDSee bears a strong resemblance to Windows Explorer in form and function, and as such, "experienced" and "expert" Windows users (which comprised 100 percent of our test subjects) could feel as if they already had experience with ACDSee, when in reality they were first-time users. PhotoSuite III's two-frame layout was very similar to many Web-based photo albums, with a strip of thumbnails on the right and the full-sized image on the left that occupied three-quarters of the screen. The presentation of images was simple, and the small number of thumbnails on the screen didn't overwhelm users at first glance. The simpler and more familiar layouts of ACDSee and PhotoSuite III, in our opinion, contributed to their strong initial performance in Tasks I and II.

In conclusion, we found PhotoFinder to be significantly better for searching through large collections of photos and complex searching tasks, whereas for small collections and simple tasks there was little or no difference between the three photo library programs.

Subjective Satisfaction

We hypothesized that users would prefer PhotoFinder in the satisfaction surveys.  However only one question proved to be statistically significant.  Question one ("Was it easy to find photos using...?") helps to back our hypothesis.  However, in questions two and five, although not statistically significant, PhotoFinder scored lower than the other photo library programs.  These mixed result are probably a result of the subjects personal preferences.  To really address this issue it would be helpful to run the experiment with a larger group of users and to have a more focused set of questions.

User Comments

PhotoSuite III:
- "Wish I could change the size of the thumbnails."
- "Pretty cool!"
- "The slideshow was good." (People thought the slideshow was fast and easy to use and convenient, and it was easier than browsing throught the thumbnails.)

PhotoFinder:
- "I really like the window layout for the different collections."
- "The thumbnails are too small when the collection is large."
- "The interface was confusing at first...but easy to pickup with a little practice."
- "I like how you can navigate the thumbnails using the arrow keys."

ACDSee:
- "The slide show feature is really useful to find photos in large collections."
- "I like the interface because I am already familiar with Windows."

Department of Computer Science: Direct questions and comments to the student editorial team