Comparison of Photo Library Programs

Experiment

Introduction and Hypothesis

There are different photo libraries that users can purchase or download off of the internet. These photo libraries have different organization, presentation, and navigation capabilities.

In our experiment we compare three different photo library programs. ACDSee is a program which users can download off of the internet. PhotoSuite III is a program which users can purchase in stores. PhotoFinder is a program which the University of Maryland Human Computer Interactions Laboratory is currently making. Our experiment compares the photo library programs in terms of ease of finding a photo in a collection.

We hypothesize that it will be easier to navigate and browse through PhotoFinder than PhotoSuite III or ACDSee. Users will be able to finish the tasks more quickly using PhotoFinder. We feel that users will be able to finish the tasks more quickly because PhotoFinder consists of three frames: the library viewer, the collection viewer, and the photo viewer. Users will not get lost in PhotoFinder, whereas in PhotoSuite III, users may forget which collection they are in. The organization of PhotoFinder also seems to be clearer than the organization of ACDSee. In ACDSee, users browse through a directory format similar to Windows Explorer. This can be misleading because not all directories will have collections of photos. In PhotoFinder, users can view the collections quickly because they are shown in the collection viewer frame.

Variables

The independent variable for our experiment was the photo library program. There were three treatments for the independent variable - ACDSee, PhotoFinder, and PhotoSuite III. Our dependent variables were time to complete the tasks and a subjective evaluation of the program.

Pilot Study Results

We used a total of three subjects for pilot testing - one for each photo library. Several changes were made to our experiment as a result. First, we decided to make a sample task for our subjects. The sample task allows our subjects to have a chance to use the photo libraries before starting the five tasks for our experiment. Second, rather than a list of tasks given to the subjects, we decided to make task cards - one task on each 3x5 card. This was to prevent the users from reading the tasks ahead of time. Third, for our fourth task (In the CHI 97 collection, find a picture of the man in photo #3.), there were multiple pictures of the man in the collection. We decided to take out all pictures of the man except for one. All subjects would then find the same picture of the man. Our final changes were to take out some questions from the background survey and the subjective satisfaction survey. Certain questions were taken out because they would provide no insight to our experiment.

Subjects

We decided that we would try to get 30 subjects for our experiment. Ten subjects would use ACDSee, ten subjects would use PhotoFinder, and ten subjects would use PhotoSuite III. We went to to three art classes (ARTT353 - Elements of Photography, ARTT354 - Elements of Computer Graphics, and ARTT458 - Graphic Design and Illustration: Advanced), to recruit subjects for our experiment. We also recruited students from CMSC311 (Computer Organization) and CMSC420 (Data Structures). In total, we had 26 subjects who were students from the University of Maryland. We had seven subjects for ACDSee, nine subjects for PhotoFinder, and ten subjects for PhotoSuite III.

Materials

Our experiment was administered on three Windows NT computers located in the Human Computer Interactions Laboratory (HCIL). Each computer had a 17 inch monitor with resolution set to 1280 x 1024. An introductory information sheet, that provided a brief overview of the experiment, was supplied to the subject. The subject was then asked to complete a background information survey that was designed in Visual Basic to automatically create tallies of the results. The background survey asked for information about age, gender, computer usage, and previous photo library experience. Before the experiment the subjects were given a brief tutorial for the program they would be working with. The tutorial covered the basic commands and operations they would need to complete the tasks and concluded with a sample task of locating a photo.

The photo collections used in the experiment were created using photos from the HCIL as well as some personal photos. The photo collections were designed to include a variety of subject matter with landscape, portrait, indoor, and outdoor photos. The collections were also created with different numbers of photos to create the size variety that might be present among personal photo collections. The photos were all stored on one of the HCIL computers in directories that corresponded to their associated collection.

Three photo library programs were used in our experiment: ACDSee, PhotoSuite III, and PhotoFinder. The first two photo library programs were selected because they are two of the most commercially popular programs. The PhotoFinder program is a part of an ongoing research project in the HCIL. The three programs were also selected because each program has a unique way of displaying and navigating the photos. We divided our subjects between these programs and had them complete five timed photo-locating tasks among the various collections.

After the subjects completed the experiment, they were asked to complete a subjective satisfaction survey. They survey was of the same Visual Basic format as the background survey and tallied the results to the same file. The subjective satisfaction survey asked subjects five questions all associated with their satisfaction with various aspects of the photo library program.

Procedures and Problems

Procedures

We used the computers in the HCIL lab in order to conduct our experiments. Before our subjects arrived, we set up our survey program and the photo library programs. Once the subjects arrived, they were asked to sign the Experimental Consent Agreement, and they were given a sheet of paper with the introduction written on it. They were then asked to fill out the background survey from the survey program, and the program was then minimized.

The photo library program was opened up, and we gave them a brief training period on how to browse and navigate through the program. We then explained that they would be given a sample task in order to familiarize themselves with the program, and afterwards, they would be given five tasks that would be timed, one at a time.

We verbally told the subjects what the sample task was. The sample task was untimed. Before each timed task, we gave the subjects the task card along with a photo (if necessary), and explained the task to them. We also set up the collections for each task before the subjects were asked to perform the task. We timed each task manually. When the subjects finished a task, they were asked to bring the photo to full-size, and asked to say that they were done. For task #2, they were only required to say "yes" or "no."

When the subjects finished all five tasks, we brought up the survey program again, and they were asked to fill out the subjective survey. Once they were done with this, we asked if there was anything they particularly liked or disliked about the photo library program they worked with. We also asked if they had any questions. This concluded our experiment.

Problems

We ran into a few problems while running our experiment. For the collections where there were a large amount of pictures, subjects often "forgot" that they were allowed to use the slideshow. We had to remind a few of them that they could use the slideshow. This may be a problem because it might have affected the time it took for the users to finish the tasks. For our second task (Is picture #2 in the Utah collection?), one subject misunderstood the question, and did not realize that it was a yes or no question. For our third task (Within the Big collection, find the only photo that contains a band of musicians.), a few users asked if the picture actually existed within the collection, since it was so large. Many also asked if there were instruments in the picture. Some subjects looked doubtful. These were the problems that we had while running our experiment.