Evaluating the Usability Evaluation Methods for Software Process Descriptions

Information Sheet

In this study, we would like you to evaluate the usability of a Software Process Description (SPD). A software process description is defined as a representation of a software process created to facilitate process communication and enactment, and also to serve as a medium for process evaluation and improvement. The objects of evaluation for this study are electronic versions of SPDs for the Unified Process for Education (UPEDU) and Extreme Programming (XP).

You will use two Usability Evaluation Methods (UEMs): Question Asking Protocol (QAP) and Perspective-Based Usability Inspection (PBUI). The goal of these evaluation methods is to detect usability problems, which can help us to understand better the usability issues related to process descriptions. Analysis of the detected usability problems can help us decide which of the evaluation methods is more effective and efficient for evaluating the usability of SPDs.

QAP involves the collection of data about how users actually interact with the SPD by performing tasks in a given work environment (e.g. preparing a software test plan based on the supplied SPD). To complete the given tasks, we as the evaluator/tutor will assist users on a one-to-one basis via question-answering.

PBUI requires the inspector to examine usability-related aspects of the SPD to detect potential usability problems. The inspector focuses on usability issues covered by one usability perspective – in this study, process performer. This perspective also provides the inspector with a list of checklist/inspection questions that represent the usability issues to be checked.

We are conducting the evaluation in August 2009. The subjects for the evaluation are volunteers and participation is open to undergrad and post-grad students of the ITEE School. For data collection and analysis purposes, audio of the evaluation sessions will be recorded. The confidentiality of the subject data is assured and no material that could identify the subjects will be used in any reports.

The evaluation session is expected to take approximately 3 hours to complete (including a short break in between the two evaluation methods). If you have any questions regarding this study, please contact the researcher using the contact details below. You can withdraw from the study at any point without prejudice or penalty of any kind.

This study has been cleared in accordance with the ethical review processes of the University of Queensland. You are, of course, free to discuss your participation with project staff (Naz’ri Mahrin, contactable at: mahrin@itee.uq.edu.au – Mobile: 0405 546 445). If you would like to speak to an officer of the University not involved in the study, you may contact the School of Information Technology and Electrical Engineering Ethics Officer directly on 3365 3476, or contact the University of Queensland Ethics Officer on 3365 3924.