Monday, June 2, 2008

Week 3, Post 5: Project Plan Draft

Project Plan

Write a recommendation report based on the results of the usability study in order to assist in creating a more effective and efficient Office of Sponsored Programs website and Limited Submission Program.

Step One: Associate Research Deans, Committee, and IRB approve proposal

  • In conjunction with step 1 as this is needed for approval

  • Finalize the instruments so that they are ready to be used for the study.

  • Step Two: Create Instruments for study

  • Pre-and-Post test survey instruments

  • Scenarios for task-based analysis

  • Confidentiality releases to participate in study

  • Confidentiality releases to be videotaped
  • Step Three: Preparing to conduct usability study

  • Recruit participants
    • URGC reviewers who used Limited Submission Program
    • Clemson faculty and staff
    • Recruit through word of mouth, email invitation, and phone calls
  • Reserve area in which to conduct the study (409 Daniel – Usability Lab, or meet in a neutral location for participants, such as their office)

  • Reserve appropriate equipment/make sure is available
    • Digital video camera, tripod, blue tooth recorder, headphones

    • Morae software

    • Adobe Premire for creating highlight reels

    Step Four: Conduct Usability Test

    Step Five: Analyze data and create recommendation report

    Methodology

    In conducting a usability test of the Limited Submission Program, this will require the perspective of both the reviewers and authors. To collect this data, pre- and post- test interviews, direct observation, and a read-aloud protocol that is video recorded would be the best means of data collection.

    A thorough usability test of the Limited Submission Program would include using those data collection methods listed above with the proposal authors and the reviewers. The pre- and post-interviews will yield perceptions, opinions and suggestions on behalf of the participants. This kind of information proves useful in making recommendations and changes to the writing and reviewing process completed internally at Clemson University. The direct observation and talk-aloud protocol allows for a real-time perspective of the challenges that are encountered when using the Limited Submission Program. Video recording the sessions is helpful in that the sessions can be reviewed, and video clips can be used to create a highlight reel that has potential to be used as a teaching tool in workshops geared especially towards new faculty.

    To conduct the actual usability test, the software Morae would be used. An eye-tracking device that calculates, graphs, and codes usability measurements, this software automatically records the session and allows the researcher to code the information in real-time. The advantages of this software include that it is easy to use; it will give a true account of using an online program such as the Limited Submission Program; and problems and challenges that are encountered while using the Limited Submission Program will be captured, recorded, coded and compared to other users ‘experiences.

    The purpose of this usability test is to improve the efficiency and ease of use of the Limited Submission Program. In doing so, it becomes more cost effective in regards to authors and reviewers time if they are able to maneuver through the process quickly.

    Provisional Project Timeline

    June 4 - submit short proposal to Dr. Galliciho for approval to conduct usability test on OSP website

    June 3 - submit thesis proposal to chair and committee

    July 11 - submit paperwork to IRB

    July 21 - begin recruiting participants

    July 30 - begin usability testing

    August - continue usability testing

    September - analyze data and begin writing recommendation report

    October - defend

    (Maybe too optimistic. It is kind of dependent if it even gets approved by Gallichio in the first place)



    July - [hopefully] hear if



    2 comments:

    Diana Thrasher said...

    Katie,

    Your project is one that is much needed at Clemson and the timing could not be better. Your project steps have substance and from reading your steps you have a clear plan. Your current situation is a rhetorically sound document. When doing your usability tests you may want to enlist some of the college grants coordinators suggestions of particpants. They will probably have several names of individuals with divers backgrounds that will be beneficial to your study.

    Mike Hovan said...

    A few things.

    1) Your project sounds ambitious and extremely useful! I'm sure Dr. Howard will be able to give you some good pointers in structuring this study. We learned in his proposal writing seminar (and research seminar, too) that a recommendation report for future changes is often more useful than simply "doing a task." I'm also sure that this will be a tremendous feather in your cap when it comes time for a job hunt. Indeed, I'm sure you could use this study for future academic studies if you chose to do so,

    2) At first I wanted to know more about the data crunching, but I'm sure you're just refining that right now. Also, how long do you think it will take to crunch the data? Do you think you'll be able to do it in a month? Don't hesitate to call on some other "MAPC eyes" if you need assistance! (I guess you would then have to train them, though, so that might not be a good idea).