OverviewUsability testing employee productivity.

Role: User Researcher / Tester, Team Leader

Tasks: Heuristic Evaluation, Usability Testing, Keystroke Analysis

Keywords: Quantitative · SharePoint · Interviews · Qualitative · Research · Test Scripts · Feedback

Description: Using usability methods to keep employee productivity up during a global technology deployment.

Chevron is one of the world's largest oil companies. They are active in more than 180 companies. More information about the project can be found at: www.computerworlduk.com.

 Problem

Global Information Link 3

Global Information Link (GIL) 3 was an IT project that set out to standardize and upgrade desktop tools, operating systems, and hardware of company workstations. The new tools would include Vista and SharePoint

Leadership fears

Leadership teams feared productivity losses due to usability issues associated with the new software.


 Strategy

Metrics

My strategy was to create and execute a test plan focused on gathering quantitative and qualitative metrics prior to the full system release.

Plans

I created two sets of user testing activities for both the small and larger pilot releases.


 Getting Started

Building a team

With so much testing, I needed help to coordinate and carry out testing in multiple locations.

Recruitment

I recruited internally, from graduates of the Human Factors International (HFI) course in usability.

External team

I also worked with contractors, interns, and development team members, who all provided invaluable support.

Outside support

I also found additional guidance from a human faculty member at the University of Houston.


Execution

 Heuristic Evaluation

Performed a heuristic evaluation/expert review of an “out‐of‐the‐box” installation of SharePoint.

 User Interviews

Conducted user interviews at employee workstations to understand how well users could use SharePoint.

 Test Scripts

Wrote test scripts for an online testing interface, for pilot participants.

 Usability Testing Facilitation

Led user testing in locations in Houston, TX and at headquarters in San Ramon, CA.

 Feedback

Created an an online feedback tool to gather qualitative and quantitative measurements during pilot releases.

 Analysis

Provided quantitative analysis of testing, showing training improved user productivity with the new system.

 Education

Created a Usability 101 document, to help introduce other Chevron employees to user experience.

Online Testing Tool

Each week, pilot participants received a series of testing scripts for a different set of functionality. Each test was a measure completion rate and satisfaction.

limited pilot

We were able to show that despite certain undesirable qualities of a standard SharePoint installation, employees who attended training courses achieved productivity quickly. This helped deployment teams plan for a “high‐touch” approach.