Making a website work more efficiently
A Comparative Benchmark Study of Two Websites
At a Glance
Problem: Prospective volunteers call and email Program Managers at United Planet and WorldTeach because they can’t find the answers to their questions on the website. Program Managers are overwhelmed by the amount of communication they need to respond to making it difficult to provide timely answers and distracting them from their other work.
Solution: Evaluate which website currently presents key information in the most efficient and clearest manner so that best practices can be gleaned and employed on both websites.
Context: graduate class project, working in a group with 2 other members
Role: designed study in UserZoom, performed quantitative analysis, designed final presentation
Time: 4 weeks
Lesson: Two websites may share similar design aesthetic, but if content strategy or information architecture is lacking, it will significantly impact usability and findability of information
REMOTE USABILITY TESTING // QUANTITATIVE ANALYSIS // INFORMATION ARCHITECTURE // CONTENT STRATEGY
Background
United Planet and WorldTeach are both non-profit organizations based in Boston, MA that offer volunteer abroad programs. They offer programs in some of the same countries, such as Colombia, Chile, Thailand, and South Africa. Programmatically, they are very similar in that the they take responsibility for pre-departure and in-country arrangements in exchange for a fee paid by the volunteer. Program managers and coordinators at each organization field many calls and emails everyday from volunteers with questions about costs, flights, and application dates. If this information were clearly presented on the website, it could reduce emails and calls and the time spent on answering them, and allow program managers and coordinators to focus on other, more important aspects of running the programs.
Methodology
An unmoderated, remote, between-subjects usability study was run for a 1-week time span. The study had 30 participants use their own devices to complete tasks through the UserZoom platform. Participants were all between 18 and 44 years old and were recruited through our personal networks.
Participants were split equally between each website so that 15 were randomly selected to go to WorldTeach’s website and 15 were randomly selected to go to United Planet’s website. The study was task-based and success was measured by the answer to a validation question. Each participant was presented with the same 4 tasks, regardless of which website they were on and they were also asked to rate the site on a number of variables at the conclusion of the study.
The four tasks were:
Task 1: Use the website to find out how many countries [Organization] currently has programs in.
Task 2: Use the website to find how much [Organization] charges to volunteer for a year in Ecuador.
Task 3: You're interested in volunteering in the South Africa summer program. Use the [Organization’s] website to find out if flights are included in the cost of the program.
Task 4: You want to participate in the year-long program in Colombia that departs in January. Use the [Organization] website to find out when is the latest that you can apply.
Success data, time on task data, and self-reported data were analyzed in this task-based UX benchmark study to compare the findability and clarity of information to understand which site has executed their information architecture (IA) and content strategy more successfully.
Key Findings
- United Planet was statistically superior in most aspects of this study, primarily regarding time on task, task success, and the Net Promoter Score.
- Approximately half of the time, participants on WorldTeach were not able to successfully find the answers they were looking for, or successfully solve their query. This has huge implications for users of this website and their business operations, because successful operation of the website can be a key recruiting tool. If users are unable to use the website to successfully find answers to their questions, they may not be able to apply to volunteer or may be less inclined to continue through the process.
- Furthermore, WorldTeach is missing out on some pivotal opportunities because their website is lacking in key areas. For example, the age group of 18-24 year olds—the primary target group for both companies—had a significantly poorer average task success when using the WorldTeach website.
- Although the average task success was significantly higher for participants using the United Planet website, it was important to show the success rate on an individual task level, which demonstrated that the task success rate for WorldTeach was significantly lower on all tasks, as compared to United Planet.
- United Planet did not have any glaring areas for improvement based on this study, but a study on a larger scale (and a more qualitative focus) could shed light on areas that they could still make better for users.
- Since the statistical data as well as the comments indicated the information was difficult to find on the WorldTeach website, starting with the information architecture would be a logical place to start restructuring the website.
Our final presentation with detailed findings:
