We recently ran our first large-scale usability study for the hotel industry.
We benchmarked the ten biggest international hotel chains by running a large-scale usability test with more than 1,000 users.
If you own or run a hotel or any website that involves availability and booking, or you are just a fan of UX and customer experience, you will find the following data extremely useful. Enjoy!
We asked users to perform a typical scenario that included three tasks:
1. Search for availability and pricing for a room for two adults and two kids (five and six years old) for four nights in Barcelona (or any other city relevant to the hotel chain).
2. Go through the booking process until you reach the final confirmation button.
3. Find the address of the hotel.
For each task, we measured the completion rate, meaning how many users succeeded in finishing the task, along with the completion time. We also asked users to rate their experience for each task on a 1-7 scale from "very difficult" to "very easy".
We also included general questions about the aesthetics and delightfulness of the website.
Finally, we added the SUS (System Usability Scale) and NPS (Net Promoter Score) questionnares.
The first task involved searching for prices and availability for a room for four people (two adults and two kids). The absolute winner was choicehotels.com with an average completion time of 2 minutes and 20 seconds. Bestwestern.com, ihg.com, and novotel.com were pretty good as well with an average time below 3 minutes. Sol-hotels.com, hyatt.com, hilton.com, achotels.marriott.com, and radissonblu.com followed with completion times from 3 to 4 minutes. Finally, tryphotels.com had an average completion time of 5 minutes and 42 seconds, indicating a serious issue with their search process.
Test completion rates show that ihg.com has made the process fail-proof with a 97,12% completion rate. The rest follow close behind, while hyatt.com and radissonblu.com had only about a 60% completion rate. This means that 40% of the people who tried to accomplish the task, did not succeed. The problem is even more prominent at tryphotels.com, where only 33% of users managed to complete the task!
While time to complete and completion ratio are objective metrics, ease of use as rated by users is subjective and influenced by factors like aesthetics and the overall functionality of the website. In fact, since users don't measure their own completion times, this metric is more user-centric as it shows how users feel about the process.
After finding room prices and availability, the next task was the booking (or checkout) process.
Again, choicehotels.com had the fastest booking process, with an average time of 1 minute and 53 seconds. Very close behind was hyatt.com, which was eighth in time completion for the first task, along with tryphotels.com, which was tenth in the previous task!
However, as the following graph shows, hyatt.com and tryphotels.com also had very low completion rates, along with radissonblu.com. That means that it's only quick to checkout if you can figure out how.
Again, the final judge is the user. Users liked the checkout process of choicehotels.com a lot, and didn't like those of hyatt.com, radissonblu.com, or tryphotels.com at all.
The third task we asked users to complete was to find the hotel address, a typical step after booking.
Bestwestern.com had an average time of 25 seconds and was the leader of this task, with radissonblu.com following at 29 seconds. For this task, sol-hotels.com and novotel.com seemed to have the most problems, as it took about 1 minute and 15 seconds for users to find the address.
The above ranking is similarly reflected in the task completion rates. While 99% of users managed to find the hotel address on ihg.com, only 83% could do the same on hyatt.com.
Users rightly rated bestwestern.com and achotels.marriott.com highest on this task.
After completing the above tasks, users were asked to rate their overall experience based on the delight to use and website attractiveness or aesthetics.
We also asked them the NPS question, how likely they were to recommend the website to friends. The following graph shows the average of the NPS scores; although this is not the formal way of calculating NPS, it gives a clear picture as to how the hotel websites rank on this aspect.
Finally, we asked users the 10-item SUS questionnaire that measures the perceived usability of the website. What was interesting here is that the SUS ranking was strongly correlated to the NPS ranking. In addition, the SUS scores reflect the overall ratings and completion times we recorded in the previous tasks.
If you would like to measure your hotel's website, please contact us.
Create user tests using a better tool. No monthly fees. No credit card required to start.
Start testing
Discover how powerful Userfeel is for doing user testing more effectively and ask us anything.
Book a demo call
