Introduction
The Web Services team is a user and data led team. We decided that a great time to do user testing, with new students, was during Freshers’ Week.
What we did
A survey was produced to test
- Open Day attendance,
- the new 360º Virtual Tour,
- our communications during the admissions process,
- social media,
- matriculation,
- the website overall.
Respondents were also asked what type of student they were (UG, PGT, PhD or Exchange) and what their domicile was (Scotland, EU, England, Wales, Northern Ireland or International). This was to allow the data to be filtered.
How we did it
From Monday 5 September to Monday 12 September 2016 there were 30 people from across External Relations going out to speak with students and ask them to complete the survey. We focused our attention on Bonar Hall, DUSA, Enquiry Centre and the Library as these areas had a high concentration of new students.
Each student was handed an iPad Mini and asked to complete the survey which took 3-5 minutes to complete.
Results
We targeted 500 responses and there were 575 students completed the survey.
Student Type
| Undergraduate | 429 |
| Postgraduate | 115 |
| Exchange | 26 |
| PhD | 5 |
Domicile
| Scotland | 308 |
| International | 106 |
| EU | 83 |
| England | 51 |
| Northern Ireland | 26 |
| Wales | 1 |
Open Days
| Did you attend one of our Open Days? | Total | % |
| No | 323 | 56% |
| Yes | 252 | 44% |
| Grand Total | 575 | 100% |
360º Virtual Tour
| Did you use the 360 virtual tour? | Total | % |
| No | 441 | 77% |
| Yes | 134 | 23% |
| Grand Total | 575 | 100% |
| Did you find the 360 virtual tour useful? | Total | % |
| No | 11 | 8% |
| Yes | 123 | 92% |
| Grand Total | 134 | 100% |
| How would you rate the 360 virtual tour? | Total | % |
| 0 | 2 | 2% |
| 50 | 21 | 17% |
| 100 | 103 | 82% |
| Grand Total | 126 | 100% |
Communications
| Before arriving, what did you think of our communication? | Total | % |
| 0 | 26 | 4.52% |
| 50 | 126 | 21.91% |
| 100 | 423 | 73.57% |
| Grand Total | 575 | 100% |
| What did you think of the way we spoke to you? | Total | % |
| 0 | 9 | 2% |
| 50 | 42 | 7% |
| 100 | 524 | 91% |
| Grand Total | 575 | 100% |
| If you applied to more than one university, how did the amount of communication from us compare? | Total | % |
| 0 | 17 | 3% |
| 50 | 156 | 30% |
| 100 | 352 | 67% |
| (blank) | 0% | |
| Grand Total | 525 | 100% |
Social Media
| Thinking about social media, how many Facebook groups were you asked to join? | Total | % |
| 0 | 104 | 18% |
| 1 | 115 | 20% |
| 2 | 132 | 23% |
| 3 | 106 | 18% |
| 4 | 9 | 2% |
| 4+ | 109 | 19% |
| Grand Total | 575 | 100% |
| What is your favourite social media platform? | Total | % |
| 388 | 67% | |
| 68 | 12% | |
| 62 | 11% | |
| Snapchat | 40 | 7% |
| I don’t use social media | 11 | 2% |
| 6 | 1% | |
| Grand Total | 575 | 100% |
| What is your preferred method of communication? | Total | % |
| 421 | 73% | |
| Social media | 139 | 24% |
| Postal letter | 15 | 3% |
| Grand Total | 575 | 100% |
Matriculation
| Had you ever heard of the word “Matriculation” before coming to Dundee? | Total | % |
| No | 250 | 43% |
| Yes | 325 | 57% |
| Grand Total | 575 | 100% |
| How did you know where to be today for matriculation? | Total | % |
| Asked a friend | 114 | 20% |
| Asked a member of staff | 59 | 10% |
| Email with a link to website | 132 | 23% |
| Email with all the information | 161 | 28% |
| Social media | 10 | 2% |
| Website | 99 | 17% |
| Grand Total | 575 | 100% |
Website
| Overall, how well did our website meet your needs? | Total | % |
| 0 | 6 | 1% |
| 50 | 106 | 18% |
| 100 | 463 | 81% |
| Grand Total | 575 | 100% |
| How easy was it to find what you were looking for on our website? | Total | % |
| 0 | 20 | 3% |
| 50 | 186 | 32% |
| 100 | 369 | 64% |
| Grand Total | 575 | 100% |
| How visually appealing is our website? | Total | % |
| 0 | 15 | 3% |
| 50 | 136 | 24% |
| 100 | 424 | 74% |
| Grand Total | 575 | 100% |
| How easy is it to understand the information on our website? | Total | % |
| 0 | 10 | 2% |
| 50 | 111 | 19% |
| 100 | 454 | 79% |
| Grand Total | 575 | 100% |
| How much do you trust the information on our website? | Total | % |
| 0 | 7 | 1% |
| 50 | 56 | 10% |
| 100 | 512 | 89% |
| Grand Total | 575 | 100% |
| How likely would you be to recommend our website to someone else? | Total | % |
| 0 | 9 | 2% |
| 50 | 110 | 19% |
| 100 | 456 | 79% |
| Grand Total | 575 | 100% |
How will the results affect change
If we can get prospective students to campus they are more likely to convert to matriculated students. This is proven from data from previous Open Days. Only Scottish respondents to the survey were in the majority that attended an Open Day. However, 92% of those that used the Virtual Tour found it useful. Work is underway to adapt the Virtual Tour for Virtual Reality headsets to be used at UCAS events to bring the students to Dundee in a fun and interactive way.
Looking at the communication section of the survey it can be seen that just over 26% of respondents were not completely satisfied with out communications.
Regarding social media, we can see that over 20% of respondents were asked to join 4 or more Facebook groups but we didn’t ask if that was positive or negative. That additional question shall be added to the survey for next year.
43% of respondents overall, rising to 50% of undergraduate respondents, have not heard the word ‘matriculation’ before coming to Dundee. This would suggest that using plain English would reduce confusion during Freshers’ Week for what is essentially registration.
40% of respondents had consulted the website to find out where matriculation took place but 30% asked a friend or staff member. Student feedback showed a lack of awareness and confusion about the matriculation process. Frequently, students wanted to feel reassured they were doing the ‘right thing’.
The website section shows that over 80% of respondents are happy with the website meeting their needs. Conversely, we need to be aware that one-third were having difficulty finding what they are looking for. This will inform work that I am looking at this semester. Using tools such as Optimal Workshop Treejack and OptimalSort, I will be investigating how we can improve the information architecture of the Study section and the website overall.
[…] couple of weeks ago The University of Dundee’s external relations team released data from some guerrilla usability testing they had conducted during their freshers’ week. It was […]
Hey Rob – it’s great to see Universities jumping into the sphere and more specifically, publishing the results of the test.
It would have been nice to see if you’re able to pull some UNG or Pg into the testing experience – and by that I mean, have them help conduct the studies – perhaps HCI students?
I do ask though, how did you go about user testing the students? Or was this study survey based?
Thanks!
Hi Preston.
Thanks for taking time out of your day to comment.
For this round of testing we had staff from our directorate out doing the testing. I felt it was important for them to speak to students directly. This testing was survey based. If you look at other posts in the UX category, you will see the top tasks workshops that we did late last year. Also take a look at our alpha website alpha.dundee.ac.uk
Cheers