Yesterday Melina and I ran our first formal usability test in nearly 2 years. We had three students in the Mary I who worked on a few tasks using the Omeka-based Digital Collections site. We had a great crowd of observers who took meticulously detailed notes and helped us whittle down a very long list of issues to a few high-priority action items for the immediate future.
The 5 task-based scenarios we had each student work through were:
The first question was designed both to see how our users understood the term “Digital Collections,” as well as to see if they could find it. None of our students had any idea of what Digital Collections meant (most thought it was anything online, put into groups). But because they had an incorrect understanding of what it was, they certainly couldn’t find it.
We discussed how Digital Collections is the kind of resource that needs some kind of facilitation, to explain to users what the collections are and why they want to use them. Melanie noted that for our on-campus users who encounter digital collections through an assignment, they will either have Leigh or a liaison introduce them, or will at least have some context provided by the professor. But for others, we discussed a few ways to help clarify not just what Digital Collections are, but what all of our separate collections are. I think there are other opportunities here for sharing this information, from social media to Web ads. And navigating to our collections is also something I want to explore in future tests. We also want to explore how easy these collections are to in the Library Search - sample searches from the users in the test showed that a lot of other stuff came up before anything in our digital collections.
(As a related note: Weave Journal of Library User Experience recently published an article on this very topic, and found that while Digital Collections was a terrible term, it as the least terrible of all the others, and most libraries use it. ¯_(ツ)_/¯ Read the whole article: What We Talk About When We Talk About Digital Libraries: UX Approaches to Labeling Online Special Collections
Questions 2-5 all focused on specific, yet common, tasks in the Digital Collections system, which is a customized version of the open source tool Omeka. Let’s just summarize by saying that Omeka didn’t do so well on this test.
Many other issues centered around Omeka’s search function. Our users mostly made assumptions that Omeka’s search would work a lot like Google’s or Summon’s: autosuggest, autocorrect, etc. But it doesn’t. In fact, Omeka’s advanced search requires you to explicitly use Boolean operators between keywords, but you have to use symbols, like “+” for AND and “-” for NOT. Super intuitive!
To top it off, a few years ago Kyle switched the main search functionality of Omeka over to a Solr index, that outperforms the built-in search dramatically. Unfortunately, the Advanced search doesn’t run on the Solr index. We found that the Advanced Search and Basic search would return totally different results for essentially the same search! And some buttons, like the “New Search” button, will take you to advanced search rather than to the basic search. Ugh.
Our plan right now is to do the following (although we need to do a little more research to make sure these will work and are the best options):
Kyle and I will get together over the next few weeks to look into making these changes happen. Then in the next few months, we’ll run another test for digital collections and see how the changes are received!
I’m planning on running another, more generalized test in November. Running a usability test on our website every month is a lot of work, but it has helped us really hammer away at some of the big issues facing our patrons. Thanks for participating, and I look forward to seeing everyone next month!