That’s a good question! Four presenters from two institutions, the University of Manitoba (UM) and Stony Brook University did some usability testing and compared usage patterns before and after implementing Serials Solutions’ Summon system.
At UM, a consultant studied the library’s website and recommended making it easier to use the site and installing a single search box for books and articles. Summon was implemented, then 9 students used it in their research assignments while being observed. Results were:
- Students took it for granted that they could search books and articles simultaneously.
- Nearly all students failed to notice Summon’s limiting options until prompted.
- Searches started broad, then were narrowed down.
- Librarians know about student behavior; many of them responded, “I could have told you that”.
Comparison between Google Scholar and Summon: 35 citations were searched for using Scholar, Windows Live Search, and Scirus. Over a period of 4 years, Scholar retrieved more than the other systems. Both Scholar and Scirus claim to be stronger in the sciences, and Summon is designed to be broader. This test refutes these claims: Scholar seemed to be the broadest.
How did journal usage change after Summon was implemented? Using usage statistics, overall usage dropped from one year to the next. Some platforms reflected the overall trends; others showed increased usage but had a small number of successful requests, and still others showed no consistent usage patterns. This was counter to expectations that usage would increase when Summon became available.
Stony Brook University found that there are too many places to begin a search, and the typical response is to purchase even more resources. This diagram illustrates the problem:
Searches regularly fail so users generally choose the fastest and easiest methods, often settling for free text instead of using the library’s resources. Users want one-stop shopping. But “The Internet is the world’s largest library; it’s just that all the books are on the floor! It’s time to start picking the up.” The semantic web is a major advance in solving users’ searching problems. A graphical user interface (GUI) consisting of a simple search box was developed to present PubMed data to users. The conclusion is that the semantic web will be useful in future search interfaces and discovery systems.
So is discovery better than Google Scholar? It appears to be, but the jury is still out.
CIL 2014 Blogger and Blog Coordinator
Editor, Personal Archiving: Preserving Our Digital Heritage