That’s incredible—it means that if you invested the small amount of effort needed to ensure that the top 14 queries performed well, you’d improve the search experience for 10% of all users. And if, say, half of your site’s users were search dominant,[5] then you’ve just improved the overall user experience by 5% (10% × 50%). Numbers like this can and should be challenged, and 5% may not sound like much. But 5% here, 3% there... these quickly add up.
It bears noting that we just started with a simple report—presented both visually and as a table—and quickly drew some useful conclusions based on the data presented. That there, folks, is analysis. And that’s why reports are only means, not goals.
And equally important, this analysis scales beautifully. Have the time and resources to go beyond the top 14 queries? No problem—tuning the top 42 queries will get you to the 20% mark. About a 100 gets you to 30%, and so on.
[4] You may not have heard of Zipf, but you’ve probably heard of the 80/20 Rule, the Pareto Principle, or Power Laws. All relate to the hockey-stick curve’s dramatic dropoff from “short head” to long tail.
[5] Usability expert Jakob Nielsen suggests that this is the case; see www.useit.com/alertbox/9707b.html
Ways to Use SSA (and This Book)
So what’s the message here? That SSA is an incredibly important tool for helping you understand what users want from your site. And once you have a sense of what they want, you can evaluate and improve all sorts of things that are there to help users get what they want. For instance, you can improve your site as follows:
Search system: SSA will help you understand how people entered searches, where they were when they entered them, and how they interpreted the search results. (We cover this in Chapter 8.)
Navigation and metadata: Do certain pages generate a lot more search activity than others? What kinds of searches? And does this suggest that certain navigational options are missing or labeled in a confusing way? SSA will also give you tips on how to shore up your site’s navigation and metadata. (We cover this in Chapter 9.)
Content: For example, you can study queries that retrieve zero results. Is this because there isn’t content on the topic? Should there be? Or is the relevant content mistitled? Or poorly written? SSA will help you determine what content is missing and what to do to existing content to make sure it gets found. (We cover this in Chapter 10.)
Whatever design challenges you face, SSA—like any other data analysis—will back up your design decisions with actual facts.
Of course, as much as you’d like to make users happy, you also have to make your employers happy. They have goals—for your organization and for the site itself. (They ought to, at least.) These can be expressed and measured as KPI—Key Performance Indicators. The types of search-related metrics that you saw in Chapter 1 can serve as components to these KPIs—in fact, many organizations that are otherwise sophisticated in their measurement of performance often fall down when it comes to measuring findability. In Chapter 3, we’ll help you do what John Ferrara did: use goal-based analysis to measure, monitor, and optimize performance, again and again.
Finally, there are some other important ways to analyze search data:
Pattern analysis: What patterns emerge when you “play” with the data? Can you use those patterns to determine what types of metadata and content are the most important to your searchers? Can you detect changes in seachers’ behavior and needs that are seasonal? Do you also find instructive surprises and outliers? (We cover this in Chapter 3.)
Failure analysis: When searches return no results—or poor results—what can we learn? And what can we do to fix those problems and improve performance? (We cover this in Chapter 4.)
Session analysis: What happens during a specific search session? How do searchers’ needs and understanding of the content change as they search? (We cover this in Chapter 5.)
Audience analysis: How might we uncover the differences between audience segments and their information needs? And how might we better address those differing needs? (We cover this in Chapter 6.)
What Gets in the Way of SSA?
So you’re wondering: if SSA is so valuable, why don’t you hear more about it? And why haven’t you been taking advantage of it?
There are a few predictable and mostly mundane reasons, such as the following:
Lack of awareness: The idea has been around for years, but so was the Web before it took off. There’s simply a lack of critical mass behind SSA getting more attention; hence this book.
Technical hurdles: Your IT people might be too busy to write the scripts to parse your log files or even provide you with access to large and unwieldy data files. This is becoming less of an issue as organizations move toward using analytics applications to access the data; still, you might need a developer’s help in writing ad hoc queries.
Political hurdles: Your IT people might be too busy (or instructed not) to answer your phone calls. Or they might feel that anything related to search is their and only their responsibility (because many equate search with a search engine). There’s no simple solution here. Often, your best and only approach will be patience and persistence—just keep trying.
Legal hurdles: Lawyers often freak out any time someone wants access to user data—even if it’s for internal use—and issue blanket denials to requests for access. If you can get the attention of your legal department’s representative for even 30 seconds, explain to that person that you’re interested in analyzing the collective behavior of your site’s users, rather than digging into the habits of individuals.
Lack of data: Many sites—your personal blog, for example—likely don’t