You don’t necessarily have to choose between moderated and automated testing, or even between lab and remote methods. You can even conduct multiple studies on the same interface, using the findings from one study to add nuance to another. That’s probably excessive for the average study, but for really large-scale projects where you just want to gather every bit of information you can (a new version of a complex software program, an overhauled IA, etc.), being comprehensive can’t hurt.
After reading this chapter, you should have a good idea of whether or not remote research suits you. Give it a try—if it’s not your thing, you can always go back to lab testing. We won’t tell anyone.
Chapter Summary
Do a lab study when you need to use special equipment, keep the interface 100% secure, see the user’s physical movements, or when you can’t use screen sharing tools.
Remote research has its own strengths, the greatest of which is that it enables Time-Aware Research, in which you observe users performing tasks you’re interested in observing right at the moment they’d naturally perform them.
Remote methods can also give you greater user diversity, cut travel costs, allow you to test from anywhere, be quicker to set up, and can be used to test context-dependent interfaces that wouldn’t make sense in a lab.
Currently, remote methods work best for testing functional computer interfaces, prototypes, wireframes, design comps, and mockups. You can test physical products with the right setup, but it’s slightly harder.
There are two kinds of remote research: moderated and automated. Moderated research has the researcher communicating with and observing users as they perform tasks; automated research has researchers using online tools and services to collect behavioral data from users automatically.
Generally, moderated research is good for collecting rich, qualitative behavioral data from a small sample, while automated research is good for collecting quantitative data over a larger sample.
Time-Aware Research.
Recruit someone who’s in the middle of a task.
Observe their behavior.
Chapter 2
Moderated Research: Setup
Gearing Up: Physical Equipment
Drafting the Research Documents
Let’s get down to setting up a typical moderated one-on-one study. We’ll walk you through gathering all the equipment and software you’ll need and explain how to prepare for your first research session. To keep things moving, we’ll stick to bare-bones basics in this chapter, explaining the simplest way to set up a generic moderated research session. Later, you’ll learn about other tools, approaches, and strategies you can use to develop a study that best suits your particularneeds.
Gearing Up: Physical Equipment
Even though you don’t need a lab to do remote research, you’ll still need some equipment to make calls, see your users’ screens, and record the sessions, and there are also a few tools that can make your life easier. Fortunately, you can find a lot of these lying around most offices (see Figure 2.1).
Figure 2.1
A standard remote lab setup: a two-line desk phone, laptop with wired Internet connection, a second monitor, and a phone headset. You may also need a phone tap and amplifier to record audio, depending on your recording setup.
A computer. Laptop or desktop—doesn’t matter. Just be sure that the system specs exceed the minimum requirements of all the screen sharing, chat, recording, and note-taking software you’ll be running. And be especially sure that your computer is compatible with the screen sharing solution you choose; lots of screen sharing solutions are currently PC-only, although this will probably change. And be doubly sure to have plenty of hard drive space free.
A wired, high-speed Internet connection. Wireless connections are too unreliable to run screen sharing software with 100% confidence. It’s very important to have an uninterrupted wired connection—the faster, the better.
A landline, two-line, touchtone desk phone. It has to be a landline desk phone, for a few reasons: batteries won’t last across multiple testing sessions, a phone tap (if you need one for recording) won’t work with a mobile or cordless phone, and most importantly, it’s more stable and reliable. And it has to be a two-line phone if your setup requires you to conference call with both your user and your observers. (Alternatively, you could use a VoIP service like Skype if you’re prepared to deal with the spottiness and instability of the typical Skype call, circa 2009.)
Headset for the phone. Your neck will thank you.
Two monitors. The screen sharing window alone takes up nearly an entire monitor, so if you actually want to be able to see your notes, chat windows, or anything else on your computer, you’ll need the screen space.
Phone recording adaptor or a phone tap and amplifier. You’ll need these tools to record the phone conversation if you’re speaking through your landline telephone (and not a VoIP service).
Peace and quiet. It’s crucial to test in a place where you can talk freely at a normal volume and won’t be disrupted, like an empty office or meeting room. Background noise can be unbelievably distracting to both the moderator and participant.
Doing a Pilot Test Right Now
So now that you’ve got your equipment together, why wait? Let’s do a simple 10-minute pilot test to get your feet wet. This pilot test will simulate a basic moderated session, not including the recruiting process (which is described in the next chapter).
The crucial piece of software you’ll need is a screen sharing application, which will allow you to see what’s on your user’s computer screen during the session. There are many screen sharing options (described at length in Chapter 8, “Remote Research Tools”), but for now we’ll stick with Adobe’s Acrobat Connect. It supports observation and chat, as well as webcam sharing (which we won’t get into here). Connect doesn’t require users to install anything on their computers; all they have to do is visit a Web site that you’ll direct them to, which means that you can get around most firewalls, antivirus software, and other barriers that might prevent you from running the screen sharing. It’s compatible with all OS platforms, so you, your observers, and your participant can be on a Mac, Linux, or PC. And finally, it comes with an optional conference-calling service that you can use to have the study participants and observers on the same line.
Now, you’ll need a pilot participant. Grab anyone at all—your coworker, sister-in-law, high school lacrosse coach—and just tell him/her in advance to be waiting near a phone and a computer with high-speed Internet access.
OK, time to get connected with your participant. For brevity’s sake, we’ll assume you can follow onscreen instructions:
1 Sign up for a free