Another kind of issue involving the researcher is the subjective evaluation of participants’ responses. Consider the situation where participants are giving responses to open-ended questions (i.e., questions where participants are free to answer as they wish and do not have preset categories from which to select) and the researcher is categorizing those responses. It is essential that the criteria for each category remain consistent across coders. One way this is accomplished is by creating clear operational definitions for each category. Operationally defining a variable is describing it in terms of the processes used to measure or quantify it. Imagine if researchers were categorizing qualities of the hospital environment in terms of Roger Ulrich’s (1991) theory of supportive design: positive distraction (PD), social support (SS), and perceived control (PC). If patients mentioned that having access to the Internet improved their experience, we would need an operational definition of each category to place the Internet in one of them. Is the Internet an aspect of positive distraction (something that redirects your attention away from worries and concerns), or is it an aspect of social support (a way to connect with others or encourage interaction)? Arriving at an operational definition can be challenging.
Statistical Regression
In statistical regression, the performance (scores) of participants who are at the extremes on a given measure will move toward the mean on subsequent administrations. In theory, you could address this issue by pretesting participants to avoid those with extreme scores, but if those with extreme scores are a focus of the research (e.g., low and high crossword puzzle–solving skills), then being aware of regression toward the mean is important in evaluating the impact of any intervention.
Differential Selection (Biased Selection of Subjects)
Differential selection (biased selection of subjects) is a problem when preexisting characteristics of participants may affect scores on the dependent variable. Imagine we were interested in the effect of exposure to nature on mood, and we unknowingly selected students raised in urban areas, all of whom had been placed in one residence hall where first-year students were recruited for the study. In this kind of situation, their upbringing cannot be separated from exposure to our intervention (e.g., a camping experience in the woods). To avoid this situation, we might do a preevaluation of demographic characteristics (type of upbringing) to avoid this lopsidedness and then randomly distribute students with different kinds of upbringing across our conditions.
Experimental Mortality
The term experimental mortality signifies not only that people have dropped out but also that participants have dropped out of a study in a nonrandom manner (e.g., more from the control than from the intervention group). That situation creates a problem because those who remain in the study no longer represent our original balance of random assignment to condition. Dropping out of studies commonly occurs in longitudinal research, where participants are followed over long periods of time (see Chapter 7). One well-known exception to longitudinal dropouts is the work involving what are known as Terman’s Termites, a group of gifted youngsters followed by the Stanford researcher Lewis Terman for decades (Leslie, 2000).
Experimental mortality: One of Campbell and Stanley’s (1963) threats to internal validity in which people drop out of studies in a nonequivalent manner (e.g., more older adults than younger adults drop out of the intervention than out of the control group).
Selection–maturation interaction: One of Campbell and Stanley’s (1963) threats to internal validity in which with quasi-experimental designs with multiple groups, some preexisting aspect of the groups might produce differences in the rate of change unrelated to the variable of interest.
Demand characteristics: “Cues available to participants in a study that may enable them to determine the purpose of the study, or what is expected by the researcher” (Corsini, 2002, p. 262).
Selection–Maturation Interaction
In quasi-experimental research designs where you have not randomly assigned participants to conditions (which is often the case in real-world situations, such as classrooms or company divisions, where you have “naturally assembled collectives”; Campbell & Stanley, 1963, p. 47), some preexisting characteristic of the group may influence the outcome of the experimental manipulation, not the power of the manipulation itself, because one group might mature or change at a faster rate than another. This is called a selection–maturation interaction. The researcher makes every attempt to use equivalent groups, but without the pretesting to guarantee sampling equivalence in the real world, Campbell and Stanley noted, this situation presents a threat to internal validity.
Behavior of the Experimenter and Demand Characteristics
Continuing with threats to internal validity, one very important threat is the researcher, as was indicated earlier in the material on instrumentation. The role of the experimenter also fits into a broader category called demand characteristics. The label suggests the meaning of the term—something in the research situation demands or shapes our behavior. The formal definition of demand characteristics is “Cues available to participants in a study that may enable them to determine the purpose of the study, or what is expected by the researcher” (Corsini, 2002, p. 262). For the researcher, these cues could come in the attire of the researcher (e.g., a lab coat), the manner of speech, eye gaze, or a host of other qualities. Cues can also come from the physical setting (e.g., size of the room), the other people in the room (e.g., only one gender), and the status of the researcher (e.g., student vs. professor).
Try This Now 3.3
Think back to your experience participating in research, and identify demand characteristics that existed and how you think they may have affected your responses.
What can be done about demand characteristics? Being aware that such characteristics exist is the first step. Evaluating your behavior and the setting in which the research is to take place is the second step. Changing any problematic aspects of either your behavior or the setting is the third step.
Behavior of the Participant: Role Attitude
Single-blind experiment: Research design in which participants are unaware of the conditions to which they have been assigned.
Double-blind experiment: Research design in which both the participant and the researcher are unaware of the condition to which the participant has been assigned.
Role attitude cues: When participants approach research with a particular attitude, such as cooperativeness; may affect results.
Cooperative attitude: Attitude of research participant who tries to help the researcher.
Ideally, we would like to have people participate in research who have no preconceived notions of what is going to be asked of them. As we know, we are dealing with humans who have schemas of the way the world works (refer to Chapter 1). Participants may think that certain kinds of behaviors are expected of them because they are in a research laboratory (for example, that it is inappropriate to challenge the experimenter about any aspect of the research). Also, students who have taken no courses in the social sciences differ in their level of naiveté or sophistication (as well as in their skepticism and hypotheses regarding deception) from those who have taken such courses. Students who have taken courses may respond with “insight rather than naiveté” (Adair, 1973, p. 19), which can be a problem for the researcher. John Adair’s book The Human Subject: The Social Psychology of the Psychological Experiment emphasizes that the research endeavor is a social interaction and that participants are not necessarily passive. Participants