1.2.3 Observational Studies Versus Experiments
When two or more subpopulations or treatments are to be compared in a biomedical research study, one of the most important aspects of the research protocol is whether the researchers can assign the units to the subpopulations or treatment groups that are being compared. When the researchers control the assignment of the units to the different treatments that are being compared, the study is called an experiment, and when units come to the researchers already assigned to the subpopulations or treatment groups, the study is called an observational study. Thus, in an experiment the researcher has the ability to assign the units to the groups that are being compared, while in an observational study the units come to the researcher already assigned to the groups.
One of the main reasons an observational study is used instead of an experiment in a biomedical research study is that it would be unethical to assign some subjects to a treatment that is known to be harmful and the remaining subjects to a treatment that is not harmful. For example, in a prospective 30-year study of the effects of smoking cigarettes, it would be unethical to assign some subjects to be smokers and others to be non-smokers.
For ethical reasons, observational studies are often used in epidemiological studies designed to investigate the risk factors associated with a disease. Also, a retrospective study is always an observational study because it looks backward in time and the units have already been assigned to the groups being compared. On the other hand, a prospective study and a clinical trial can be run as either experiments or observational studies depending on whether it is possible for the researcher to assign the units to the groups.
Example 1.8
Determine whether it would be possible to perform an experiment in each of the scenarios given below.
1 a. A nutritionist is interested in comparing several different diets in a prospective study. The treatments that will be compared are 10% fat in the diet, 15% fat in the diet, and 25% fat in the diet.
2 b. A pediatrician is interested in studying the effects of a mother’s use of tobacco on the birth weight of her baby. The two treatments that are to be compared are smoking during pregnancy and not smoking during pregnancy.
3 c. A medical researcher is studying the efficacy of vitamin C as a preventive measure against the common cold. The two treatments that are being compared are 1000 mg vitamin C and 1000 mg placebo.
Solutions
1 a. Because the researcher can assign the subjects to each of the three diets in this study, it could be performed as an experiment.
2 b. Because smoking is known to have harmful effects on a fetus, it would be unethical for a pediatrician to assign some mothers smoke during pregnancy and others to not smoke during pregnancy. This study would have to be performed as an observational study by comparing the weights of babies born to mothers who chose to smoke during pregnancy with babies born to mothers who did not smoke during pregnancy.
3 c. Because the medical researcher could assign the subjects to these two treatments, it could be performed as an experiment.
An important advantage experiments have over observational studies is that it is possible in an experiment to control for external factors that might cause differences between the units of the target population. By controlling for the external factors in an experiment, it is possible to make the groups of units assigned to different treatments (i.e., treatment groups) as alike as possible before the treatments are applied. Moreover, in a well-designed experiment when the value of an explanatory variable is changed while no other changes take place in the experimental conditions, any differences in the responses are most likely due to the change in the value of this explanatory variable.
On the other hand, it is much harder to control external factors in an observational study because the units come to the researcher already assigned to the treatments, and thus, in an observational study there is no guarantee that the treatment groups were alike before the treatments were assigned to the units. Because experiments can be designed to control external factors, they can be used to establish evidence of causal relationships; an observational study generally cannot provide strong evidence of a causal relationship because uncontrolled external factors cannot be ruled out as the potential cause of the results.
Example 1.9
To study whether echinacea is effective in shortening the duration of the common cold, a random sample of 200 volunteers is taken. The 200 subjects are divided into two groups of size 100. Each group gets a supply of 300 mg pills and is instructed to take a 300 mg pill as soon as they recognize cold symptoms and to continue taking a pill each day until their cold symptoms go away. One group will receive 300 mg echinacea pills and the other group 300 mg placebo pills. The subjects are asked to record the duration of each cold they have in the following year.
1 Is this study an experiment or an observational study?
2 What is the target population in this study?
3 What is the response variable in this study?
4 What are the treatments in this study?
5 Is this a prospective or retrospective study?
Solutions
1 This study is an experiment because the researcher assigned the subjects to the treatments.
2 The target population in this study is people having the common cold.
3 The response variable in this study is the duration of the common cold.
4 The treatments in this study are 300 mg echinacea and 300 mg placebo.
5 This is a prospective study because the subjects are being followed forward in time.
Example 1.10
To study whether or not there is a relationship between childhood obesity and parental obesity in the United States, a random sample of 500 families was selected. The parents and children in each family were then classified as normal weight, overweight, or obese. The goal was to compare the percentage in each of the weight classifications of the children with normal weight parents against the percentages of the children with overweight and obese parents.
1 Is this study an experiment or an observational study?
2 What is the target population in this study?
3 What is the response variable in this study?
4 What are the treatments in this study?
Solutions
1 This study is an observational study because the subjects came to the researcher assigned to their respective weight classifications.
2 The target population in this study is parents with children living in the United States.
3 The response variable in this study is weight classification of a child. The weight classification of the parent is an explanatory variable.
4 The treatments in this study consist of weight classifications of the parents (normal, overweight, or obese).
1.3 Clinical Trials
Clinical trials are generally associated with biomedical research studies that are carried out on people for testing how well a new medical approach works, for testing the efficacy and safety of new drugs, for evaluating new biomedical procedures or technological advances, and for diagnosing, treating, managing, or preventing a disease. In the United States, a clinical trial is often highly regulated to ensure that it follows a well-designed research protocol that is ethical and preserves the safety of the participants.
For example, in the development of a new drug, a pharmaceutical company often begins