10.2 Observational Studies and Designed Experiments
- Key Idea 2: In an observational study, the treatment of interest is assigned to units arbitrarily, or in ways uncontrolled by the investigator...
Discussion
- In 1948-49, Doll and Hill studied 709 people with lung cancer, and 709 without. For the men (N = 1230) they found the following:
Men Cancer No Cancer Smokers 647 (53%) 583 (47%) Non-Smokers 2 (3%) 66 (97%) - For the men, smoking and lung cancer seem heavily associated.
- For the women, the rate of lung cancer among the smokers was about 59%,. compared to 37% among the nonsmokers. Again, the association is clear, but with the intriguing observation that a much higher percentage of the nonsmoking women had lung cancer than their nonsmoking male counterparts.
- How were the people chosen? Doll and Hill found 709 people with lung cancer in various London hospitals, then searched for 709 controls, that is, people without lung cancer. They were careful to select this control group to be similar to the people with lung cancer.
- Although the researchers did decide who was to be in their study, they could not assign the condition "smoke" vs. "don't smoke" to an individual. Nor could they assign the outcome "cancer" or "no cancer" to an individual. They could merely observe the presence or absence of each - hence it is an observational study.
- This evidence establishes association, but not causation. In the absence of further scientific evidence, it could just as well be that lung cancer causes smoking, or that a third factor causes both to tend to occur together.
- The eminent statistician Sir Ronald Fisher (1890 - 1962) argued forcefully against drawing the conclusion from these data that smoking causes cancer.
Read More
- Key Idea 3: Since the investigator cannot determine which units receive which treatments, randomized assignment of treatments to units is impossible in an observational study...
Key Idea 4: Analyzing the Key Problem (an observational study)...
Discussion
Review the key problem.
Does smoking really result in longer life? In fact, a third factor "age" may explain the results.
Consider this expanded version of the data shown in Table 10.1.
The youngest group does not differ much - hardly anyone has died after the 20-year period (whether smoker or non-smoker).
The oldest group also does not differ much - the vast majority are dead (whether smoker or non-smoker).
Among middle ages, though, more smokers have died within the 20-year followup period.
The overall total is skewed by the fact that there are so many nonsmokers in the oldest group, leading to the false impression that nonsmokers are more likely to die.
Key Idea 5: Thus, in this observational study, the influence of a third variable (age) is so strong that, when not controlled for, it entirely hides the fact that smoking reduces life expectancy...
Discussion
Observational studies are valuable, but they cannot fail to leave doubt about how to interpret the results. There is always the possibility of some third factor explaining an observed association.
One way to control such a third factor is to study the data separately for restricted ranges of the third factor ("young," "middle-aged," "old").
Of course we cannot always be sure that we have identified all such potentially trouble-making third factors.
Key Idea 6: A randomized experiment avoids the third factor problem by forcing the groups being compared to be approximately alike in every way except the treatment under study...
Discussion
- In 1954, a randomized trial was conducted of a new polio vaccine. It had the following features:
- Placebo - a medically inactive treatment that, to the recipient, looks and feels the same as the real treatment.
- Randomized controls - subjects are randomly assigned to get placebo or treatment.
- Double-blind study - the evaluators (doctors, in this case) are not aware which children received the vaccine and which received the placebo.
- (Had the doctors known who received placebo and who received vaccine, it would have been a single-blind study)
- The results show that the vaccine was effective:
Number of subjects Number with polio Polio rate per 100,000 Randomized
control study
Vaccinated (volunteers)
200,745 57 28 Control (volunteers)
201,229 142 71
Key Idea 7: Compare the results of the randomized polio study to the NFIP study, which lacked randomized assignment of children to placebo and treatment groups...
Discussion
Results of NFIP and Randomized Control Polio
Number of subjects Number with polio Polio rate per 100,000 NFIP study Vaccinated
(2nd-grade volunteers)
221,998
56
25
Control
(1st and 3rd grade)
725,173 391 54 Randomized control
study
Vaccinated
(volunteers)
200,745
57
28
Control
(volunteers)
201,229
142
71
- While the results for the vaccinated groups are similar, the polio rate for the randomized controls is substantially higher than for the NFIP controls, implying that the NFIP study underestimated the effect of the vaccine.
- Although not randomized, the NFIP design was fairly good. Still, there were some important ways the treatment and control groups may differ. Read more about the design, the "volunteer effect," "change in behavior," and "effect on diagnosis."
- "The randomized control study was not subject to the criticisms aimed at the NFIP study, and thus its validity and believability were much greater.
Key Idea 8: Randomized controlled experiments are not always possible, so observational studies must often be relied upon...
Discussion
- The conditions required to conduct a randomized, controlled, double-blind study of smoking are hard to imagine:
- A group of people is identified (volunteers?) to be in the study, say, all about 18 years old. Half are randomly assigned to smoke for the rest of their lives; half are randomly assigned to not smoke. But they should not know which group they are in, so we need placebo. We create placebo cigarettes that look (and taste) just like another cigarettes, and have the same detectable effect on people. (If they did not, then it would not be long before the subjects would figure out which group they are in.) But these placebo cigarettes cannot have cancer-causing ingredients of usual cigarettes. (Do we know really what those are?) These people would have to followed and supplied with the correct type of cigarettes until they die or, say, for 40 years.
Key Idea 9: Back to smoking - why observational studies are convincing....
Discussion The idea that the relationship between smoking and lung cancer is causative is convincing because of a number of factors:
Key Idea 10: Those amazing placebos....
Discussion
- True healing can occur simply by believing the placebo will work.
- Controlled studies, therefore, include a placebo group, and the investigator looks for improvement beyond any improvement achieved by placebo alone.
- Review details of how the placebo effect mislead investigators examining treatments for heart disease (tying off the mammary artery) and back pain (injections of an enzyme into a slipped disc).
Summary....
Discussion
Review how examples in this section illustrate two benefits to running a well-designed experiment - validity and believability.