Kuller Presidential Address Identifies Seven Curable “Sins of Epidemiology”

Says Future of Epidemiology Depends on Doing More Consequential Research

What can we do besides having a better PR (public relations) firm? That’s a question asked by Lewis Kuller in his 1996 presidential address in front one of the largest groups of epidemiologists ever to attend a Society for Epidemiologic Research (SER) meeting in Boston in mid-June. Noting all the publicity about epidemiologic findings in the last year, including a Science article highly critical of epidemiology, Kuller offered a list of curable sins which he said might be contributing to epidemiology’s fall from scientific grace.

The seven sins described by Kuller, who is chairman of the Department of Epidemiology at the University of Pittsburgh, are:

1) the sin of biological implausibility

2) the sin of confusing reproducibility of a measure with accuracy or validity—the “parrot effect”

3) the sin of adjusting data unnecessarily—the “adjustment of data syndrome”

4) the sin of failing to determine temporal associations—the “epi phenomenon”

5) the sin of using multivariate models to evaluate biological or causal associations

6) the sin of replicating the same epidemiological studies over and over again with minor variations on the theme

7) the sin of disassociating academic epidemiology from public health

Biological Implausibility

This occurs when the investigator does not know or understand the biology or pathophysiology of the disease of interest or the measures of the independent and dependent variables. The reported results of the study, i.e. the measurements, are non-physiological. The investigator is unaware of the problem. This is more likely to occur when the data, usually the independent variable, is transformed and the units of measurement are abstruse. Investigators, reviewers, and journal editors have no idea of the range of the true measurement.

“Parrot Effect”

Comparison of different methods of measuring the same variable does not necessarily prove that any of the measures are accurate. It just assumes that the correlations among the measurements are fairly high, i.e. repeatability but not accuracy. According to Kuller, “you can teach a parrot to say good morning. The parrot says good morning all day long very repeatably, but the parrot is unfortunately inaccurate most of the time.”

The Adjustment of Data Syndrome

“I do not believe that one can truly adjust for sex, except by a surgical procedure that is very unpopular,”  says Kuller. Also, he believes that adjustment for host specific factors eliminates host susceptibility as an item to be studied, and calls it “a dreadful mistake.” Furthermore, age adjustment presumes similar biological relationships for young and old individuals, and this is “clearly illogical.”

The Failure to Determine Temporal Associations

According to Kuller, it is likely that the independent variables are really due to the dependent variables for many of the associations reported in epidemiology. The levels of many risk factors in both cardiovascular disease, cancer, and probably other chronic diseases are modified by the disease in its early subclinical phases.

Multivariate Models To Evaluate Biological or Causal Associations

According to Kuller, epidemiologists often use multivariate models to exclude co-variates because their p-value was less than .10 or .05 in the model. Some of the variables that remain in the model are clearly dependent variables of the independent variables. “There is nothing wrong with generating a new testable model of associations,” says Kuller, “the epidemiologist however must document the nature of the new model, how it relates to existing biological interpretations and how the model can be further tested in both epidemiological and laboratory sciences...The use of complex statistical methodology is not an answer for either good study design or biological plausibility.”

Replicating Epidemiolgic Studies

To make his point, Dr. Kuller asked—Are many epidemiological studies to be classified as inconsequential because they only represent fine tuning of previous studies with slightly different populations with little likelihood of enhancing the science of epidemiology and the public health and preventive medicine implications? According to Kuller, such studies are often methodologically outstanding, the analysis is superior, and the study is dull! In his view, the basic problem is the failure to move from descriptive to observational studies to clinical trials/natural experiments in order to further our understanding of potential etiologic associations.

Schism Between Academic Epi and Public Health

This sin is probably the most important, according to Kuller, and involves failure on the part of both the public health community and the epidemiology community. According to Kuller, the public health establishment in the United States has, to a considerable degree, lost its epidemiology base and similarly epidemiology has lost its public health base to a considerable degree. “It is time for academic epidemiologists to take a very hard look at what is public health epidemiology and preventive medicine and re-focus some of our efforts in establishing a strong scientific basis to evolving public health an