MN505 Unit 3 study design critique example

Do Farmers Markets Prevent Diabetes? A Study Design Critique of a County-Level Correlation

[Student Name]

Purdue University Global

MN505: Epidemiology and Health Promotion

Unit 3 Assignment

[Instructor Name]

[Date]

The health department release and its figures are composites written as a model document. No real agency's release is quoted.

The Release in Four Facts

The release reports that across the state's 62 counties, those with more farmers markets per 10,000 residents had lower adult diabetes prevalence, and it concludes that "opening farmers markets will prevent type 2 diabetes." Four facts describe what it actually measured. First, the unit of observation was the county; there were 62 data points. Second, market density came from one year of market registrations with the state agriculture office, divided by county population. Third, adult diabetes prevalence came from model-based county estimates, not from counts of diagnosed individuals in each county. Fourth, the association was a correlation of -0.44, with each additional market per 10,000 residents associated with prevalence 1.3 percentage points lower.

Classifying the Design

The design follows from the unit of analysis. Every number in the release describes a county: market density is a county average and prevalence is a county estimate. No individual's market use was ever linked to that individual's diabetes status. That makes the study ecological, regardless of how the press office framed it, because a design is defined by what was measured, not by what was concluded. Ecological studies compare groups rather than individuals and are useful for generating hypotheses and for describing patterns across populations, but they carry specific limits when used to support claims about individuals (Morgenstern, 1995).

The design has a second feature that limits it further: it is cross-sectional. Market density and prevalence were measured for the same period, so the study cannot show that markets came before any change in diabetes. The outcome was also prevalence rather than incidence. Prevalence depends on how many people develop diabetes and on how long they live with it, so a county could have lower prevalence because fewer people develop diabetes, because people with diabetes move away or die sooner, or because the model underestimates cases where fewer people see clinicians.

Threat One: Confounding by County Income

Confounding can be measured. County median household income is associated with both variables: wealthier counties have more farmers markets, and wealthier counties have lower diabetes prevalence for many reasons, including access to health care, education and neighborhood conditions. Using the same 62 counties, market density and median household income were correlated at 0.58. When income was added to a regression of prevalence on market density, the slope shrank from 1.3 to 0.4 percentage points lower prevalence per additional market per 10,000 residents.

Market density per 10,000: unadjusted slope -1.3 points; adjusted for income, -0.4 points. Median household income: correlated 0.58 with market density.

Most of the association disappears once one confounder enters, and other plausible confounders, such as the age structure of the county, rurality and the proportion of residents with health insurance, were not measured at all. The remaining 0.4 points should not be read as the true effect of markets, because unmeasured confounding may account for it too. A further adjustment for age structure, which differs sharply between the state's rural and university counties, would likely shrink it again, perhaps close to zero, and would leave little for markets to explain.

Threat Two: The Ecological Fallacy

The second threat cannot be measured with these data at all. The ecological fallacy is the error of inferring that a relationship observed between groups also holds for the individuals within them (Robinson, 1950). A concrete case shows how the county pattern could arise without any individual benefit. Suppose markets cluster in counties with a university and a growing population of young professionals. Those residents shop at the markets and have low diabetes risk because of their age and income. Meanwhile, older residents with the highest diabetes risk in the same counties never shop at the markets at all. The county would show many markets and low prevalence, yet no one who used a market avoided diabetes because of it. Nothing in the release shows that the people who shop at markets are the people who stayed well.

Three Stronger Designs, Ranked

First, a stepped-wedge cluster randomized rollout of produce vouchers redeemable at farmers markets. Counties or clinics would be randomized to start the program at different times, and every cluster would eventually receive it, which is often acceptable to communities that expect a benefit (Hemming et al., 2015). Measuring incident diabetes, or a precursor such as hemoglobin A1C, among enrolled adults would allow a causal claim about the program. This design ranks first because randomization of timing addresses confounding, and individual outcomes avoid the ecological fallacy.

Second, a prospective cohort study that records individual adults' market use and diet at baseline and follows them for new diabetes diagnoses, adjusting for income, age and other confounders. It could show that market users develop diabetes less often, but residual confounding would remain, since people who choose markets may differ in ways that are not measured.

Third, a controlled interrupted time series of county diabetes incidence before and after new markets open, compared with counties where no markets opened. It improves on the release by adding timing and a comparison, but it remains ecological and cannot link market use to individuals.

What the Release Should Have Said

An accurate version of the release would have been more modest and still newsworthy. It might have read: "Counties with more farmers markets per resident tend to have lower rates of diagnosed diabetes, but wealthier counties also have more markets, and much of the pattern is explained by income. The department will test whether helping residents buy fresh produce at markets lowers diabetes risk, starting with a pilot in selected counties." That statement reports the association, names the main confounder, avoids a causal verb and commits to a design that could answer the question. Health departments communicate to the public and to legislators who may fund programs on the strength of a headline, so the verbs in a release carry real consequences. Words such as prevent, cause and reduce imply a causal design; words such as associated with and tend to have fit an ecological correlation.

Conclusion

The release deserves credit for raising a testable idea and for using data that are easy to obtain. Its claim that markets will prevent diabetes goes far beyond what an ecological, cross-sectional correlation can support, and most of the association disappears when a single confounder is added. The county pattern is a reason to pilot and evaluate a program such as produce vouchers, not a basis for a statewide policy. If the voucher pilot shows a reduction in new diabetes among participants, the department will then have evidence that justifies the claim it made too early.

References

Hemming, K., Haines, T. P., Chilton, P. J., Girling, A. J., & Lilford, R. J. (2015). The stepped wedge cluster randomised trial: Rationale, design, analysis, and reporting. BMJ, 350, h391. https://doi.org/10.1136/bmj.h391

Morgenstern, H. (1995). Ecologic studies in epidemiology: Concepts, principles, and methods. Annual Review of Public Health, 16, 61-81. https://doi.org/10.1146/annurev.pu.16.050195.000425

Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15(3), 351-357. https://doi.org/10.2307/2087176