A recent post on Andrew Gelman's blog (here)--ironically prompted by a grad student's frustration with efforts to detect treatment effects in, of all things, the weight-training context--caught my eye not for any insights into weight-training but, rather, for some of the post's practical research design and multi-level modeling suggestions.
An truncated version of the student's question follows: "Consider the field of exercise science, and in particular studies on people who lift weights... Because muscle gain, fat loss, and (to a lesser extent) strength gain happen very slowly, and these studies usually last a few weeks to a few months at most, the effect sizes are all quite small. This is especially the case when comparing any two interventions that are not wholly idiotic for a difference in means."
Gelman's multi-step reply begins with the assumption that "the way to go is to have detailed within-person trajectories. Crude between-person designs just won’t cut it...." Executing a within-person trajectory approach from a research design and data-gathering perspectives involves: "At the design stage, it would be best to try multiple treatments per person. And you’ll want multiple, precise measurements. This last point sounds kinda obvious, but we don’t always see it because researchers have been able to achieve apparent success through statistical significance with any data at all." Gelman's final suggestion focuses on model selection. "The next question is how to analyze the data. You’ll want to fit a multilevel model with a different trajectory for each person: most simply, a varying-intercept, varying-slope model."
While the example in the post features R code, it nicely illustrates some of the critical ex ante decisions necessary for successful empirical studies.
Comments