ANOVA is regression.

More precisely, it’s a computational trick to efficiently compute a particular type of nested model comparison under additional assumptions.

It wasn’t a bad idea back in the slide-rules and mechanical adding machine days. But all the extra assumptions (e.g. sphericity) and shortcuts aren’t worth it with today’s computing hardware.

We’ve moved forward in computation but not thinking

We stop learning statistics after we finish our doctorates.

But we use statistics like our mentors. And they learned ANOVA. So even when we advance the methods, we don’t advance the thinking and the explaining.

Think in terms of effect sizes and not p-values

• Mixed models give you an explicit estimate of effect size: the coefficient estimate!
• If you want standardized effect sizes, then you can also standardize your coefficient.
• Confidence intervals give strictly more information than p-values and can also be used for significance testing.

Think in terms of specific hypotheses and comparisons

• ANOVA encourages thinking in a weird omnibus + posthoc test framework

  • “There was a significant difference somewhere between these 3 groups”
  • let’s follow that up with t-tests amongst all possible pairs

• Instead, we should be thinking of specific comparisons that we care about.

• Moreover, we can do all of this in a single step instead of in two stages.

Learn to love and leverage contrast coding

• Got a hypothesis that A > B > C? Test that hypothesis directly!
• Got a hypothesis that A != B and A != C? Test that!
• Make the intercept interesting again!
• Too many options to discuss here, but contrast coding and centering and scaling of covaries should be part of your design (and not specifying it makes your analysis unintepretable Brehm & Alday, 2022)

Treat population-/group-level and individual analyses holistically

• in addition to the fixed effects, i.e. “population-level” estimates, mixed models also provide predictions about the individual grouping levels
• these can be interpreted like individual estimates, but are technically predictions
• there are sometimes called conditional modes, conditional means, or the best linear unbiased predictions (BLUPs)

General model fit

BLUPs

Subject-level predictions

Comparison of individual regression vs mixed model

Embrace continuous covariates

• ANOVA forces discrete thinking, both in terms of significance and in terms of predictors
• Mixed models can handle continuous measures
• This allows you to control for additional potential confounders:
  • screen brightness
  • reaction time
  • presentation sequence
• You dont have to assume a constant a priori impact of baseline (Alday, 2019)