MANOVA

About MANOVA

Multivariate Analysis of Variance (MANOVA) extends ANOVA to analyze multiple dependent variables simultaneously. It tests whether group means differ across several dependent variables, accounting for their intercorrelations.

MANOVA is used for multivariate data, where observations include multiple continuous dependent variables and one or more categorical independent variables (factors).

Examples include test scores across subjects (math, science) grouped by teaching method or physiological measures (heart rate, blood pressure) by treatment.

One-way MANOVA involves one independent variable (e.g., teaching method affecting math and science scores).

Two-way MANOVA includes two independent variables (e.g., teaching method and student gender), allowing analysis of main effects and interactions.

Performing MANOVA

1. Define hypotheses:
   • $H_0$ (Null hypothesis): No group differences across dependent variables.
   • $H_1$ (Alternative hypothesis): Differences exist.
2. Check assumptions: See Assumptions for MANOVA
3. Prepare data: Organize data with categorical independent variables and continuous dependent variables.
4. Compute test statistics: Use statistical software (e.g., Python’s statsmodels, R’s manova) to calculate test statistics. See MANOVA formulas for manual computation, and Differences between MANOVA test statistics to pick most suitable test statistic for your data.
5. Evaluate significance: Compare p-values to significance level. Optionally, use MANOVA rejection rules.

Optional:

1. Post-hoc analysis: If significant, conduct univariate ANOVAs or discriminant analysis to identify specific group differences.
2. Interpret results: Report test statistics, p-values, and effect sizes (e.g., partial eta-squared). Discuss practical implications.

Assumptions for MANOVA

• Dependent variables are continuous and multivariate normally distributed.
• Homogeneity of variance-covariance matrices across groups (use Box’s M test).
• Linear relationships among dependent variables.
• No extreme outliers.
• Adequate sample size (more observations than dependent variables per group).

Formulas used for performing MANOVA

• Wilks’ Lambda: $\Lambda =\frac{|\mathbf{E}|}{|\mathbf{E}+\mathbf{H}|}$
• Pillai’s Trace: $V^{(s)}=\mathrm{tr}[(\mathbf{E}+\mathbf{H}{)}^{-1}\mathbf{H}]={\sum }_{i=1}^s\frac{{\lambda }_i}{1+{\lambda }_i}$
• Lawley-Hotelling Trace: $U^{(s)}=\mathrm{tr}({\mathbf{E}}^{-1}\mathbf{H})={\sum }_{i=1}^s{\lambda }_i$
• Roy’s Largest Root: $\theta =\frac{{\lambda }_1}{1+{\lambda }_1}$

Base formulas

• $p$: Number of dependent variables
• $n$: Total number of observations across all groups
• $k$: Number of groups
• ${\mathbf{y}}_{ij}$ is the $p\times 1$ vector of observations for the $j$-th subject in the $i$-th group
• ${\bar{\mathbf{y}}}_{i\cdot }$ is the $p\times 1$ mean vector for group $i$
• ${\bar{\mathbf{y}}}_{\cdot \cdot }$ is the $p\times 1$ overall mean vector

Derived formulas:

• $\mathbf{E}$: Error sum of squares
• $\mathbf{H}$: Hypothesis sum of squares
• $\mathbf{E}={\sum }_{i=1}^k{\sum }_{j=1}^n({\mathbf{y}}_{ij}-{\bar{\mathbf{y}}}_{i\cdot })({\mathbf{y}}_{ij}-{\bar{\mathbf{y}}}_{i\cdot }{)}^{\prime }$
• $\mathbf{H}=n{\sum }_{i=1}^k({\bar{\mathbf{y}}}_{i\cdot }-{\bar{\mathbf{y}}}_{\cdot \cdot })({\bar{\mathbf{y}}}_{i\cdot }-{\bar{\mathbf{y}}}_{\cdot \cdot }{)}^{\prime }$
• ${\lambda }_i$: Eigenvalues of ${\mathbf{E}}^{-1}\mathbf{H}$, where ${\lambda }_1$ is the largest
• ${\nu }_H=k-1$: Hypothesis degrees of freedom
• ${\nu }_E=n-k$: Error degrees of freedom
• $s=\min ({\nu }_H,p)$

F-transformation (when ${\nu }_H=1$, $s=1$):

• $F=\frac{{\nu }_E-p+1}{p}{U}^{(1)}$, where $U^{(1)}={\lambda }_1=\frac{\theta }{1-\theta }$, follows $F_{p,{\nu }_E-p+1}$

MANOVA rejection rules

• Wilks’ Lambda:
  $\Lambda \le {\Phi }_{\Lambda }^{-1}(\alpha )$

• Pillai’s Trace:

  $V^{(s)}\ge {\Phi }_V^{-1}(1-\alpha )$

  Where:

  • $m=\frac{1}{2}(|{\nu }_H-p|-1)$
  • $n=\frac{1}{2}({\nu }_E-p-1)$

• Lawley-Hotelling Trace:
  $\frac{U^{(s)}{\nu }_E}{{\nu }_H}>{\Phi }_U^{-1}(1-\alpha )$

• Roy’s Largest Root:
  $\theta \ge {\Phi }_{\theta }^{-1}(1-\alpha )$

• F-transformation (when ${\nu }_H=1$, $s=1$):
  $F>{\Phi }_F^{-1}(1-\alpha )$

Differences between MANOVA test statistics

• Wilks’ Lambda ($\Lambda$):
  Measures ratio of error to total variance-covariance. Sensitive to group differences, widely used, but less robust to assumption violations (e.g., non-normality, unequal covariances).

• Pillai’s Trace ($V^{(s)}$):
  Sums eigenvalues of $(\mathbf{E}+\mathbf{H}{)}^{-1}\mathbf{H}$. Most robust to violations of assumptions (e.g., heterogeneity of variances), preferred for unequal sample sizes or non-normal data.

• Lawley-Hotelling Trace ($U^{(s)}$):
  Sums eigenvalues of ${\mathbf{E}}^{-1}\mathbf{H}$. Powerful for detecting group differences, but sensitive to assumption violations, particularly non-normality.

• Roy’s Largest Root ($\theta$):
  Uses largest eigenvalue. Focuses on maximum group separation, powerful when differences are concentrated in one dimension, but less robust and not ideal for multiple group differences.