# Statistics: Canonical Correlation

Let $Y=\left( \begin{matrix} {{Y}_{1}} \\ {{Y}_{2}} \\ \end{matrix} \right)$ a random vector with ${{Y}_{i}}\,$ ${{k}_{i}}\times 1\,$. Let $\operatorname{E}(Y)=0$ and $\operatorname{Var}\left( \begin{matrix} {{Y}_{1}} \\ {{Y}_{2}} \\ \end{matrix} \right)=\Sigma =\left[ \begin{matrix} {{\Sigma }_{11}} & {{\Sigma }_{12}} \\ {{\Sigma }_{21}} & {{\Sigma }_{22}} \\ \end{matrix} \right]$

There are many ways of arriving at the canonical correlations between $Y_1\,$ and $Y_2\,$. Here we present an approach based on the singular value decomposition of an arbitrary matrix.

Let $\Sigma _{11}^{-1/2}{{\Sigma }_{12}}\Sigma _{22}^{-1/2}=UDV$ where $U\,$ and $V\,$ are orthogonal matrices and $D\,$ is a diagonal (not necessarily square) matrix, with dimension the same as that of Σ12, and with non-negative diagonal elements ordered from largest to smallest.

Then, let ${{S}_{1}}=\Sigma _{11}^{-1/2}U$ and ${{S}_{2}}=\Sigma _{22}^{-1/2}{V}'$

so that $\Sigma _{ii}={{S}_{i}}{{{{S}}}_{i}^{\prime}}\,$ and $\Sigma _{12}={{S}_{1}}D{{{{S}}}_{2}^{\prime}}\,$

Many of the major matrices of linear models can be expressed in terms of canonical correlations:

$\Sigma _{ii}^{-1}={S}_{i}^{\prime -1}S_{i}^{-1}$

$\Sigma _{12}\Sigma _{22}^{-1}={{S}_{1}}D{{{{S}}}_{2}^{\prime}}{S}_{2}^{\prime -1}S_{2}^{-1}={{S}_{1}}DS_{2}^{-1}$

$\Sigma _{22\cdot 1}={{\Sigma }_{22}}-{{\Sigma }_{21}}\Sigma _{11}^{-1}{{\Sigma }_{12}}={{S}_{2}}\left( I-{{D}^{2}} \right){{{{S}}}_{2}^{\prime}}$

$\Sigma _{22\cdot 1}^{-1}={S}_{2}^{\prime -1}{{\left( I-{{D}^{2}} \right)}^{-1}}S_{2}^{-1}$