Statistics: Canonical Correlation

From Wiki1

Jump to: navigation, search

Let Y=\left( \begin{matrix}
   {{Y}_{1}}  \\
   {{Y}_{2}}  \\
\end{matrix} \right) a random vector with {{Y}_{i}}\, {{k}_{i}}\times 1\,. Let \operatorname{E}(Y)=0 and \operatorname{Var}\left( \begin{matrix}
   {{Y}_{1}}  \\
   {{Y}_{2}}  \\
\end{matrix} \right)=\Sigma =\left[ \begin{matrix}
   {{\Sigma }_{11}} & {{\Sigma }_{12}}  \\
   {{\Sigma }_{21}} & {{\Sigma }_{22}}  \\
\end{matrix} \right]

There are many ways of arriving at the canonical correlations between Y_1\, and Y_2\,. Here we present an approach based on the singular value decomposition of an arbitrary matrix.

Let \Sigma _{11}^{-1/2}{{\Sigma }_{12}}\Sigma _{22}^{-1/2}=UDV where U\, and V\, are orthogonal matrices and D\, is a diagonal (not necessarily square) matrix, with dimension the same as that of Σ12, and with non-negative diagonal elements ordered from largest to smallest.

Then, let {{S}_{1}}=\Sigma _{11}^{-1/2}U and {{S}_{2}}=\Sigma _{22}^{-1/2}{V}'

so that \Sigma _{ii}={{S}_{i}}{{{{S}}}_{i}^{\prime}}\, and \Sigma _{12}={{S}_{1}}D{{{{S}}}_{2}^{\prime}}\,

Many of the major matrices of linear models can be expressed in terms of canonical correlations:

\Sigma _{ii}^{-1}={S}_{i}^{\prime -1}S_{i}^{-1}

\Sigma _{12}\Sigma _{22}^{-1}={{S}_{1}}D{{{{S}}}_{2}^{\prime}}{S}_{2}^{\prime -1}S_{2}^{-1}={{S}_{1}}DS_{2}^{-1}

\Sigma _{22\cdot 1}={{\Sigma }_{22}}-{{\Sigma }_{21}}\Sigma _{11}^{-1}{{\Sigma }_{12}}={{S}_{2}}\left( I-{{D}^{2}} \right){{{{S}}}_{2}^{\prime}}

\Sigma _{22\cdot 1}^{-1}={S}_{2}^{\prime -1}{{\left( I-{{D}^{2}} \right)}^{-1}}S_{2}^{-1}

Personal tools