Matrix diagonalization

A diagonalizable matrix $A$ can be written as

A = P Δ P^{- 1}

where $Δ$ is a diagonal matrix (the eigenvalues at the diagonal) and $P$ is a matrix whose columns are the eigenvectors.

Idea

The idea is that we have a basis of the vector space in which the transformation given by $A$ is simply made of scale changes (even negative or null) in the main axes. When the basis change is given by means of an orthogonal matrix then the matrix $A$ is symmetric. That is, we have

Diagonal matrix: scale changes in the main axes.
Symmetric matrix: scale changes in the main axes, but seen from a different point of view obtained by a rigid transformation (element of O(n))
General diagonalizable matrix: scale changes in the main axes, but seen from a different point of view obtained by a general linear transformation (element of the general linear group).

It is related to singular value decomposition. Indeed they are equal when the matrix is symmetric and positive-semidefinite. See MSE.

Diagonalization method

Solve $d e t (A - λ I) = 0$ . The solutions $λ_{i}$ are called eigenvalues.
For every $λ_{i}$ we look for a basis of the subspace $V_{λ_{i}}$

(A - λ_{i} I) (\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \end{matrix}) = (\begin{matrix} 0 \\ 0 \\ ⋮ \end{matrix})

They are the eigenvectors associated to the eigenvalue $λ_{i}$ .
3. If we have enough eigenvectors to complete a basis of $R^{N}$ then the matrix $A$ is diagonalizable, and $Λ$ is made with the eigenvalues (repeated if necessary, according to the dimension of $V_{λ_{i}}$ ). The $P$ matrix is made with the eigenvectors. If we don't have enough eigenvectors, the matrix is not diagonalizable and we look for the Jordan canonical form of a matrix. The reasons could be:
a. There are complex solutions in step 1.
b. Even with enough solutions, or even in the case of a complex matrix, any $V_{λ_{i}}$ has not dimension enough "to fill" the multiplicity of $λ_{i}$ in step 1.

If a matrix is diagonalized, its diagonal form is unique, up to a permutation of the diagonal entries. This is because the entries on the diagonal must be all the eigenvalues. For instance,

[\begin{matrix} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{matrix}] and [\begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{matrix}]

are examples of two different ways to diagonalize the same matrix.

Important fact: common eigenvectors.