Orthogonal matrix

Definition and particular cases

Orthogonal matrices form the group $O(n)$, known as the orthogonal group. See orthogonal and special orthogonal groups for the general signature.
They are the isometries of ${\mathbb{R}}^n$ that fix $O$. They are linear transformations (see @stillwell2008naive, page 37, exercises). It can also be shown (see @stillwell2008naive, page 37) that any element of this group is the product of fewer than $n$ reflections.
It is satisfied that

where $SO(n)$ represents $n$-dimensional rotations, and ${\mathbb{Z}}_2$ corresponds to reflections.
Furthermore,

(the number of ways to choose two axes out of $n$). It is not a normal subgroup.

• $SO(2)$

These are the rotations of the plane. They can be seen as the complex numbers of modulus 1 or as the subgroup of matrices of the form:

It is an abelian group, so every subgroup is normal. It has many finite subgroups (rotations of regular polygons).

• $O(3)$

This is the group of 3x3 orthogonal matrices, meaning matrices such that $A\cdot A^t=I$. Its determinant can be either 1 or -1. They represent transformations of ${\mathbb{R}}^3$, considered as a vector space with an orthonormal basis, that do not change distances or angles, i.e., they preserve the dot product. In other words: the linear transformations of ${\mathbb{R}}^3$ that preserve the dot product, when expressed in an orthonormal basis, yield an orthogonal matrix. They are rotations and reflections (and their compositions).
It is not a connected space. The component containing the identity is SO(3).

The Lie algebra

The Lie algebra is the set of skew-symmetric matrixes (see SO(3)#And what about the Lie algebra so 3).
The Lie algebras of the orthogonal group $O(n)$ and the special orthogonal group $SO(n)$ can be computed by considering their defining properties and the infinitesimal generators of their respective transformations.

Lie Algebra of $O(n)$:

The group $O(n)$ consists of all $n\times n$ orthogonal matrices, i.e., matrices $A$ that satisfy:

Differentiating this condition with respect to a small parameter $t$, we obtain:

which leads to:

This implies that:

Thus, the infinitesimal generators (i.e., elements of the Lie algebra) are skew-symmetric matrices. Therefore, the Lie algebra of $O(n)$, denoted $\mathfrak{o}(n)$, consists of all skew-symmetric $n\times n$ matrices:

This Lie algebra has dimension $\frac{n(n-1)}{2}$, as the number of independent components in a skew-symmetric matrix is $\frac{n(n-1)}{2}$.

Lie Algebra of $SO(n)$:

The group $SO(n)$ consists of the special orthogonal matrices, which are orthogonal matrices with determinant 1:

To find the Lie algebra of $SO(n)$, we differentiate the condition $det(A(t))=1$ with respect to $t$, which gives:

Since $det(A)=1$, we have:

For small variations $\dot{A}$, this means the matrix $\dot{A}$ must satisfy the additional condition that the trace of $A^{-1}\dot{A}$ is zero. In the Lie algebra of $SO(n)$, this results in a subspace of the skew-symmetric matrices with the trace condition. Therefore, the Lie algebra of $SO(n)$, denoted $\mathfrak{so}(n)$, consists of the skew-symmetric matrices with trace zero. But this is redundant, since all skew-symmetric matrices have null trace. Therefore: