Thermal Equilibrium

Suppose a system has finitely many states $i = 1, \dots, n$ , each with energy $E_{i}$ . If the probability $p_{i}$ that the system is in state $i$ maximizes entropy subject to a constraint on its expected energy, then the system is said to be in thermal equilibrium.

The expected energy is:

⟨ E ⟩ = \sum_{i = 1}^{n} p_{i} E_{i} .

In thermal equilibrium, the probabilities $p_{i}$ are given by the Boltzmann distribution:

p_{i} = \frac{e^{- β E_{i}}}{\sum_{j = 1}^{n} e^{- β E_{j}}}

where:

$β$ is a constant (in physics, $β = \frac{1}{k T}$ , with $T$ the temperature and $k$ the Boltzmann constant. See gas in a box).
The denominator $Z = \sum_{j = 1}^{n} e^{- β E_{j}}$ is called the partition function.

Derivation insight: principle of maximum entropy

The Boltzmann distribution is derived by maximizing the Shannon entropy:

S = - \sum_{i = 1}^{n} p_{i} \ln p_{i}

subject to:

Normalization: $\sum_{i = 1}^{n} p_{i} = 1$
Fixed mean energy: $\sum_{i = 1}^{n} p_{i} E_{i} = ⟨ E ⟩$

Using Lagrange multipliers, we introduce multipliers $α$ and $β$ and form the function:

L = - \sum_{i = 1}^{n} p_{i} \ln p_{i} + α (1 - \sum_{i = 1}^{n} p_{i}) + β (⟨ E ⟩ - \sum_{i = 1}^{n} p_{i} E_{i})

Taking partial derivatives with respect to $p_{i}$ , we obtain the condition:

\ln p_{i} = - 1 - α - β E_{i} \Rightarrow p_{i} \propto e^{- β E_{i}}

Normalization then gives the Boltzmann form above.