Maximum entropy principle

When only partial information about a distribution is available (e.g., a known expected value from experiments), the principle of maximum entropy states:

Among all distributions consistent with the known constraints, choose the one with maximum entropy.

This gives the least biased estimate possible, introducing no assumptions beyond the data.

For example, if repeated rolls of a die yield an average of 5 (instead of the expected 3.5 for a fair die), the distribution maximizing entropy under this constraint reflects the most impartial model given that average. This led to a Boltzmann distribution.