discrete information entropy

symbols, variables and units

p(x): probability; H: bit; agreed to 0 log 0=0.

applicable conditions and boundaries

Discrete distribution; continuous differential entropy differs from this definition and can be negative.

formula source code

The following is a copyable LaTeX expression.

H(X)=-\sum_x p(x)\log_2p(x)

Reference and Extended Learning

is organized according to model definition and assumptions. Please check actual conditions and original literature before engineering, research and clinical use.