[2003SM/LNP-03] Lecture-03-- Uncertainty in Statistics

For an experiment whose outcomes of simple events 1,...n have probabilities \(p_1, p_2, ...p_n\), the uncertainty, \(H(p)\) ,is defined a

$$ H(p_1, p_2, .., p_n) = − \sum_n   p_n \ln p_n $$

Some important properties of uncertainty are Taking several examples, several properties of the uncertainty are brought out. The uncertainty is maximum when all probabilities are equal. It is zero when one of the events has probability1 and all other events have zero probability. The uncertainty for case of two independent random variables is the sum of individual uncertainties. By means of examples, it is shown that increase (decrease) in uncertainty is associated with decrease (increase) in information.