# Show Reference: "Model Selection and multimodal Inference---A Practical Information-Theoretic Approach"

Model Selection and multimodal Inference---A Practical Information-Theoretic Approach (2002) by Kenneth P. Burnham, David R. Anderson
@book{burnham-and-anderson-2002,
address = {New York, Berlin, Heidelberg},
author = {Burnham, Kenneth P. and Anderson, David R.},
citeulike-article-id = {4425594},
edition = {Second},
isbn = {0-387-95364-7},
keywords = {model-selection, statistics},
posted-at = {2014-06-16 11:24:17},
priority = {2},
publisher = {Springer},
title = {Model Selection and multimodal {Inference---A} Practical {Information-Theoretic} Approach},
year = {2002}
}



Kullback-Leibler divergence $D_{KL}(P,Q)$ between probability distributions $P$ and $Q$ can be interpreted as the information lost when approximating $P$ by $Q$.
For discrete probability distributions $P,E$ with the set of outcomes $E$, Kullback-Leibler divergence is defined as $$D_{KL}(P,Q)=\sum_{e\in E} P(e)\log\left(\frac{P(e)}{Q(e)}\right).$$