Concentration parameter

In probability theory and statistics, a concentration parameter is a special kind of numerical parameter of a parametric family of probability distributions. Concentration parameters occur in two kinds of distribution: In the Von Mises–Fisher distribution, and in conjunction with distributions whose domain is a probability distribution, such as the symmetric Dirichlet distribution and the Dirichlet process. The rest of this article focuses on the latter usage. The larger the value of the concentration parameter, the more evenly distributed is the resulting distribution (the more it tends towards the uniform distribution). The smaller the value of the concentration parameter, the more sparsely distributed is the resulting distribution, with most values or ranges of values having a probability near zero (in other words, the more it tends towards a distribution concentrated on a single point, the degenerate distribution defined by the Dirac delta function). In the case of multivariate Dirichlet distributions, there is some confusion over how to define the concentration parameter. In the topic modelling literature, it is often defined as the sum of the individual Dirichlet parameters, when discussing symmetric Dirichlet distributions (where the parameters are the same for all dimensions) it is often defined to be the value of the single Dirichlet parameter used in all dimensions. This second definition is smaller by a factor of the dimension of the distribution. A concentration parameter of 1 (or k, the dimension of the Dirichlet distribution, by the definition used in the topic modelling literature) results in all sets of probabilities being equally likely, i.e., in this case the Dirichlet distribution of dimension k is equivalent to a uniform distribution over a k-1-dimensional simplex. Note that this is not the same as what happens when the concentration parameter tends towards infinity. In the former case, all resulting distributions are equally likely (the distribution over distributions is uniform).