Home > Estimation > Estimation of a distribution from i.i.d. sums

Estimation of a distribution from i.i.d. sums

August 1, 2009

Here’s an estimation problem that I ran into not long ago while working on a problem in entity co-reference resolution in natural language documents.

Let X be a random variable taking on values in \{0,1,2,\ldots\}.  We are given data D=\{(N_1,S_1), (N_2,S_2),\ldots,(N_k,S_k)\}, where S_i is the sum of N_i independent draws of X for i = 1, 2,\ldots, k. We are required to estimate the distribution of X from D.

For some distributions of X we can use the method-of-moments. For example if X \sim \mbox{Poisson}(\lambda), we know that the mean of X is \lambda. We can therefore estimate \lambda as the sample mean, i.e., \hat{\lambda}=\frac{S_1+S_2+\ldots+S_k}{N_1+N_2+\ldots+N_k}. Because of the nice additive property of the parameters for sums of i.i.d. poisson random variables, the maximum likelihood estimate also turns out be the same as \hat{\lambda}.

The problem becomes more difficult when X is say a six-sided die (i.e., the sample space is \{1,2,3,4,5,6\}) and we would like to estimate the probability of the faces . How can one obtain the maximum likelihood estimate in such a case?

Categories: Estimation
%d bloggers like this: