Home > Estimation > Estimation of a distribution from i.i.d. sums

Estimation of a distribution from i.i.d. sums

August 1, 2009

Here’s an estimation problem that I ran into not long ago while working on a problem in entity co-reference resolution in natural language documents.

Let $X$ be a random variable taking on values in $\{0,1,2,\ldots\}$.  We are given data $D=\{(N_1,S_1), (N_2,S_2),\ldots,(N_k,S_k)\}$, where $S_i$ is the sum of $N_i$ independent draws of $X$ for $i = 1, 2,\ldots, k$. We are required to estimate the distribution of $X$ from $D$.

For some distributions of $X$ we can use the method-of-moments. For example if $X \sim \mbox{Poisson}(\lambda)$, we know that the mean of $X$ is $\lambda$. We can therefore estimate $\lambda$ as the sample mean, i.e., $\hat{\lambda}=\frac{S_1+S_2+\ldots+S_k}{N_1+N_2+\ldots+N_k}$. Because of the nice additive property of the parameters for sums of i.i.d. poisson random variables, the maximum likelihood estimate also turns out be the same as $\hat{\lambda}$.

The problem becomes more difficult when $X$ is say a six-sided die (i.e., the sample space is $\{1,2,3,4,5,6\}$) and we would like to estimate the probability of the faces . How can one obtain the maximum likelihood estimate in such a case?