## Probability density estimation as classification

Perhaps it has always been obvious to exalted statistical minds that density estimation can be viewed as classification and (perhaps) done using a classifier.

Assume that we have samples of a random vector whose distribution has bounded support. In fact, without loss of generality, let the support be the unit hypercube . We are required to estimate the density of .

Now assume that we generate a bunch of samples uniformly distributed in . We assign a label to all the samples and a label to all and a build a classifier between the two sample sets. In other words we construct an estimate of the posterior class probability .

Now, we know that

where , the uniform distribution over the unit hypercube. The above equation can be solved for to obtain an estimate

Because is in our control, ideally we would like to obtain

The question is, because we know the distribution of the samples for class 0 (uniform!), for any particular classifier (say the Gaussian process classifier or logistic regression) can the limit be computed/approximated without actually sampling and then learning?

This paper (which I haven’t yet read) may be related.

Update Aug 24, 2009. The uniform distribution can be substituted by any other proposal distribution from which we can draw samples and which has a support that includes the support of the density we wish to estimate. George, thanks for pointing this out.