Unvoreingenommener Schätzer des Exponentialmaßes einer Menge?

Angenommen, wir haben eine (messbare und angemessen gut erzogene) Menge $S\subseteq B\subset\mathbb R^n$ , wobei $B$ kompakt ist. Nehmen wir außerdem an, wir können Proben aus der gleichmäßigen Verteilung über $B$ über das Lebesgue-Maß $\lambda(\cdot)$ und das Maß $\lambda(B)$ . Zum Beispiel ist $B$ vielleicht eine Box $[-c,c]^n$ die $S$ .

für festes $\alpha\in\mathbb R$ eine einfache unvoreingenommene Möglichkeit, $e^{-\alpha \lambda(S)}$ zu schätzen, indem Punkte in $B$ gleichmäßig abgetastet und überprüft werden, ob sie innerhalb oder außerhalb von $S$ ?

Nehmen wir als Beispiel für etwas an, das nicht ganz funktioniert, nehmen wir $k$ Punkte $p_1,\ldots,p_k\sim\textrm{Uniform}(B)$ . Dann können wir die Monte - Carlo - Schätzung verwenden

λ (S) \approx \hat{λ} := \frac{# {p_{i} \in S}}{k} λ (B) .

$\lambda(S)\approx \hat\lambda:= \frac{\#\{p_i\in S\}}{k}\lambda(B).$ Doch während

ein unverzerrter Schätzer von ist

, ich glaube nichtdass es der Falldass

ist ein unverzerrter Schätzer von

. Gibt es eine Möglichkeit, diesen Algorithmus zu ändern?

\hat{λ}

$\hat\lambda$

λ (S)

$\lambda(S)$

e^{- α \hat{λ}}

$e^{-\alpha\hat\lambda}$

e^{- α λ (S)}

$e^{-\alpha\lambda(S)}$

probability monte-carlo unbiased-estimator measure-theory Justin Solomon
quelle

Antworten:

Angenommen, Ihnen stehen folgende Ressourcen zur Verfügung:

Sie haben Zugriff auf einen Schätzer $\hat{\lambda}$ .
$\hat{\lambda}$ ist für unvoreingenommene $\lambda ( S )$ .
$\hat{\lambda}$ ist fast sicher nach oben beschränkt durch $C$ .
Sie kennen die Konstante $C$ und
Sie können unabhängige Realisierungen bilden so oft wie Sie möchten. $\hat{\lambda}$

Beachten Sie nun, dass für jedes $u > 0$ Folgendes gilt (durch die Taylor-Erweiterung von $\exp x$ ):

\begin{aligned} e^{- α λ (S)} & = e^{- α C} \cdot e^{α (C - λ (S))} \\ = e^{- α C} \cdot \sum_{k ⩾ 0} \frac{{(α [C - λ (S)])}^{k}}{k!} \\ = e^{- α C} \cdot e^{u} \cdot \sum_{k ⩾ 0} \frac{e^{- u} \cdot {(α [C - λ (S)])}^{k}}{k!} \\ = e^{u - α C} \cdot \sum_{k ⩾ 0} \frac{u^{k} e^{- u}}{k!} {(\frac{α [C - λ (S)]}{u})}^{k} \end{aligned}

$\begin{align} e^{-\alpha \lambda ( S ) } &= e^{-\alpha C} \cdot e^{\alpha \left( C - \lambda ( S ) \right)} \\ &= e^{- \alpha C} \cdot \sum_{k \geqslant 0} \frac{ \left( \alpha \left[ C - \lambda ( S ) \right] \right)^k}{ k! } \\ &= e^{- \alpha C} \cdot e^u \cdot \sum_{k \geqslant 0} \frac{ e^{-u} \cdot \left( \alpha \left[ C - \lambda ( S ) \right] \right)^k}{ k! } \\ &= e^{u -\alpha C} \cdot \sum_{k \geqslant 0} \frac{ u^k e^{-u} }{ k! } \left(\frac{ \alpha \left[ C - \lambda ( S ) \right]}{u} \right)^k \end{align}$

Gehen Sie nun wie folgt vor:

Probe $K \sim \text{Poisson} ( u )$ .
Form als IId unvoreingenommene Schätzer von $\hat{\lambda}_1, \cdots, \hat{\lambda}_K$ $\lambda(S)$ .
Geben Sie den Schätzer zurück

\hat{Λ} = e^{u - α C} \cdot {(\frac{α}{u})}^{K} \cdot \prod_{i = 1}^{K} {C - {\hat{λ}}_{i}} .

$\hat{\Lambda} = e^{u -\alpha C} \cdot \left(\frac{ \alpha }{u} \right)^K \cdot \prod_{i = 1}^K \left\{ C - \hat{\lambda}_i \right\}.$

$\hat{\Lambda}$ ist dann eine nicht negative, unverzerrter Schätzer von $\lambda(S)$ . Das ist weil

\begin{aligned} E [\hat{Λ} | K] & = e^{u - α C} \cdot {(\frac{α}{u})}^{K} E [\prod_{i = 1}^{K} {C - {\hat{λ}}_{i}} | K] \\ = e^{u - α C} \cdot {(\frac{α}{u})}^{K} \prod_{i = 1}^{K} E [C - {\hat{λ}}_{i}] \\ = e^{u - α C} \cdot {(\frac{α}{u})}^{K} \prod_{i = 1}^{K} [C - λ (S)] \\ = e^{u - α C} \cdot {(\frac{α}{u})}^{K} {[C - λ (S)]}^{K} \end{aligned}

$\begin{align} \mathbf{E} \left[ \hat{\Lambda} | K \right] &= e^{u -\alpha C} \cdot \left(\frac{ \alpha }{u} \right)^K \mathbf{E} \left[ \prod_{i = 1}^K \left\{ C - \hat{\lambda}_i \right\} | K \right] \\ &= e^{u -\alpha C} \cdot \left(\frac{ \alpha }{u} \right)^K \prod_{i = 1}^K \mathbf{E} \left[ C - \hat{\lambda}_i \right] \\ &= e^{u -\alpha C} \cdot \left(\frac{ \alpha }{u} \right)^K \prod_{i = 1}^K \left[ C - \lambda ( S ) \right] \\ &= e^{u -\alpha C} \cdot \left(\frac{ \alpha }{u} \right)^K \left[ C - \lambda ( S ) \right]^K \end{align}$

and thus

\begin{aligned} E [\hat{Λ}] & = E_{K} [E [\hat{Λ} | K]] \\ = E_{K} [e^{u - α C} \cdot {(\frac{α}{u})}^{K} {[C - λ (S)]}^{K}] \\ = e^{u - α C} \cdot \sum_{k ⩾ 0} P (K = k) {(\frac{α}{u})}^{K} {[C - λ (S)]}^{K} \\ = e^{u - α C} \cdot \sum_{k ⩾ 0} \frac{u^{k} e^{- u}}{k!} {(\frac{α [C - λ (S)]}{u})}^{k} \\ = e^{- α λ (S)} \end{aligned}

$\begin{align} \mathbf{E} \left[ \hat{\Lambda} \right] &= \mathbf{E}_K \left[ \mathbf{E} \left[ \hat{\Lambda} | K \right] \right] \\ &= \mathbf{E}_K \left[ e^{u -\alpha C} \cdot \left(\frac{ \alpha }{u} \right)^K \left[ C - \lambda ( S ) \right]^K \right] \\ &= e^{u -\alpha C} \cdot \sum_{k \geqslant 0} \mathbf{P} ( K = k ) \left(\frac{ \alpha }{u} \right)^K \left[ C - \lambda ( S ) \right]^K \\ &= e^{u -\alpha C} \cdot \sum_{k \geqslant 0} \frac{ u^k e^{-u} }{ k! } \left(\frac{ \alpha \left[ C - \lambda ( S ) \right]}{u} \right)^k \\ &= e^{-\alpha \lambda ( S ) } \end{align}$

by the earlier calculation.

πr8
quelle

Interesting! Doesn't the estimator for

\hat{λ}

$\hat\lambda$ described in the question work here, since it's bounded above by

λ (B) < \infty

$\lambda(B)<\infty$ ? Also how come this doesn't contradict @whuber 's answer below? Is there an easy argument why this is unbiased? Sorry for many questions, my probability theory is weak :-)

Justin Solomon

The estimator you describe works, since you know

λ (B)

$\lambda (B)$ . I think this doesn't contradict the other answer because of assumption

5

$5$ ; given finite access to unbiased estimators, I don't think this construction would work. The unbiasedness comes by comparing the expectation of

\hat{Λ}

$\hat{\Lambda}$ to the power series above; I'll make that clearer in the answer.

πr8

Are you sure you can interchange the product and expectation in the second line of the proof of unbiasedness?

jbowman

Seems like it's ok because they're computed iid, right?

Justin Solomon

+1 I think this is an interesting and instructional example. It succeeds by not making an assumption implicit to my answer: that the sample size is either specified or at least bounded.

whuber

The answer is in the negative.

A sufficient statistic for a uniform sample is the count $X$ of points observed to lie in $S.$ This count has a Binomial $(n,\lambda(S)/\lambda(B))$ distribution. Write $p=\lambda(S)/\lambda(B)$ and $\alpha^\prime = \alpha\lambda(B).$

For a sample size of $n,$ let $t_n$ be any (unrandomized) estimator of $\exp(-\alpha \lambda(S)) = \exp(-(\alpha\lambda(B)) p) = \exp(-\alpha^\prime p).$ The expectation is

E [t_{n} (X)] = \sum_{x = 0}^{n} (\binom{n}{x}) p^{x} (1 - p)^{n - x} t_{n} (x),

$E[t_n(X)] = \sum_{x=0}^n \binom{n}{x}p^x (1-p)^{n-x}\, t_n(x),$

which equals a polynomial of degree at most $n$ in $p.$ But if $\alpha^\prime p \ne 0,$ the exponential $\exp(-\alpha^\prime p)$ cannot be expressed as a polynomial in $p.$ (One proof: take $n+1$ derivatives. The result for the expectation will be zero but the derivative of the exponential, which itself is an exponential in $p,$ cannot be zero.)

The demonstration for randomized estimators is nearly the same: upon taking expectations, we again obtain a polynomial in $p.$

Consequently, no unbiased estimator exists.

whuber
quelle

Ah, that's a downer! Thanks for the nice proof. But, the Taylor series for

\exp (t)

$\exp(t)$ converges fairly quickly --- perhaps there's an "approximately unbiased" estimator out there? Not sure what that means (I'm not much of a statistician :-) )

Justin Solomon

How quickly, exactly? The answer depends on the value of

α^{'} p

$\alpha^\prime p$ --and therein lies your problem, because you don't know what that value is. You know only that it lies between

0

$0$ and

α .

$\alpha.$ You could use that to establish a bound on the bias if you like.

whuber

In my application I expect

S

$S$ to occupy a large portion of

B

$B$ . I'd like to use this value in a pseudo-marginal Metropolis-Hastings acceptance ratio, not sure if that method can handle even controllable levels of bias...

Justin Solomon

BTW I'd really appreciate your thoughts on the other answer to this question!

Justin Solomon