Sufficient Statistics & Rao-Blackwell

The concept of sufficiency originates with R. A. Fisher’s 1922 paper, On the Mathematical Foundations of Theoretical Statistics (§7 of that paper introduces “sufficient statistics” as one of the cornerstones of his program). Fisher’s argument was operational: estimators that ignore the sufficient statistic discard usable information, and one should prefer those that don’t. The measure-theoretic formalization — the modern statement and proof of the factorization theorem in full generality — came nearly thirty years later in Halmos & Savage (1949), Application of the Radon-Nikodym Theorem to the Theory of Sufficient Statistics. The Halmos–Savage paper handles the dominated-family case rigorously via the Radon–Nikodym derivative, which is what Topic 16 §16.3 quietly invokes when it speaks of densities $f(x; \theta) = dP_\theta/d\mu$ for a common dominating measure $\mu$.

Sufficiency is a strong claim: $T$ is a lossless compression of the data with respect to $\theta$. Once you know $T$, the rest of the data is parameter-free noise — the conditional distribution $X \mid T$ does not depend on $\theta$ at all. This is information-theoretic in spirit but does not require Shannon information; it is a structural statement about the family $\{P_\theta\}$. The compression is sometimes dramatic — for $n$ Bernoulli trials, the sufficient statistic is a single integer in $\{0, 1, \ldots, n\}$; for $n$ Normal observations with both parameters unknown, it’s a 2-vector $(\sum X_i, \sum X_i^2)$. Topics 7, 13, 14, and 15 used this fact implicitly. §16.10 will reveal why the compression stays small only in exponential families.

A statistic $T(X)$ is sufficient for $\theta$ if the conditional distribution of $X$ given $T(X) = t$ does not depend on $\theta$ — that is, $P_\theta(X \in \cdot \mid T = t)$ is the same for every $\theta \in \Theta$ and every $t$ in the range of $T$.

Operationally: knowing $T$ leaves no further parameter information in the data. The residual randomness of $X \mid T$ is “noise” in the strict sense that its distribution is fixed regardless of the truth.

A vector statistic $T(X) = (T_1(X), \ldots, T_k(X))$ is jointly sufficient for a parameter $\theta = (\theta_1, \ldots, \theta_p) \in \Theta$ if the conditional distribution of $X$ given $T(X) = t$ does not depend on $\theta$. The dimension $k$ of the sufficient statistic need not equal the parameter dimension $p$ — though when $T$ is minimal (§16.4), and the family is regular (§16.10), the two dimensions match.

For iid data $X_1, \ldots, X_n$ from any common distribution $P_\theta$, the order statistic $T(X) = (X_{(1)}, \ldots, X_{(n)})$ is sufficient for $\theta$. Topic 29 makes the order statistic the central object of nonparametric statistics — Track 8 begins there precisely because this theorem tells us no nonparametric procedure ever discards information in the sorted sample.

Let $X_1, \ldots, X_n$ be iid $\mathcal{N}(\mu, \sigma^2)$ with $\sigma^2$ known. Define $T = \sum_{i=1}^n X_i$. We check sufficiency directly: the conditional density of $X$ given $T = t$ is

$f(x \mid T = t; \mu) \;=\; \frac{f(x; \mu)}{f_T(t; \mu)} \cdot \mathbf{1}\!\left\{\sum x_i = t\right\}.$

The numerator is the iid Normal density $(2\pi\sigma^2)^{-n/2}\exp(-\sum(x_i - \mu)^2 / 2\sigma^2)$. Expanding the squared-deviation sum,

$\sum (x_i - \mu)^2 \;=\; \sum x_i^2 - 2\mu\sum x_i + n\mu^2 \;=\; \sum x_i^2 - 2\mu t + n\mu^2.$

The denominator $f_T(t;\mu)$ is the density of $T \sim \mathcal{N}(n\mu, n\sigma^2)$, which contains the same $-2\mu t / 2\sigma^2 = -\mu t/\sigma^2$ term and the same $n\mu^2/2\sigma^2$ term. The two cancel exactly, leaving the conditional density independent of $\mu$. Hence $T$ is sufficient.

Let $X_1, \ldots, X_n$ be iid $\mathrm{Bernoulli}(p)$, and define $T = \sum X_i$. The joint pmf is $p^t (1-p)^{n-t}$ where $t = T(x)$. Conditional on $T = t$, every binary vector $x \in \{0,1\}^n$ with $\sum x_i = t$ has probability

$P_p(X = x \mid T = t) \;=\; \frac{p^t(1-p)^{n-t}}{\binom{n}{t}p^t(1-p)^{n-t}} \;=\; \binom{n}{t}^{-1},$

uniform over the $\binom{n}{t}$ vectors in the level set. The conditional is parameter-free — $T$ is sufficient.

The constant statistic $T(X) = n$ is trivially sufficient: the “conditional distribution of $X$ given $T = n$” is just the marginal joint distribution of $X$, which depends on $\theta$ in general — so this is not a counterexample, just a degenerate case of the definition. The genuinely degenerate sufficient statistic is the identity, $T(X) = X$ itself, which is sufficient because conditioning on the full data leaves nothing to vary. Both extremes — too coarse to compress, too fine to compress — motivate minimal sufficiency (§16.4): the coarsest sufficient statistic that still captures everything.

The conditional-distribution definition (Def 1) is conceptually clean but operationally heavy — checking it requires writing out the joint density and dividing through, as in Examples 1 and 2. The Fisher–Neyman factorization theorem (§16.3) gives an equivalent and easier operational test: $T$ is sufficient iff the joint density factors as $g(T(x); \theta) \cdot h(x)$. The two definitions agree for dominated families (the standard setting), with the equivalence proved in §16.3. From here forward, factorization is the tool of choice; the conditional definition is the conceptual anchor.

Let $\{P_\theta : \theta \in \Theta\}$ be a family of distributions on a measurable space $(\mathcal{X}, \mathcal{A})$, dominated by a common $\sigma$-finite measure $\mu$, with densities $f(x; \theta) = dP_\theta/d\mu$. A statistic $T : \mathcal{X} \to \mathcal{T}$ is sufficient for $\theta$ if and only if there exist non-negative measurable functions $g(\cdot; \theta)$ on $\mathcal{T} \times \Theta$ and $h(\cdot)$ on $\mathcal{X}$ such that

$f(x; \theta) \;=\; g(T(x); \theta) \cdot h(x) \qquad \mu\text{-a.s.}$

For $X_1, \ldots, X_n$ iid $\mathcal{N}(\mu, \sigma^2)$ with both parameters unknown, the joint density expands to

$f(x; \mu, \sigma^2) \;=\; (2\pi\sigma^2)^{-n/2}\exp\!\left(-\tfrac{1}{2\sigma^2}\sum x_i^2 \;+\; \tfrac{\mu}{\sigma^2}\sum x_i \;-\; \tfrac{n\mu^2}{2\sigma^2}\right).$

Setting $T_1 = \sum x_i$ and $T_2 = \sum x_i^2$, the parameter-dependent factor is $g(T_1, T_2; \mu, \sigma^2) = (2\pi\sigma^2)^{-n/2}\exp(\mu T_1/\sigma^2 - T_2/(2\sigma^2) - n\mu^2/(2\sigma^2))$ and the data-only factor is $h(x) = 1$. Hence $T = (\sum X_i, \sum X_i^2)$ is jointly sufficient — a 2-vector matching the parameter dimension.

For $X_1, \ldots, X_n$ iid $\mathrm{Uniform}(0, \theta)$, the joint density is

$f(x; \theta) \;=\; \theta^{-n} \cdot \mathbf{1}\!\left\{0 \le x_{(1)} \text{ and } x_{(n)} \le \theta\right\} \;=\; \underbrace{\theta^{-n} \mathbf{1}\!\{x_{(n)} \le \theta\}}_{g(T;\theta)} \;\cdot\; \underbrace{\mathbf{1}\!\{x_{(1)} \ge 0\}}_{h(x)},$

so $T(X) = X_{(n)} = \max_i X_i$ is sufficient by factorization. The factorization is clean — but the family is not an exponential family: the support $[0, \theta]$ depends on $\theta$, the support indicator cannot be absorbed into the exp-family form, and Topic 7’s construction does not apply. This example will return in §16.10 as the canonical counterexample to the Pitman–Koopman–Darmois theorem: a non-exp-family with a fixed-dimensional sufficient statistic, escaping the converse only by violating the regularity condition that support not depend on $\theta$.

For $X_1, \ldots, X_n$ iid $\mathrm{Poisson}(\lambda)$, the joint pmf is

$f(x;\lambda) \;=\; \prod_{i=1}^n \frac{e^{-\lambda}\lambda^{x_i}}{x_i!} \;=\; \underbrace{e^{-n\lambda}\lambda^{\sum x_i}}_{g(T;\lambda)} \;\cdot\; \underbrace{\left(\prod_i x_i!\right)^{-1}}_{h(x)},$

so $T = \sum X_i$ is sufficient. As an exponential family in canonical form $f(x;\lambda) = (x!)^{-1}\exp(x\log\lambda - \lambda)$, the natural sufficient statistic is exactly this sum.

Compare Example 1 (the direct conditional check for Normal $\mu$, which required expanding the squared-deviation sum and showing two terms cancel) with Example 4 (the factorization, which is two lines). For most families that arise in practice, factorization gives the sufficient statistic immediately by inspection: write down the joint density, group terms by $\theta$-dependence, and read off $T$. The conditional-distribution definition remains the conceptual anchor — it tells us why sufficiency means data reduction — but the factorization theorem is what we compute with.

A sufficient statistic $T$ is minimal sufficient for $\theta$ if, for every other sufficient statistic $T'$, there exists a measurable function $\phi$ with $T = \phi(T')$ almost surely. Equivalently, the partition of the sample space induced by $T$ is the coarsest partition under which sufficiency holds.

Suppose the family $\{P_\theta\}$ has densities $f(x;\theta)$ with respect to a common dominating measure. A statistic $T(X)$ is minimal sufficient for $\theta$ if and only if, for every pair of sample points $x, y$,

$\frac{f(x; \theta)}{f(y; \theta)} \text{ does not depend on } \theta \;\Longleftrightarrow\; T(x) = T(y).$

For iid Normal data with both parameters unknown, the likelihood ratio is

$\frac{f(x;\mu,\sigma^2)}{f(y;\mu,\sigma^2)} \;=\; \exp\!\left(\tfrac{\mu}{\sigma^2}\bigl[\textstyle\sum x_i - \sum y_i\bigr] - \tfrac{1}{2\sigma^2}\bigl[\sum x_i^2 - \sum y_i^2\bigr]\right).$

This is constant in $(\mu, \sigma^2)$ if and only if $\sum x_i = \sum y_i$ AND $\sum x_i^2 = \sum y_i^2$ — equivalently, $\bar x = \bar y$ AND $S^2(x) = S^2(y)$. By Theorem 3, $T(X) = (\bar X, S^2)$ is minimal sufficient. The dimension matches the parameter dimension — a feature shared by every regular exponential family (and forced, under regularity, by Pitman–Koopman–Darmois).

For Normal data with $\sigma^2$ known, $T = \bar X$ is minimal sufficient. The expanded statistic $T' = (\bar X, X_1)$ is also sufficient (any function from which $T$ can be recovered inherits sufficiency), but it is not minimal: for any pair of samples $x, y$ with $\bar x = \bar y$ but $x_1 \ne y_1$, the likelihood ratio is constant in $\mu$ (since the joint density depends on $x$ only through $\bar x$), but $T'(x) \ne T'(y)$. Theorem 3’s biconditional fails, so $T'$ is not minimal. Carrying $X_1$ along is information-wasting: it refines the partition without buying any inferential content.

A theorem due to Bahadur (1957) states that a complete sufficient statistic is minimal sufficient. The converse is false: the standard counterexample is Uniform$(\theta, \theta+1)$, where the minimal sufficient statistic $T = (X_{(1)}, X_{(n)})$ is not complete (Example 13, §16.6). Completeness is a strictly stronger property than minimality. The Lehmann–Scheffé theorem (§16.7) requires completeness, not just minimality, because it needs to rule out the existence of multiple unbiased functions of $T$ — and minimality alone does not.

Given an estimator $\hat\theta(X)$ and a sufficient statistic $T(X)$, the Rao-Blackwellized estimator is

$\tilde\theta(T) \;=\; \mathbb{E}[\hat\theta(X) \mid T].$

By sufficiency, the conditional distribution of $X \mid T$ does not depend on $\theta$, so the conditional expectation is a function of $T$ alone (not of $\theta$) — hence a statistic. The construction is constructive: integrate $\hat\theta(X)$ against the parameter-free conditional distribution $X \mid T$.

Let $T$ be sufficient for $\theta$ and let $\hat\theta$ be an unbiased estimator of $\theta$ with finite variance. Define $\tilde\theta = \mathbb{E}[\hat\theta \mid T]$. Then:

1. $\tilde\theta$ is a statistic (function of $T$ alone).
2. $\tilde\theta$ is unbiased: $\mathbb{E}_\theta[\tilde\theta] = \theta$.
3. $\mathrm{Var}_\theta(\tilde\theta) \le \mathrm{Var}_\theta(\hat\theta)$, with equality iff $\hat\theta = \tilde\theta$ almost surely (i.e., $\hat\theta$ was already a function of $T$).

Since both estimators are unbiased, MSE = variance, so $\mathrm{MSE}(\tilde\theta) \le \mathrm{MSE}(\hat\theta)$.

For iid $\mathrm{Bernoulli}(p)$ with $T = \sum X_i$ sufficient, the crude estimator $\hat p_0 = X_1$ is unbiased ($\mathbb{E}[X_1] = p$) but lies in $\{0, 1\}$ — variance is $p(1-p)$, the maximum possible for an estimator with this support. Conditioning on $T$:

$\tilde p \;=\; \mathbb{E}[X_1 \mid T = t] \;=\; \frac{t}{n} \;=\; \bar X,$

since by exchangeability all $X_i$ have the same conditional distribution given $T$, and their conditional sum is $T$. The Rao-Blackwellized estimator is the sample mean, with $\mathrm{Var}(\bar X) = p(1-p)/n$ — a factor $n$ smaller than $\mathrm{Var}(X_1)$. The RaoBlackwellImprover above visualizes this dramatic shrinkage at $n = 20$.

Suppose we want to estimate $g(\lambda) = P_\lambda(X = 0) = e^{-\lambda}$ from iid $\mathrm{Poisson}(\lambda)$ data. The crude estimator $\hat g_0 = \mathbf{1}\{X_1 = 0\}$ is unbiased ($\mathbb{E}[\mathbf{1}\{X_1 = 0\}] = e^{-\lambda}$), but it is a 0/1 indicator. Conditioning on $T = \sum X_i$ (which is $\mathrm{Poisson}(n\lambda)$):

$\tilde g \;=\; \mathbb{E}[\mathbf{1}\{X_1 = 0\} \mid T = t] \;=\; P(X_1 = 0 \mid \textstyle\sum X_i = t).$

The conditional distribution of $(X_1, \ldots, X_n)$ given $\sum X_i = t$ is $\mathrm{Multinomial}(t; 1/n, \ldots, 1/n)$. So $X_1 \mid T = t \sim \mathrm{Binomial}(t, 1/n)$, and $P(X_1 = 0 \mid T = t) = (1 - 1/n)^t$. The Rao-Blackwellized estimator is

$\tilde g \;=\; \left(1 - \tfrac{1}{n}\right)^{\sum X_i},$

which is the UMVUE of $e^{-\lambda}$ (since $\sum X_i$ is complete sufficient — see §16.6 Lemma 1 — and Lehmann–Scheffé applies). A 0/1 estimator promoted to a smooth, sample-size-aware estimator with provably minimum variance among unbiased estimators.

Suppose we want to estimate $\sigma^2$ for iid $\mathcal{N}(\mu, \sigma^2)$ with $\mu$ known. The crude estimator $\hat\sigma^2_0 = (X_1 - \mu)^2$ is unbiased ($\mathbb{E}[(X_1 - \mu)^2] = \sigma^2$) but uses only one observation. Conditioning on the sufficient statistic $T = \sum (X_i - \mu)^2 \sim \sigma^2 \chi^2_n$:

$\tilde\sigma^2 \;=\; \mathbb{E}[(X_1 - \mu)^2 \mid T = t] \;=\; \frac{t}{n} \;=\; \frac{1}{n}\sum (X_i - \mu)^2,$

by exchangeability of $(X_i - \mu)^2$ given $T$. The variance ratio is $n$ — a factor-$n$ reduction. We finish this calculation in §16.7 Example 14, where Lehmann–Scheffé promotes it to the UMVUE.

Rao–Blackwell does more than prove the existence of a better estimator — it tells you exactly what it is: the conditional expectation $\mathbb{E}[\hat\theta \mid T]$, computable by integrating against the parameter-free conditional distribution $X \mid T$. For the canonical families (Bernoulli, Poisson, Normal, Exponential, Gamma) the conditional distribution has a closed form, and the Rao-Blackwellized estimator is a direct calculation. This constructive aspect is what makes the theorem operational: Rao (1945) and Blackwell (1947) independently realized that every unbiased estimator can be mechanically upgraded by this recipe, without needing to know whether a UMVUE even exists.

A family $\{P_\theta : \theta \in \Theta\}$ of distributions of a statistic $T$ is complete if, for every measurable function $g$ with $\mathbb{E}_\theta[|g(T)|] < \infty$ for all $\theta$,

$\mathbb{E}_\theta[g(T)] \;=\; 0 \text{ for all } \theta \in \Theta \;\Longrightarrow\; g(T) \;=\; 0 \text{ almost surely under every } P_\theta.$

A statistic $T$ is complete sufficient for $\theta$ if it is sufficient and the family of its distributions $\{P_\theta^T : \theta \in \Theta\}$ is complete. The statistic is boundedly complete if the implication above holds for every bounded measurable $g$. Bounded completeness is strictly weaker than full completeness, and in fact suffices for the Lehmann–Scheffé theorem (§16.7 Remark 7).

Let $T$ be the canonical sufficient statistic of an exponential family $f(x; \eta) = h(x)\exp(\eta^\top T(x) - A(\eta))$ with natural parameter $\eta$ ranging over an open subset $H \subseteq \mathbb{R}^k$. Then $T$ is complete for $\eta$.

By Lemma 1, every exponential family in its natural parameter has a complete sufficient statistic. Concretely:

• Bernoulli($p$): $T = \sum X_i \sim \mathrm{Binomial}(n, p)$ is complete for $p$.
• Binomial($n, p$) with $n$ known: $T = X$ itself is complete for $p$.
• Poisson($\lambda$): $T = \sum X_i \sim \mathrm{Poisson}(n\lambda)$ is complete for $\lambda$.
• Normal($\mu, \sigma^2$) with $\sigma^2$ known: $T = \bar X$ is complete for $\mu$.
• Normal($\mu, \sigma^2$) jointly: $T = (\bar X, S^2)$ is complete for $(\mu, \sigma^2)$.
• Gamma($\alpha, \beta$) with $\alpha$ known: $T = \sum X_i$ is complete for $\beta$.
• Exponential($\lambda$): $T = \sum X_i$ is complete for $\lambda$.

Each delivers a UMVUE via Lehmann–Scheffé (§16.7).

Consider $X_1, \ldots, X_n$ iid $\mathrm{Uniform}(\theta, \theta + 1)$ — a unit-width interval shifted by $\theta$. The minimal sufficient statistic is the pair $T = (X_{(1)}, X_{(n)})$ — two-dimensional for a one-dimensional parameter, an immediate structural warning sign.

Why incompleteness? Because $\theta$ is a pure location parameter, the range $R = X_{(n)} - X_{(1)}$ is ancillary for $\theta$: its distribution depends only on $n$, not on $\theta$. (Specifically, $R$ has mean $(n-1)/(n+1)$ and a $\mathrm{Beta}(n-1, 2)$-shaped density, both free of $\theta$.) Therefore the centered range

$g(T) \;=\; R \;-\; \frac{n-1}{n+1} \;=\; (X_{(n)} - X_{(1)}) \;-\; \frac{n-1}{n+1}$

satisfies $\mathbb{E}_\theta[g(T)] = 0$ for every $\theta \in \mathbb{R}$ — yet $g(T)$ is not identically zero. This is the definition of incompleteness: a non-trivial function of $T$ with zero expectation under every $P_\theta$.

The CompletenessProbe above lets you visualize this directly: choose Uniform$(\theta, \theta+1)$, evaluate the centered range across the $\theta$-grid, and see the empirical $\mathbb{E}_\theta[g(T)] \approx 0$ curve. By contrast, on the same grid a non-witness like $g'(T) = X_{(1)} - 1/(n+1)$ traces a non-zero linear function of $\theta$ — illustrating the difference between an ancillary-derived witness (which makes incompleteness visible) and a generic test function (which does not).

The Lehmann–Scheffé theorem (§16.7) is usually proved using full completeness. But a closer look at the proof shows that only bounded completeness is needed — the only test functions that arise are differences $h_1(T) - h_2(T)$ between unbiased estimators of the same quantity, and these are bounded whenever both have finite second moment. Bounded completeness is strictly weaker than full completeness, and a small number of important families (notably some non-regular location-scale families) are boundedly complete without being completely complete. For the canonical exponential families in this topic, the distinction does not matter — all are fully complete by Lemma 1.

An estimator $\tilde\theta$ of a parameter (or parametric function) $g(\theta)$ is the uniformly minimum-variance unbiased estimator (UMVUE) if:

1. $\tilde\theta$ is unbiased: $\mathbb{E}_\theta[\tilde\theta] = g(\theta)$ for every $\theta \in \Theta$.
2. For every other unbiased estimator $\hat\theta$ of $g(\theta)$, $\mathrm{Var}_\theta(\tilde\theta) \le \mathrm{Var}_\theta(\hat\theta)$ for every $\theta \in \Theta$.

The “uniformly” applies to the parameter — the variance dominance must hold simultaneously at every $\theta$, not just on average or at a particular value. UMVUE existence is non-trivial: there are families with no UMVUE at all (Remark 8). When a UMVUE exists, it is essentially unique (Theorem 6).

Let $T$ be a complete sufficient statistic for $\theta$, and let $\tilde\theta = h(T)$ be an unbiased estimator of $g(\theta)$ that is a function of $T$. Then $\tilde\theta$ is the unique UMVUE of $g(\theta)$.

If a UMVUE of $g(\theta)$ exists, then it is unique up to almost-sure equality under every $P_\theta$ — that is, any two UMVUE candidates $\tilde\theta_1$ and $\tilde\theta_2$ agree on a set of $P_\theta$-measure 1 for every $\theta$.

Let $X_1, \ldots, X_n$ be iid $\mathcal{N}(\mu, \sigma^2)$ with $\mu$ known and $\sigma^2$ unknown. The complete sufficient statistic is $T = \sum (X_i - \mu)^2$, which has distribution $\sigma^2 \chi^2_n$. The unbiased function of $T$ for $\sigma^2$ is

$\hat\sigma^2_{\text{UMVUE}} \;=\; \frac{T}{n} \;=\; \frac{1}{n}\sum_{i=1}^n (X_i - \mu)^2,$

since $\mathbb{E}[T] = n\sigma^2$. By Lehmann–Scheffé (Theorem 5), this is the UMVUE of $\sigma^2$ in this model. Variance: $\mathrm{Var}(\hat\sigma^2_{\text{UMVUE}}) = \mathrm{Var}(T)/n^2 = (2n\sigma^4)/n^2 = 2\sigma^4/n$ — exactly the CRLB (Topic 13 §13.7). The UMVUE is efficient in this case.

Note the crucial assumption $\mu$ known. With $\mu$ unknown the model is exactly the same family but with two parameters, and the complete sufficient statistic becomes $(\bar X, \sum(X_i - \bar X)^2)$ — leading to the famous Bessel correction; see Example 19 below.

Let $X_1, \ldots, X_n$ be iid $\mathrm{Gamma}(\alpha, \beta)$ with shape $\alpha$ known and rate $\beta$ unknown — an exponential family in $\beta$. The complete sufficient statistic is $T = \sum X_i \sim \mathrm{Gamma}(n\alpha, \beta)$, with $\mathbb{E}[T^{-1}] = \beta / (n\alpha - 1)$ for $n\alpha > 1$ (a standard Gamma reciprocal-moment identity). Hence the unbiased function of $T$ for $\beta$ is

$\hat\beta_{\text{UMVUE}} \;=\; \frac{n\alpha - 1}{\sum X_i}.$

By Lehmann–Scheffé this is the UMVUE. Compare with the MLE / MoM: $\hat\beta_{\text{MLE}} = \hat\beta_{\text{MoM}} = \alpha / \bar X = n\alpha / \sum X_i$. The two estimators differ by exactly the bias-correction factor $(n\alpha - 1)/(n\alpha)$ — a small but non-trivial gap that the §16.11 UMVUEComparator visualizes. This is the example that fulfills the method-of-moments.mdx:1062 promise.

Two cautions worth flagging. First, a UMVUE need not exist. In families without a complete sufficient statistic, the set of unbiased estimators may have no member that uniformly dominates the others — the Lehmann–Scheffé existence proof relies critically on completeness. Second, even when the UMVUE exists, it does not always attain the Cramér–Rao lower bound. The CRLB is a bound on variance, derived from the information inequality; UMVUE is the best unbiased estimator. The UMVUE attains the CRLB precisely in full-rank exponential families with the parameter being the natural one — exactly the same condition for which MLE = UMVUE = MoM coincide (§16.11 Theorem 9). Outside that boundary, the UMVUE has variance strictly above the CRLB, and the gap measures the unattainability of efficient estimation in that family.

$T = \sum X_i$ is complete sufficient for $p$, and $\mathbb{E}[T/n] = p$, so $\hat p_{\text{UMVUE}} = \bar X$. The MLE is also $\bar X$ (Topic 14, Example 14.1), and so is the MoM (Topic 15, Example 15.3). All three coincide — the Bernoulli is exp family in its natural parameter, so Theorem 9 (§16.11) applies. Variance: $\mathrm{Var}(\bar X) = p(1-p)/n$, which equals the CRLB $1/I(p) = p(1-p)/n$ — efficient.

$T = \sum X_i \sim \mathrm{Poisson}(n\lambda)$ is complete sufficient for $\lambda$, and $\mathbb{E}[T/n] = \lambda$, so $\hat\lambda_{\text{UMVUE}} = \bar X$. The MLE (Topic 14, Example 14.4) and MoM (Topic 15, §15.2) both also equal $\bar X$. Triple coincidence again, and again CRLB-attaining: $\mathrm{Var}(\bar X) = \lambda/n = 1/I(\lambda)$.

With $\sigma^2$ known, $T = \bar X \sim \mathcal{N}(\mu, \sigma^2/n)$ is complete sufficient and unbiased for $\mu$, hence the UMVUE. The MLE is also $\bar X$ (Topic 14, Example 14.2 with $\sigma^2$ fixed), and the MoM is too (Topic 15, Example 15.1). All three coincide; the variance $\sigma^2/n$ equals the CRLB.

With $\mu$ also unknown, the complete sufficient statistic is the pair $T = (\bar X, \sum (X_i - \bar X)^2)$. The unbiased function of $T$ for $\sigma^2$ is the Bessel-corrected sample variance,

$\hat\sigma^2_{\text{UMVUE}} \;=\; S^2_{n-1} \;=\; \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2,$

since $\sum (X_i - \bar X)^2 \sim \sigma^2 \chi^2_{n-1}$ and $\mathbb{E}[\chi^2_{n-1}] = n - 1$. The MLE and MoM, by contrast, both land at the un-corrected

$\hat\sigma^2_{\text{MLE}} \;=\; \hat\sigma^2_{\text{MoM}} \;=\; S^2_n \;=\; \frac{1}{n}\sum_{i=1}^n (X_i - \bar X)^2,$