Fisher information: how well can you know your parameter
Curvature of the log-likelihood at its peak. Sharp curvature = small standard error = parameter well-identified. Flat curvature = big SE = parameter the data refuses to pin down.
Every calibration output reports standard errors next to its parameters. Fisher information is the formal quantity those SEs come from: the *negative expected curvature* of the log-likelihood at the true parameter. Sharp curvature means the likelihood drops fast as you move away from the maximum, so the MLE is well-localised. Flat curvature means there's a whole region of parameters that fit the data nearly as well, so the MLE is wobbly. The Cramér-Rao bound makes this precise: $\mathrm{Var}(\hat\theta) \ge 1/I_n(\theta)$ for any unbiased estimator, with the MLE asymptotically achieving the bound. This tutorial covers the score function, the information matrix, the CR bound, the asymptotic normality of the MLE, and the practical identifiability story for two-parameter models like Heston's $\kappa, \theta$.
✓ Intro · expand
Try first (productive failure)
Before the worked example: spend 60 seconds taking your best shot at this.
A guess is fine — being briefly wrong about a problem makes the explanation
land harder when you read it. This appears once per tutorial; skip
if you already know the trick.
60s
✓ Try first · expand
Worked example
You have $n = 100$ iid samples from a Normal distribution with unknown mean $\mu$ and known variance $\sigma^2 = 4$. Compute the Fisher information $I_n(\mu)$, the Cramér-Rao lower bound on the variance of any unbiased estimator of $\mu$, and the resulting standard error.
✓ Worked example · expand
Practice 1 of 3Type a fraction, decimal, or expression — mathjs parses it.
✓ Practice · expand
Reflection
In practice the FIM has off-diagonal entries that entangle parameters (Heston's $\kappa$ and $\theta$ are the textbook case). Why does this make calibration unstable even when each parameter looks well-identified on its own, and what does the FIM eigendecomposition tell you about which *directions* in parameter space are identifiable?