Statistics on Randy Lai

MSE of sample standard deviation (Lean by AI)

Wed, 21 Oct 2026 00:00:00 +0000

Back in 2023, I wrote a post using SymPy to compute the second order expansion of the mean squared error (MSE) of the sample standard deviation up to $O(n^{-2})$. Earlier this year, I revisited the problem in MSE of sample standard deviation (Gen AI) and showed that a modern LLM can derive the entire second order expansion directly from a prompt.

When an AI produces a long asymptotic derivation with dozens of polynomial moment terms, how can we trust that every coefficient is right and that no terms of order $n^{-2}$ were dropped? To check the derivation rigorously, I formalized and verified the entire proof in Lean 4 with Mathlib.

MSE of sample standard deviation (Gen AI)

Tue, 20 Jan 2026 00:00:00 +0000

In the era of gen AI, I am trying to use LLM to compute the second order expansion of $MSE(S)$ that I did in 2023.

The standard sample s.d. is given by S=sqrt(\sum_{i=1}^n(x_i - \bar x)^2/ (n-1)). Without assumption the underlying distribution, compute the MSE(S) of it in terms of the first few moments of x_i up. Expand the terms up to 1/n^2.

And it just nails it: https://gemini.google.com/share/fa35ab43436f

MSE of sample standard deviation (cont')

Thu, 20 Apr 2023 00:00:00 +0000

Continue our previous post of computing the MSE of sample standard deviation. We want to have higher order expansion of $E(S_n)$. The algebric computation is a bit tedious, we will use sympy to help us.

Open notebook in full screen.

The result is quite interesting. For normal distribution, the MSE of $S_n$ and $\hat \sigma$ are the same up to the order $1/n^2$. If the distribution has a thicker tail than normal, i.e. kurtosis > 3, then $\hat \sigma_n$ will have a smaller MSE than $S_n$. On the other hand, if the distribution has a lighter tail than normal, than $S_n$ will have a smaller MSE.

MSE of sample standard deviation

Wed, 19 Apr 2023 00:00:00 +0000

In a previous post, we have discussed the MSE of sample variance. One could argue that we should compute the MSE of the sample standard deviation as sample s.d. has the same unit as the measurements.

Let $S_n$ be the sample s.d. and $\sigma$ be the true s.d., the goal is to compute

$$ MSE(S_n) = E(S_n-\sigma)^2 = E(S_n^2) - 2 \sigma E(S_n) + \sigma^2 = 2 \sigma^2 - 2 \sigma E(S_n). $$

To compute $E(S_n)$, first, we consider Taylor’s expanding $g(x) = \sqrt{x}$ about $x=\sigma^2$, we have

MSE of sample variance

Sat, 01 Aug 2020 00:00:00 +0000

In introductory statistics courses, we often focus on unbiased estimators such as the sample variance,

$$ S^2 = \frac{\sum(X_i - \bar X)^2}{n - 1} $$

rather than a more initiative estimator

$$ \hat \sigma^2 = \frac{\sum(X_i - \bar X)^2}{n}. $$

The common argument is that $S^2$ is unbiased whereas $\hat \sigma^2$ is biased for estimating $\sigma^2$. While it is true, this statement is a bit over-stressed. A more statistical way to evaluate performance of estimators is to compare the mean squared error (MSE) of the estimators.

Variance of sample mean squared

Sat, 01 Aug 2020 00:00:00 +0000

It is well known that $\bar X^2$ is a biased estimator for $\mu^2$, but how about its variance?

Before we start working on $Var(\bar X^2)$, let’s recall that $\bar X$ is an unbiased estimator for $\mu$, i.e. $E(\bar X) = \mu$. Moreover, $Var(\bar X) = \sigma^2/n$ and $E(\bar X^2) = Var(\bar X) + E(\bar X)^2 = \frac{\sigma^2}{n} + \mu^2$.

Let $Z_i = X_i - \mu$,

$$ \begin{align*} E\left[(\sum X_i)^4\right] &= E\left[(n \mu + \sum Z_i)^4\right]\\ &= E\left[[n^4 \mu^4 + 4 n^3 \mu^3 \sum Z_i+ 6 n^2 \mu^2 (\sum Z_i)^2 + 4 n \mu (\sum Z_i)^3 + (\sum Z_i)^4 \right]. \end{align*} $$

By the fact that,

Girl ratio

Sun, 10 Sep 2017 00:00:00 +0000

https://stats.stackexchange.com/questions/93830/expected-number-of-ratio-of-girls-vs-boys-birth

The Zorganian Republic has some very strange customs. Couples only wish to have female children as only females can inherit the family’s wealth, so if they have a male child they keep having more children until they have a girl. If they have a girl, they stop having children. What is the ratio of girls to boys in Zorgania?

Let $X$ be the number of children in a family, it is geometric random variable with $p=0.5$, i.e.,

Two cool things about dice!

Thu, 14 Apr 2016 00:00:00 +0000

Just discover two cool things about dice, Suppose we have two (fair) 6-sided dice, let $Y$ be the reminder of the product of the two numbers divided by 7, i.e., $Y = X1 \times X2 (\text{ mod } 7)$, guess what the distribution of $Y$ is? Uniform 1,…, 6

If it doesn’t surprise you, now consider two n-sided dice and repeat the above with $Y = X_1 \times X_2 (\text{ mod } n+1)$. Guess what values of $n$ will yield a uniform distribution? It turns out that the values of $n$ are prime numbers minus 1. (As 6 = 7-1 and 7 is a prime number, a regular 6-sided dice has this property).