MSE of sample standard deviation
In a previous post, we have discussed the MSE of sample variance. One could argue that we should compute the MSE of the sample standard deviation as sample s.d. has the same unit as the measurements.
Let Sn be the sample s.d. and σ be the true s.d., the goal is to compute
MSE(Sn)=E(Sn−σ)2=E(S2n)−2σE(Sn)+σ2=2σ2−2σE(Sn).
To compute E(Sn), first, we consider Taylor’s expanding g(x)=√x about x=σ2, we have g(x)=σ+12σ(x−σ2)−18σ3(x−σ2)2+R(x), where R(x)=−(18˜σ3−18σ3)(x−σ2)2 for some ˜σ between √x and σ.
Let κ=E(X−μ)4/σ4 be the kurtosis. It could be shown that E[√n(S2n−σ2)]2→σ4(κ−1) and nER(S2n)→0 (and the proofs are beyond the discussion of this post. One could consider the fact that √n(S2n−σ2) converges to N(0,σ4(κ−1))).
Thus, E(Sn)=Eg(S2n)=σ+12σE(S2n−σ2)−18σ3E(S2n−σ2)2+o(n−1). =σ−σ8(κ−1n)+o(n−1).
Therefore, MSE(Sn)=σ24(κ−1n)+o(n−1).
Similar calculation could be done for ˆσn=√∑(Xi−ˉX)2n,
MSE(ˆσn)=E[√n−1nSn−σ]2=E[(1−12n+o(n−1))Sn−σ]2=σ24(κ−1n)+o(n−1).Unfortunately, MSE of Sn and ˆσn match at the first order. We will need to consider higher order expansion next time.