Derivation of formulae
for weighted averaging


(Document prepared in connection with this report)

Define weighted mean or average as:
y = x1 w1 + x2 w2 + ... + xN wN = N
Σ
i = 1 
xi wi,
(1)
where wi are weights and xi are arithmetic averages themeselves. Assume all xi come from averaging of the same stationary normal (Gaussian) process with mean x and standard deviation sigma (σ), and the averaging is done over the same number of samples. This means that expected value of the standard deviation of each of xi is the same and is √N times higher than σ:

σi = √N σ.
Our derivation is nevertheless carried out as if all σi were different since we would like to relate them to the measured variances, which are different from each other.
The expected value of the weighted mean is
< y > = < Σ xi wi > = Σ wi < xi > = x Σ wi.
Thus, in order that y will represent an unbiased estimate of the mean value we need this condition:
Σwi = 1.
The quantity y has the variance:
v = < (y – < y > )2 > = < y2 > – < y > 2.
We have < y2 > = < (Σ xi wi)2 > = < Σ xi xj wi wj > = Σ wi wj < xi xj > . Noting that < xi xj > = < (xi – x + x)( xj – x + x) > and < xi – x > = 0 while < xi – x > < xj – x > is nonzero only for i = j, when it assumes the value of expected variance of the i-th mean, vi = σi2, we straightforwardly arrive at
< y2 > = x2 Σ wi wj + Σ σi2 wi2,
so that the expected variance of y is
x2 Σ wi wj + Σ σi2wi2 – (x Σ wi)2 or just
v = Σ σi2 wi2 = Σ vi wi2.
(2)
This is quite general formula for the variance of weighted mean. Now we define the following weighting:
wi  =   1/vi

Σ1/vi
,
(3)
where vi are sample variances of real measurements. These weights satisfy the condition of summing up to 1. Inserting them into the expression of Eq. (2) we obtain v = Σ vi[(1/vi)/Σ (1/vi)]2 = (Σ vi/vi2) / (Σ 1/vi)2, that reduces to this very simple equation
v  =   1

Σ1/vi
.
(4)
This formula is known in literature in the form (see e.g. Wikipedia):

v = 1

Σ1/σi2
.
A hastily made search for derivation of this expression turned out unsuccessful therefore we present it here in this small document. An attentive reader might have noted an inconsistency in the above derivation of Eq. (4). Namely, the variances in Eq. (3) assume the sample values while those in Eq. (2) are the expected values! One might have substituted in Eq. (2) N σ2 for vi to easily arrive at more proper final formula:

v = Nσ2 Σ(1/vi)2/(Σ 1/vi)2.

Unfortunately such a formula is much more complex and would require computation of some approximation to σ (could be just the rms of all the data). Sticking to this less strict solution of Eq. (4), we believe, should not cause any problems in our VLBI practice since ultimately the measured values converge to those expected and the error estimate need not to exactly reflect the true dispersion. It can be also shown (see this box) that the same formula obtains by the Gaussian error propagation method (valid for small errors).
Earlier we have assumed vi = N σ2 so that if the data are not contaminated we can expect Eq. (4) to lead to the variance of 1/[Σ 1/(N σ2)] = σ2, i.e. the same as while calculating the arithmetic (not weighted) mean of the same data.
To sum up, the variance of the weighted mean computed according to Eqs. (1) and (3) can be calculated according to Eq. (4), thus the error estimate of such a mean is

SIG   =   1 

( Σ1/vi )1/2
.

(5)

In case of N = 2 and our weighting mathod of Eq. (3) equtions for the mean and variance simplify to

x  =   x1 v2 + x2 v1

v1 + v2
.
(6)

v  =  SIG2 =   v1 v2

v1 + v2
.
(7)



Approximation by error propagation method

Suppose the dispersions σi of our means are small, i.e. measurement errors Δxi in xi are small. General formula for calculation of error in weighted mean y is in this case (provided errors in the xi samples are completely independent):

Δy   =    √

Σ
( ∂y

∂xi
Δxi) 2

 
 
.
Since (∂y/∂xi) Δxi  =  wi Δxi = [(1/σi2) / (Σ1/σj2)] Δxi by substituting Δxi = σi we obtain

Δy   =    √

Σ
( 1/σi2

Σ1/σj2
σi) 2

 
 
  =  
1 




Σ1/σj2
  =  ( Σ
1

vj
) 1/2

 
,
which is the same as Eq. (5).


— K.M. Borkowski  

Posted: May 29, 2007; last updated: June 11, 2007  


File translated from TEX by TTH, version 3.77, on 25 May 2007.