Derivation of formulae
for weighted averaging
Define weighted mean or average as:
y = x_{1} w_{1} + x_{2} w_{2} + ... + x_{N} w_{N} = 
N Σ
i = 1

x_{i} w_{i}, 
 (1) 
where w_{i} are weights and x_{i} are arithmetic averages themeselves.
Assume all x_{i} come from averaging of the same stationary normal (Gaussian)
process with mean x and standard deviation sigma (σ),
and the averaging is done over the same number of samples. This means that expected
value of the standard deviation of each of x_{i} is the same and is
√N times higher than σ:
Our derivation is nevertheless carried out as if all
σ_{i} were different since we would like to relate
them to the measured variances, which are different from each other.
The expected value of the weighted mean is
< y > = < Σ x_{i}
w_{i} > = Σ w_{i}
< x_{i} > = x Σ w_{i}. 

Thus, in order that y will represent an unbiased estimate of the mean value
we need this condition:
The quantity y has the variance:
v = < (y –
< y > )^{2} > = < y^{2} >
– < y > ^{2}. 

We have < y^{2} > = <
(Σ x_{i} w_{i})^{2} >
= < Σ x_{i} x_{j} w_{i}
w_{j} > = Σ w_{i} w_{j}
< x_{i} x_{j} > . Noting that < x_{i} x_{j} > = < (x_{i} –
x + x)( x_{j} –
x + x) > and < x_{i} –
x > = 0 while
< x_{i} –
x > < x_{j} – x >
is nonzero only for i = j, when it assumes the value of expected variance of
the ith mean, v_{i} = σ_{i}^{2},
we straightforwardly arrive at
< y^{2} > = x^{2} Σ w_{i}
w_{j} + Σ σ_{i}^{2} w_{i}^{2},
so that the expected variance of y is
x^{2} Σ w_{i} w_{j} +
Σ σ_{i}^{2}w_{i}^{2} –
(x Σ w_{i})^{2} or just
v = Σ σ_{i}^{2} w_{i}^{2} = Σ v_{i} w_{i}^{2}. 
 (2) 
This is quite general formula for the variance of weighted mean.
Now we define the following weighting:
w_{i} = 
1/v_{i}
Σ1/v_{i}

, 
 (3) 
where v_{i} are sample variances of real measurements. These weights satisfy
the condition of summing up to 1.
Inserting them into the expression of Eq. (2) we obtain
v = Σ
v_{i}[(1/v_{i})/Σ
(1/v_{i})]^{2} = (Σ
v_{i}/v_{i}^{2}) / (Σ
1/v_{i})^{2},
that reduces to this very simple equation
This formula is known in literature in the form
(see e.g. Wikipedia):
A hastily made search for derivation of this expression turned out unsuccessful
therefore we present it here in this small document. An attentive reader might
have noted an inconsistency in the above derivation of Eq. (4).
Namely, the variances in Eq. (3) assume
the sample values while those in Eq. (2) are the expected values! One might have
substituted in Eq. (2) N σ^{2}
for v_{i} to easily arrive at more proper final formula:
v = Nσ^{2}
Σ(1/v_{i})^{2}/(Σ
1/v_{i})^{2}.
Unfortunately such a formula is much more complex and would require computation
of some approximation to σ (could be just the rms of
all the data). Sticking to this less
strict solution of Eq. (4), we believe, should not cause any problems in our
VLBI practice since ultimately the measured values converge to those expected
and the error estimate need not to exactly reflect the true dispersion.
It can be also shown (see this box) that the same formula
obtains by the Gaussian error propagation method (valid for small errors).
Earlier we have assumed v_{i} = N σ^{2}
so that if the data are not contaminated
we can expect Eq. (4) to lead to the variance of
1/[Σ 1/(N σ^{2})] = σ^{2},
i.e. the same as while calculating the arithmetic (not weighted) mean of the same data.
To sum up, the variance of the weighted mean computed according to Eqs. (1) and (3)
can be calculated according to Eq. (4), thus the error estimate of such a mean is
SIG =  1^{ }
(
Σ1/v_{i} )^{1/2}

.

 (5) 
In case of N = 2 and our weighting mathod of Eq. (3) equtions for
the mean and variance simplify to
x = 
x_{1} v_{2} + x_{2} v_{1}
v_{1} + v_{2}

. 
 (6) 
v = SIG^{2} = 
v_{1} v_{2}
v_{1} + v_{2}

. 
 (7) 
Approximation by error propagation method
Suppose the dispersions σ_{i} of our means
are small, i.e. measurement errors Δx_{i}
in x_{i} are small. General formula for calculation of error
in weighted mean y is in this case (provided errors in the x_{i}
samples are completely independent):
Since (∂y/∂x_{i}) Δx_{i}
= w_{i} Δx_{i} = [(1/σ_{i}^{2}) /
(Σ1/σ_{j}^{2})] Δx_{i} by substituting Δx_{i} = σ_{i}
we obtain
Δy = 
√ 
Σ
 
( 
1/σ_{i}^{2}
Σ1/σ_{j}^{2}

σ_{i}  ) 
2


= 
1

=  ( 
Σ
 
1
v_{j}
 ) 
–1/2

, 

which is the same as Eq. (5). 
— K.M. Borkowski
Posted: May 29, 2007; last updated: June 11, 2007
File translated from
T_{E}X
by
T_{T}H,
version 3.77, on 25 May 2007.