Astron. Astrophys. 157,^{ }91–95 (1986) 

Toruń Radio Astronomy Observatory, Nicolaus Copernicus University,
ul. Chopina 12/18, PL87100 Toruń, Poland
Received July 4, 1983; accepted August 9, 1985
Summary. The analysis presented in this paper shows that the
usual small signal approximation in VLBI is acceptable in the
majority of applications. However, it leads to nonlinear systematic
overestimation of the correlation coefficient, which in some
cases cannot be considered negligible. For example, the correlation
of about 0.26 estimated from the fringe amplitude of 0.1, as
measured with the Mark II system, is 3.4% too high. Formulae
suitable for accurate conversion of measured amplitudes into the
correlation coefficient for a few useful fringe rotation schemes
are given. The analysis also brings evidence for a small bias in
the phase measurements made with present day VLBI correlators,
which is entirely due to imperfect fringe rotation. Arguments
are presented in favour of fivelevel rotators.
Key words: VLBI observables – data analysis – instruments
1. IntroductionVery long baseline interferometry (VLBI) is a relatively new powerful technique for studying radio source structures, their positions and baseline orientations with angular accuracies as high as a fraction of milliarcsecond and in special cases approaching the microarcsecond level. The high resolution and sensitivity of this technique may be easily degraded by imperfect signal processing. By comparison with results obtained with the MERLIN, Wilkinson (1983) has demonstrated that the dynamic range of VLBI maps is considerably less than might be expected. In fact it is by no means a rare event that VLBI users report the presence of unknown sources of errors in their data. Factors that place limits on accuracy of VLBI measurements are numerous and generally well documented in literature. Here we shall concentrate on two which received little, if any, attention and which result in relatively small but systematic baseline related errors. One of them arises with the use of a small signal approximation to the Van Vleck relation when determining the correlation coefficient from the fringe amplitude measurements made with digital VLBI systems. In effect the usual signal processing leads to somewhat biased estimates of the correlation. The other seemingly overlooked source of biases, this time affecting also the measured phase of fringes, results from the use of an imperfect fringe rotation in VLBI correlators. This problem is less fundamental and can be dealt with by improving the correlation process. A correlator with a fivelevel fringe rotator would perform much better in this respect, offering in addition considerably smaller losses in the signal to noise ratio (hereafter SNR). 
2. Expected fringesThe process of correlation of VLBI data involves a fringe rotation, or fringe stopping, that is the multiplication of one of the two data streams by an approximation to a sinusoid whose frequency is set at a predicted rate of the fringes. The fringe rotation and correlation of two binary data streams, x_{i} and y_{i}, yield the stream of
The Van Vleck relation (Van Vleck, Middleton 1966) is
Now it is important to observe that the r is not a sinusoid, and that the expected SNR after correlation of the original signals may be expressed by

 (6) 
which is followed by another approximation,

both of which are obviously good for very small r_{o}. Note, however, once again that variations in signal transform nonlinearly into variations of the correlation coefficient.
Neglecting the difference between (5) and (6) already gives rise to an error. The infinite clipping leads to another nonlinear transformation — that expressed by Eq. (3). These, already serious, problems with analytical treatment of the expected correlator output are still further complicated by the approximation of the FRF. Nevertheless, the output can easily be predicted numerically using Eqs. (2), (3) and (5). The results of such an approach are presented in Sect. 4.
In order to check the validity of the results to be presented in the next section and to introduce our notation here we first recall the commonly used small signal approximation and then derive a large signal approximation.
With small signals, r_{o} ≈ 0, the sampled form of Eq. (3) may be safely approximated by
 (7) 
For a large number of averaged samples, which is almost always the case in VLBI practice, the sums in (2) may be approximated by the integrals. Taking now for the FRF a pure sinewave one obtains
 (8) 
In case of the threelevel FRF, which is a squarewave zeroed over intervals of length π – 2α centred at each of its zerocrossings, the result of integration is
 (9) 
In both these cases the sine channel output gives r_{s}(φ_{o}) = r_{c}(π/2 – φ_{o}).
It is clear from the above that to calculate the correlation coefficient, r_{o}, from the fringe amplitude measurements, R = (r_{s}^{2} + r_{c}^{2})^{1/2}, a proportionality factor of r_{o}/R is needed. It equals 2, or (π/2)α/sinα when the unquantized sinewave, or the threelevel FRF, respectively, are used. The factor α/sinα appears in literature under the numerical value of 1.28 corresponding to α = 3π/8 (e.g. Reid et al., 1980). The fringe phase, determined as Φ = atg(r_{s}/r_{c}), appears here as the unbiased estimate of the true phase φ_{o}.
When a signal is very large, i.e. the correlation coefficient is very close to 1, the Van Vleck relation reduces to
 (10) 
and the expected correlator output at the cosine channel becomes
 (11) 
for the sinewave FRF, and
 (12) 
for the threelevel FRF. Outside the ±π/2 range of phase these outputs possess the cosine function periodicity with the symmetry r_{c}(φ_{o}) = –r_{c}(π – φ_{o}). The relation r_{s}(φ_{o}) = r_{c}(π/2 – φ_{o}) now holds also true.
The amplitude of the fringes is here twice that of the small signal approximation in the sinewave FRF case: R = r_{o}. For the threelevel FRF it is now also phase dependent and has the magnitude
 (13) 
 (14) 
The formulae presented in this section for the threelevel FRF apply also to the twolevel FRF, for which α = π/2. Similar analyses are easily carried out for higherlevel quantized FRFs.
In general, the VLBI signals to be correlated are neither small nor large enough for the considered approximations to be used for calculation of the true observables without introducing biases. We have computed the expected outputs, for a number of r_{o} values, using the complete model for fringes (see Sect. 2):
 (15) 
About 2000 equidistant samples taken over one cycle of the model fringes were thus averaged with the help of the R32 computer (IBM 360 near equivalent), to yield the correlator responses for each set of the initial fringe parameters (i.e. r_{o} and φ_{o}) and for a few different FRFs. Some of the numerical results are presented in Table 1. The results, whenever appropriate, were found to agree within five significant digits with the analytical results obtained for the small and large signal approximations as derived in the previous section. As could be expected, the output amplitude and phase biases, if they exist, display the π/2 periodicity with respect to, and are even functions of, the true phase.
Table 1. Expected correlator output amplitudes (R) for two fringe rotation functions (FRF) and some fringe phases, φ_{o}, as a function of the correlation coefficient, r_{o} 

When the unquantized sinewave is used as the FRF, the correlator output amplitude R depends only on the input correlation coefficient r_{o}, and not on the phase φ_{o}. Using the least squares fit to some data points it was found that the correlation coefficient can be determined from measurements of the amplitude through the following approximation
 (16) 
with an accuracy of 0.05%. The expected fringe phase Φ for such a correlator is error free, i.e. it is equal to the true phase φ_{o}.
Correlators with other FRFs produce outputs that depend on both, the correlation coefficient and the fringe phase. In case of optimally designed three and higherlevel correlators these outputs are very close to the output of the sinewave FRF correlator. The closeness is such that the formula (16) may be used always, whenever errors not exceeding 2% of r_{o} are acceptable, regardless of actual phase. The following formula represents somewhat better fit for the common threelevel correlator:
 (17) 
which departures from the true values less than 1% below r_{o} = 0.5, and less than 1.5% elsewhere, for all phases. This fit cannot be made much better without accounting for the phase dependence of the real output. The phase measurements for this correlator are in systematic error. The error (see insert in Fig. 1.) falls monotonically from about 4° at r_{o} = 1, to 1° at r_{o} = 0.5, to vanish at r_{o} = 0 (these are the maximum values as the φ_{o} is varied).
A near optimal correlator with a fivelevel FRF (α = 13π/30 plus doubling the x_{i}y_{i} product over the interval of 7π/15 placed at the center of each nonzero interval of the threelevel FRF) would have still closer response to that with the ideal FRF. The measured phases would differ from the true phases by no more than 0.1° for r_{o} < 0.65, and by less than 0.5° for r_{o} < 0.99. The following approximation
 (18) 

would be applicable for such a correlator within the tolerance of deviation of 0.3% of r_{o} for r_{o} < 0.6, and for all phases.
VLBI signal processing normally introduces various corrections to the measured observables. The nonlinearity of the correlator response analysed in this paper rises practical question: to which quantity a particular correction should be applied? Besides the Van Vleck correction and the one due to the shape of an FRF, which are explicitly included in Eqs. (16) through (18), there are three other categories of corrections: those that apply to (1) the measured amplitude — R; (2) the correlation coefficient — r_{o}; and (3) the signal to noise ratio — μ_{o} = r_{o}/(1 – r_{o}). In our opinion the first category should include factors due to the following effects: smoothing caused by averaging of not exactly stopped fringes, loss of quadrature between channels, and bias (nonzero mean) of the clippersampler (this loss is proportional to the sum of the absolute values of the biases in the two data streams, assuming the biases are small). Other effects, such as those due to the finite coherence time of interferometer, the delay tracking by integral bit shifts, or the aliased noise from outside the standard frequency band, belong either to the second or to the third category depending on the way the numerical factors were derived.
With the Mark II system a composite correction factor of about 2.6 is used to convert the measured amplitudes into the correlation coefficient. From this value, according to our distinction, only a factor k = 1.3 remains to be applied to (17) to make the formula practically applicable:
 (19) 
Though in the above it was assumed that the k factor applies fully to the correlation coefficient, the multiplicatory coefficient related to the signal to noise ratio (μ_{o}) can, if required, be portioned out of it and applied separately to r^{^}_{o}/(1 – r^{^}_{o}) to yield the final μ^{^}_{o}.
With assumption of small signals the Eq. (19) is reduced back to the commonly used approximation
 (20) 

The quality of this approximation to the correlation coefficient can now be easily judged by comparison of the last two equations. While for very small amplitudes the difference is evidently negligible, it becomes more and more significant as the signal grows. More convincing insight is given by Fig. 2., which illustrates the effect of neglecting the nonlinearity on the estimates of the SNR, μ_{o} (or corresponding ratio of antenna temperature to system temperature), which is essential for the flux density scale calibration. The plotted error, the overestimation, has been calculated as the difference between the small signal approximated μ_{o} and the true μ_{o}, in percent of the latter. Note that an error in μ_{o} is always greater than the corresponding error in r_{o}, although for small signals the difference is again negligible.
In practice, with a pair of large telescopes and reasonably good system temperatures, the correlation coefficient, ro as in (17), of 0.1 may be exceeded for sources of the flux density of only a few Janskys. At this level of correlation the discussed overestimation of flux density is already about 2%. Consequently, if such a source is taken for the calibrator, then all fluxes ofweaker sources are underestimated by up to 2%. The dependence of this error on actual flux will cause distortions in maps of strong sources because different baselines produce different correlations being a function of the source visibility and the telescope sensitivities.
As a result of imperfect fringe rotation the phase of the measured correlation coefficient is also slightly biased. This bias is, however, of negligibly small magnitude for practically all astrophysical applications of VLBI (with the threelevel FRF it is smaller than 1° for r_{o} < 0.5). Anyway, it is worth remembering that the resulting errors do not close in the closure phase observable. In applications which require very precise phase measurements, such as in astrometry, geodesy and navigation, this phase bias may be a source of appreciable errors, especially when strong artificial sources are used. Such errors again do not cancel in differential observables, because they depend upon the actual fringe phase on a given baseline. Note, for instance, that the phase error as small as 0.1° on a 10^{6} λ baseline is equivalent to the apparent source position offset of about 60 microarcseconds.
Having in mind the inconveniences caused by the imperfect threelevel fringe rotation we have analysed also the properties of a fivelevel correlator, which differs from the former only in that it requires the doubling of the correlation product around the extrema ofthe corresponding sine rotation function. The analysis showed that the phase bias here is reduced about 10 times, as compared to the threelevel FRF case. Additional study, not presented in this paper, proves also that the fivelevel correlator would introduce 3 times smaller losses in SNR than does the threelevel correlator. To emphasize this point, note that the 4% loss in SNR of the presently used correlators is equivalent to the dropout of about 8% of recorded data, i.e. roughly one hour in every 12 hours of observations. With the fivelevel correlator this loss reduces to about 20 minutes only.
Presented analysis of the digital radio interferometer response shows that the output amplitude of fringes is in general a highly nonlinear function of the strength of an input signal as expressed by the correlation coefficient, or the signal to noise ratio. The cause of this nonlinearity lies in the behavior of the clippersampler embodied in the Van Vleck relation. Further complication is due to the quantization of the lobe rotation function which introduces a phase dependence to the correlator output.
In case of the threeIevel fringe rotator, which presently is almost exclusively used in digital VLBI correlators, the nonlinearity, if neglected, produces a bias (overestimation) in the signal power determination. The bias is smaller than 2% if the correlation coefficient, r_{o}, remains below about 0.1 or, equivalently, the measured output amplitude of the fringes is less than 0.055 for the Mark II and Mark III system. This error grows quickly for greater amplitudes (see Fig. 2.). The normal signal processing fails to account for this nonlinearity. In fact, the usual small signal approximation breaks down completely near the center of the output amplitude range. The analysis revealed also a small bias in the phase of the fringes as measured with the use of the threelevel fringe rotation function. Its magnitude is smaller than 1° for r_{o} < 0.5 (see insert in Fig. 1).
For three fringe rotation schemes of practical interest approximate, relatively simple formulae are presented [Eqs. (16) through (18)] that allow one to calculate the correlation coefficient with biases reduced to less than 1.5% for all measured amplitudes and phases.
Considering the drawbacks of the threelevel correlator, arguments were put forward in favour of a fivelevel correlator. Such a correlator has the potential of not only almost eliminating the fringe phase influence on observables, but also of reducing the signal to noise ratio losses by a factor of 3, i.e. from 4% to about 1.3%.
Finally, the placement of various correction factors in signal processing was discussed. Distinction was made between effects that cause losses in the fringe amplitude as measured by a VLBI correlator, in the correlation coefficient, and in the signal to noise ratio. This was necessary because, in view of the analysed nonlinearity, a correction of each of these quantities has in general a different effect on the final result.
Acknowledgements. The author thanks Dr. A. Kus and Prof. S. Gorgolewski for numerous discussions and help in preparation of this paper. Thanks are also due to Dr. D. Graham who as the reviewer suggested to combine two earlier papers originally submitted to this Journal in July 1983 and April 1984 which, along with other criticisms, resulted in this improved version.
References
Reid, M.J., Haschick, A.D., Burke, B.F., Moran, J.M., Johnston, K.J., Swenson, G.W.: 1980, Astrophys. J. 239, 89
Wilkinson, P.N.: 1983, in Very Long Baseline Interferometry Techniques, CNES, Cepadues Editions, Toulouse, p. 375
Van Vleck, J.H., Middleton, D.: 1966, Proc. IEEE 54, 2