VOACAP Quick Guide: A comparison of broadcast quality to VOACAP using Methods 21, 22, and 30 by Allen B. Richardson (International Broadcasting Bureau/USIA)

<
A comparison of broadcast quality to VOACAP using Methods 21, 22, and 30 by Allen B. Richardson (International Broadcasting Bureau/USIA)
VOACAP Quick Guide: Home
[This document is part of the help files integrated into the ITS HFBC software package.]
Purpose: The purpose of the comparison is to determine which prediction method in VOACAP agrees the best with actual measurements in the distance range greater than 7,000 km.
Measurements: The International Broadcasting Bureau employs a number of professional shortwave listeners to monitor VOA broadcasts. The circuit paths, selected for this comparison study range from 7,054 to 9,469 km. Of these circuits, 73% are 7,000 to 8,500 km. The data consist of subjective scoring on a 5 point scale system (where 5 is excellent and 1 is nil) similar to that recommended by the CCIR [1]. VOA monitors score signal, degradation, and overall reception quality. Frequencies are monitored once per half hour in short duration auditions (typically 30 sec). The half hour observations are averaged to derive the hourly observation for that day. The hourly observations for the month are then ordered and the lower decile extracted. A database of 51 circuit hours comprising 608 observations is selected for April and May 1995 for use in this comparison. The International smoothed sunspot numbers for these months were 22 and 20, respectively.
Prediction Methodology: VOACAP input parameters conform to the VOA Engineering Standard [2] except the actual transmitter carrier power and transmit antenna are used. A transmission line loss of 1.5 dB is assumed. The receive antenna is the "short whip" [2] which approximates the performance of the antenna attached to the monitors' shortwave receivers. The sporadic-E model in VOACAP is not used. Background noise at the receive location is assumed to be a combination of atmospheric and residential man-made noise (i.e. -145 dBW in 1 Hz BW at 3 MHz). The desired "Reliability" is set at .90 so that the output term SNRxx will be for the hourly median signal-to-noise density ratio which is exceeded on 90% of the days of the month. For this comparison VOACAP version 95.0629 is used for output Methods 21(long path), 22 (short path) and 30 (smoothed). The output variable SNRxx is converted to its equivalent signal quality value for comparison with actual monitor scores.
Conversion of SNR to Monitor Scores: The standard 5-point quality and impairment scale for aural assessment of sound broadcasting has been shown to equate to an equivalent carrier-to-noise density ratio for DSB-AM broadcasts [3]. This relationship was obtained using linear regression analysis with a correlation coefficient (r²) of 95% and is given as follows:
S  = 0.077 CNR  - 2.04               Eq. (1)

       where:

S	=  Signal quality score (1 through 5)
CNR	=  Carrier-to-Noise density ratio in dB/Hz
Using equation (1), the lower decile of the monthly monitor scores (S.9) for a given hour and circuit is computed from the predicted CNR.9 from VOACAP. For DSB applications, the SNRxx from VOACAP is the same as the hourly median carrier-to-noise ratio which is exceeded on 90% of the days over the month. Values computed to be greater than a "5" are truncated to "5"; likewise values less than "1" are rounded to "1." Comparison: The premise of the comparison is that the difference between the predicted quality and the observed quality should be zero. The mean difference and the standard deviation of the differences are computed. This gives the bias and variance of the predicted quality to that observed. The lower decile for the monitored broadcast hour is used. The number of observations for the month for these data range from 6 to 18 with 12 as the typical value. All days were used; disturbed days were not excluded. Samples containing severe interference (i.e. degradation of 1 or 2) are excluded.
The results of the difference comparison obtained from the 51 monthly circuit hours are shown in Table I. The observed S and O are 0.1 to 0.2 units better on average than predicted using Method 21 (forward scatter model). For Method 30 (smoothing algorithm) the observed S and O are 0.3 units better. For Method 22 (ray hop) the observed S and O are 0.4 units better than predicted. Theconfidence levels at 90% for these differences range from ±1.2 to 1.3 units on the 5 point scale.
Table I.  Difference between Observed and
Predicted Quality at 90% Confidence Interval

Parameter,		Signal	   	Overall
Method	    		Quality	   	Quality
Lower Decile, M21	0.1 +/- 1.3	0.2 +/- 1.2
Lower Decile, M30	0.3 +/- 1.3	0.3 +/- 1.2
Lower Decile, M22	0.4 +/- 1.3	0.4 +/- 1.2
Discussion: One unit on the signal quality score equates to a difference in SNRxx of 13.0 dB. In terms of predicted signal-to-noise ratio the prediction models in VOACAP agree with observation within 1 to 5 dB. The variance is rather large due to the coarseness of the monitor scores (e.g. monitors score in whole units of signal quality; 1, 2..., or 5). The measurement data for this comparison indicate fairly good levels of reception for the long paths. This probably explains why the less conservative forward scatter model in Method 21 gave the best agreement with these data.
In all cases, the predicted lower decile of the signal quality was lower than that observed. The ray hop calculations in Method 22 predicted larger variations around the monthly median signal value than were observed. The long path predictions of Method 21 indicated slightly less path loss at these distances. The results from the smoothing function of Method 30 gave results that fell closer to Method 22 because the majority of the path lengths in this comparison were less than 8,500 km.
The measurement period in this study is for very low sunspot activity. Generally, ionospheric propagation is more stable from one day to the next during this portion of the solar cycle. It is not clear that the same comparison results would be obtained when the sunspot number is higher and geomagnetic conditions are less stable.
Conclusions and Recommendations:

VOACAP predictions and coverage maps run with REL set at 0.90 provide a good estimate of signal quality 90% of the time. Employing the smoothing algorithm between the "long path" and "short path" signal power calculations for circuit distances greater than 7,000 km is preferable to using Method 20 (*) or 22 (ray hop) alone.
At 90% circuit reliability using the Method 21 long path model for circuits ranging from 7,000 to 10,000 km, the difference between the actual and predicted signal quality amounts to 1 to 2 dB of carrier-to-noise ratio or 0.1 units on the signal quality scale. The difference using the smoothing algorithm, Method 30, is about 4 dB or 0.3 units; the difference using the ray hop model Method 22 is 5 dB or 0.4 units on the signal quality scale. The actual signal quality is slightly better than predicted any of these methods.
A similar test should be conducted during a period of higher sunspot activity.

__________
(*) Method 20 automatically switches from the ray hop method to the forward scatter model when the circuit length equals 10,000 km.
References:

CCIR, 1982. "Subjective Assessment of Sound Quality," Rec. 562-1, X-1, 215, ITU, Geneva.
Lane, G. and Toia M., 1985. "High Frequency (Shortwave) Broadcast System Design; Requirements Definition," Voice of America Engineering 16775.01, Washington DC USA.
Lane, G., Richardson, A. B. and DeBlasio L. M., 1994. "Signal-To-Noise Ratio and Aural Assessment of Broadcast Reception Quality," Sixth International Conference on HF Radio Systems and Techniques, Conference Publication Number 392, 129, Institution of Electrical Engineers, London.

* * *