Australian Bureau of Statistics

Rate the ABS website
Relative Standard Error
 

Sources of Variability

There are two sources of uncertainty or variability associated with survey estimates that are released by Survey TableBuilder. The first source of variability is due to sampling and the second is due to random adjustment of cell values.

Variability due to Sampling

Since the estimates from surveys are based on information obtained from a sub-sample of usual residents of a sample of dwellings, they are subject to sampling variability; that is, they may differ from those that would have been produced if all usual residents of all dwellings had been included in the survey. This component of variability is measured by the group Jackknife method.

Variability due to Random Adjustment

The random adjustment of totals and subtotals introduces another source of variability into the estimates. As these adjustments are generated in a predictable way the impact they have on estimates can be measured directly.

Standard Errors

The variability due to sampling and random adjustment is combined into a single measure called the Standard Error (SE). The SE indicates the extent to which an estimate might have varied by chance, because only a sample of dwellings was included, and by random adjustment.

There are about two chances in three that a sample estimate will differ by less than one SE from the number that would have been obtained if all dwellings had been included and there was no random adjustment. There are about 19 chances in 20 that the difference will be less than two SEs. Another measure of the likely difference is the relative standard error (RSE), which is obtained by expressing the SE as a percentage of the estimate.

Image: Standard Error

RSEs of proportions and percentages

Proportions and percentages formed from the ratio of two estimates are also subject to sampling errors. The size of the error depends of the accuracy of both the numerator and denominator. A formula to approximate the RSE of a proportion is given below:

Image: RSEs of proportions and percentages

Note - this formula only holds when the x is a subset of y. It should not be used if this is not the case i.e. estimates of 'rates' as opposed to proportions.


SEs may also be used to calculate SEs for the difference between two survey estimates (numbers or percentages). The sampling error of the difference between the two estimates depends on their individual SEs and the relationship (correlation) between them. An approximate SE of the difference between two estimates (x-y) may be calculated by the following formula:
SE(x-y) =sqrt[SE(x)]2 +[SE(y)]2

While this formula will only be exact for differences between separate and uncorrelated characteristics of subpopulations, it is expected to provide a reasonable approximation for most differences likely to be of interest in relation to this survey.

Note that it is the RSE of a percentage that is displayed, from which the SE may be calculated. For example if the estimated proportion is 30% with an RSE of 20%, then the SE for the proportion is 6%.

In some cases, the formula for the approximation of the RSE of a proportion may be unsuitable to use because the RSE of the numerator is very close to, or below, the RSE of the denominator. In this case the RSE will be suppressed. It is recommended to use the RSE of the numerator as a proxy for the RSE of the proportion in the event of this occurrence.

Standard Errors of Means and Sums

The estimates of means and sums of continuous variables are subject to sampling variability and random adjustment. As for population estimates, the variability due to sampling and random adjustment is combined into the calculated Standard Error, and the Relative Standard Error is reported. The component of variability arising from sampling is calculated using the Jackknife method.

Standard Errors of Quantiles

The estimates of quantiles such as medians, quartiles, quintiles and deciles are subject to sampling variability and random adjustment. As for population estimates, the variability due to sampling and random adjustment is combined into the calculated Standard Error, and the Relative Standard Error is reported. The component of variability arising from sampling is calculated using the Woodruff method. This is also true for Equal Distribution Quantiles.

Reliability of Estimates

Estimates with RSEs of 25% or more are not considered reliable for most purposes. Estimates with RSEs greater than 25% but less than or equal to 50% are annotated by an asterisk to indicate they are subject to high SEs and should be used with caution. Estimates with RSEs greater than 50% have their RSE suppressed in order to prevent the release of confidential data, and are annotated by a double asterisk. These estimates are considered too unreliable for general use.

Non-Sampling Error

The imprecision due to sampling variability and random adjustment should not be confused with inaccuracies that may occur because of imperfections in reporting by respondents and recording by interviewers, and errors made in coding and processing data. Inaccuracies of this kind are referred to as non-sampling error, and they may occur in any enumeration, whether it be a full count or a sample. Every effort is made to reduce non-sampling error to a minimum by careful design of questionnaires, intensive training and supervision of interviewers, and efficient operating procedures.

Bookmark and Share. Opens in a new window

© Commonwealth of Australia 2013

Unless otherwise noted, content on this website is licensed under a Creative Commons Attribution 2.5 Australia Licence together with any terms, conditions and exclusions as set out in the website Copyright notice. For permission to do anything beyond the scope of this licence and copyright terms contact us.