Sampling error is the expected difference that could occur between the published estimates, derived from repeated random samples of persons, and the value that would have been produced if all persons in scope of the survey had been included. The magnitude of the sampling error associated with an estimate depends on the sample design, sample size and population variability.
Measures of sampling error
A measure of the sampling error for a given estimate is provided by the Standard Error (SE), which is the extent to which an estimate might have varied by chance because only a sample of persons was obtained.
Another measure is the Relative Standard Error (RSE), which is the SE expressed as a percentage of the estimate. This measure provides an indication of the percentage errors likely to have occurred due to sampling.
Standard errors of proportions and percentages
Proportions and percentages formed from the ratio of two estimates are subject to sampling errors. The size of the error depends on the accuracy of both the numerator and the denominator. The RSEs of proportions and percentages for the National Survey of Mental Health and Wellbeing: Summary of Results, 2007 (cat. no. 4326.0) were provided in spreadsheet format as an attachment to the publication. The RSEs were calculated using the full delete-a-group jackknife technique, which is described in the following segment.
Replicate weights and directly calculated standard errors
The SEs on estimates from this survey were obtained through the delete-a-group jackknife variance technique. In this technique, the full sample is repeatedly subsampled by successively dropping households from different groups of clusters of households and then the remaining records are reweighted to the survey benchmark population. Through this technique, the effect of the complex survey design and estimation methodology on the accuracy of the survey estimates is stored in the replicate weights. For the 2007 SMHWB, this process was repeated 60 times to produce 60 replicate weights for each sample unit. The distribution of the 60 replicate estimates based on the full sample estimate is then used to directly calculate the standard error for each full sample estimate.
Replicate weights enable variances of estimates to be calculated relatively simply. They also enable unit records analyses such as chi-square and logistic regression to be conducted, which take into account the sample design. Replicate weights for any variable of interest can be calculated from the 60 replicate groups, giving 60 replicate estimates. The distribution of this set of replicate estimates, in conjunction with the full sample estimate (based on the general weight) is then used to approximate the variance of the full sample.
The person level and household level records on the CURFs contain 60 replicate weights. The Standard Error (SE) for each estimate produced from the CURFs can be calculated using the replicate weights provided. When calculating SEs it is important to select the replicate weights which are most appropriate for the analysis being undertaken. For more information see 'Use of weights'.
The formula for calculating the Standard Error (SE) and Relative Standard Error (RSE) of an estimate using this technique is shown below.
This method can also be used when modelling relationships from unit record data, regardless of the modelling technique used. In modelling, the full sample would be used to estimate the parameter being studied, such as a regression co-efficient, the 60 replicate groups used to provide 60 replicate estimates of the survey parameter. The variance of the estimate of the parameter from the full sample is then approximated, as above, by the variability of the replicate estimates. For more information on the replicate weights technique refer to Appendix 2 of the Users' Guide.
Use of the delete-a-group jackknife technique for complex estimates, such as regression parameters from a statistical model, is not straightforward and may not be appropriate. The technique described does not apply to investigations where survey weights are not used, such as unweighted statistical modelling. More information on the delete-a-group jackknife technique is provided in the Research Paper: Weighting and Standard Error Estimation for ABS Household Surveys (Methodology Advisory Committee), Jul 1999 (cat. no. 1352.0.55.029).
CURF users should be aware that estimates produced from the CURFs may differ from those in the published data due to actions taken to preserve confidentiality.