QUALITY INDICATORS FOR HOUSEHOLD SURVEYS
The ABS is currently developing a standard set of quality indicators relevant to most household surveys run by the ABS. These quality indicators cover all components of the survey cycle, and together they provide a consolidated framework of indicators from which users can assess the quality of survey data and ABS can monitor quality issues. Data quality encompasses many aspects, especially accuracy, timeliness, relevance and cost of production.
The quality indicators will be implemented in three phases, based on the importance of the indicator and the extent of work required to implement it. Phase 1 indicators involve those indicators that are relatively easy to implement in terms of data availability and cost. Phases 2 and 3 indicators require greater effort in terms of developing methodology, setting up procedures and resolving cost implications.
Many facets of the survey cycle are covered in the core set of quality indicators, from aspects such as frame quality and sample design, through to the questionnaire and timeliness. However, accurately measuring the quality of some aspects, especially the questionnaire, is difficult to do quantitatively and hence these quality indicators should be used with caution. Although many of the quality indicators will only be used internally within the ABS, some of them may be useful to external clients. Both internal and external users of ABS household survey data should be able to use, and should actually be using, information about the quality of our data. This information should be used to help make appropriate judgements on the quality of the data when making informed decisions.
Quality indicators for household surveys are useful for:
- Assisting survey managers and data users to assess the quality of survey data and determine whether that quality is sufficiently good for the intended use of survey outputs.
- Providing an objective basis for continuous improvement of the survey process.
Monitoring of appropriate quality indicators provides a mechanism for ensuring that survey outputs are of an appropriate quality for their intended use. In addition to routine monitoring of quality indicators, survey managers should conduct special investigations aimed at identifying and reducing/removing the factors which would (or would likely) cause a reduction of quality. For continuous improvement of the survey operation, it is important that quality indicators on components of a survey operation be evaluated on a regular basis.
It is envisaged that in future a standard set of quality indicators will be produced for each ABS household survey. By standardising the measures to be produced, their definitions and the infrastructure for their production and dissemination, both internal and external users will be able to compare the quality of data between different surveys, as well as monitor, for repeated surveys, the change in quality over time. In addition, survey areas will be able to adopt best practices in quality management and help raise ABS's capability in the measurement and dissemination of quality information.
In addition to the core set of standard quality indicators that encompass all surveys, there are other important quality indicators which are specific to the content or methodology of each survey which will be implemented on a survey basis.
This project is one of several ABS projects that are currently investigating the issue of quality.
For further information, please contact Matthew Cronin on (02) 6252 5594.
- Enabling managers to optimise the resources spent on different parts of the survey process.