|Page tools: Print Page Print All|
These sources of error are discussed in turn below.
Errors related to survey scope
Some dwellings may have been inadvertently included or excluded because, for example, the distinctions between whether they were private or non-private dwellings may have been unclear. All efforts were made to overcome such situations by constant updating of lists both before and during the survey. Furthermore, some persons may have been inadvertently included or excluded because of difficulties in applying the scope rules concerning who was identified as usual residents, and concerning the treatment of some overseas visitors. Other errors which can arise from the application of the scope and coverage rules are outlined in Chapter 3: Survey Methodology.
In this survey response errors may have arisen from three main sources: deficiencies in questionnaire design and methodology; deficiencies in interviewing technique; and inaccurate reporting by the respondent.
Errors may be caused by misleading or ambiguous questions, inadequate or inconsistent definitions of terminology used, or by poor overall survey design (e.g. context effects where responses to a question are directly influenced by the preceding questions). In order to overcome problems of this kind, individual questions and the overall questionnaire were thoroughly tested before being finalised for use in the survey.
Testing took two forms:
As a result of both forms of testing, modifications were made to question design, wording, ordering and associated prompt cards, and some changes were made to survey procedures. In considering modifications it was sometimes necessary to balance better response to a particular item/topic against increased interview time or effects on other parts of the survey. The result is that in some instances it was necessary to adopt a workable/acceptable approach rather than an optimum approach. Although changes would have had the effect of minimising response errors due to questionnaire design and content issues, some will inevitably have occurred in the final survey enumeration.
Response errors may also have occurred due to the long nature of the interview, resulting in interviewer and/or respondent fatigue (i.e. loss of concentration). While efforts were made to minimise errors arising from deliberate misreporting or non-reporting by respondents (including emphasising the importance of the data and checking consistency within the survey instrument), some instances will have inevitably occurred.
Recall error may also have led to response error. Information recorded in this survey is essentially 'as reported' by respondents, and hence may differ from information available from other sources or collected using different methodologies. Responses may be affected by imperfect recall or individual interpretation of survey questions. Reference periods used in relation to each topic were selected to suit the nature of the information being sought; in particular to strike the right balance between minimising recall errors while ensuring the period was meaningful, representative (from both respondent and data use perspectives) and would yield sufficient observations in the survey to support reliable estimates. It is possible that the reference periods did not suit every person for every topic, and that difficulty with recall may have led to inaccurate reporting in some instances.
A further source of response error is lack of uniformity in interviewing standards. Methods employed to achieve and maintain uniform interviewing practises included training and re-training programs, and regular supervision and checking of interviewers' work. These programs aimed to ensure that a high level of response accuracy was achieved. An advantage of the CAI technology used in conducting interviews for this survey is that it potentially reduced non-sampling error by enabling edits to be applied as the data was being collected. The interviewer was alerted immediately if information entered into the computer was either outside the permitted range for a question, or contradictory to information previously recorded during the interview. These edits allowed the interviewer to query respondents and resolve issues during the interview. CAI sequencing of questions was also automated such that respondents were asked only relevant questions and only in the appropriate sequence, eliminating interviewer sequencing errors.
Some respondents may have provided responses that they felt were expected, rather than those that accurately reflected their own situation. Every effort has been made to minimise such bias through the development and use of culturally appropriate survey methodology. Respondent perception of the personal characteristics of the interviewer can also be a source of error as the age, sex, appearance and manner of the interviewer, may influence the answers obtained.
One of the main sources of non-sampling error is non-response by persons selected in the survey. Non-response can affect the reliability of results and introduce bias. The magnitude of any bias depends upon the level of non-response and the extent of the difference between the characteristics of those people who responded to the survey and those who did not, as well as the extent to which non-response adjustments can be made during estimation through the use of benchmarks.
To reduce the level and impact of non-response, the following methods were adopted in this survey:
Of the dwellings selected in the 2010 GSS, 12.4% did not respond fully or adequately. As the non-response to the GSS was low, the impact of non-response bias is considered to be negligible.
Errors in processing
Errors may also occur during data processing, between the initial collection of the data and final compilation of statistics. These may be due to a failure of computer editing programs to detect errors in the data, or during the manipulation of raw data to produce the final survey data files; for example, in the course of deriving new data items from raw survey data or during the estimation procedures or weighting of the data file.
To minimise the likelihood of these errors occurring a number of quality assurance processes were employed, including: