|Page tools: Print Page Print All
These sources of error are discussed in turn below.
Errors related to survey scope
Some dwellings may have been inadvertently included or excluded because, for example, the distinction between whether they were private or non-private dwellings may have been unclear. All efforts were made to overcome such situations by constant updating of dwelling listings both before and during the survey. Additionally, some people may have been inadvertently included or excluded because of difficulties in applying the scope rules concerning who was identified as usual residents, and concerning the treatment of some visitors. For more information on scope and coverage see the Methodology chapter.
In this survey, response errors may have arisen from three main sources:
Errors may have been caused by misleading or ambiguous questions, inadequate or inconsistent definitions of terminology, or by poor overall survey design (eg context effects, where responses to a question are directly influenced by the preceding question/s). In order to overcome these types of issues, individual questions and the overall questionnaire were tested before the survey was enumerated. Testing included:
More information on pre- and field testing is provided in the Survey development chapter.
As a result of testing, modifications were made to:
In considering modifications it was sometimes necessary to balance better response to a particular item/topic against increased interview time, effects on other parts of the survey and the need to minimise changes to ensure international comparability. Therefore, in some instances it was necessary to adopt a workable/acceptable approach rather than an optimum approach. Although changes would have had the effect of minimising response errors due to questionnaire design and content issues, some will have inevitably occurred in the final survey enumeration.
Response errors may also have occurred due to the length of the survey interview because of interviewer and/or respondent fatigue (ie loss of concentration). While efforts were made to minimise errors arising from deliberate misreporting or non-reporting, some instances will have inevitably occurred.
Accuracy of recall may also have led to response error, particularly in relation to the lifetime questions. Information in this survey is essentially 'as reported', and therefore may differ from information available from other sources or collected using different methodologies. Responses may be affected by imperfect recall or individual interpretation of survey questions, particularly when a person was asked to reflect on experiences in the 12 months prior to interview. The questionnaire was designed to strike a balance between minimising recall errors and ensuring the data was meaningful, representative (from both respondent and data use perspectives) and would yield sufficient observations to support reliable estimates. It is possible that the reference periods did not suit every person for every topic, and that difficulty with recall may have led to inaccurate reporting in some instances.
A further source of response error is lack of uniformity in interviewing standards. To ensure uniform interviewing practices and a high level of response accuracy, extensive interviewer training was provided. An advantage of using Computer Assisted Interviewing (CAI) technology to conduct survey interviews is that it potentially reduces non-sampling error. More information on interviews, interviewer training, the survey questionnaire and CAI is provided in the Survey development and Data collection chapters.
Response errors may have also occurred due to language or reading difficulties. In some instances, a proxy interview was conducted on behalf of another person who was unable to complete the questionnaire themselves due to language problems and where an interpreter was unable to be organised. A proxy interview was only conducted where another person in the household (aged 15 years or over) was considered suitable. The proxy may also have been a family member who did not live in the selected household, but lived nearby. A proxy arrangement was only undertaken with agreement from the selected person, who were first made aware of the topics covered in the questionnaire. Aside from difficulties in understanding English verbally/orally, there may have been difficulties in understanding written English. The 2014–15 NATSISS incorporated the extensive use of prompt cards, as pre-testing indicated that these could aid interpretation by selected persons. People were asked if they would prefer to read the cards themselves or have them read out by the interviewer. It is possible that some of the terms or concepts used on the prompt cards were unfamiliar and may have been misinterpreted, or that a response was selected due to its position on the prompt card.
Some respondents may have provided responses that they felt were expected, rather than those that accurately reflected their own situation. Every effort has been made to minimise such issues through the development and use of culturally appropriate survey methodology. Non-uniformity of interviewers themselves is also a potential source of error, in that the impression made upon respondents by personal characteristics of individual interviewers such as age, sex, appearance and manner, may influence the answers obtained.
Undercoverage is one potential source of non-sampling error and is the shortfall between the population represented by the achieved sample and the in-scope population. It can introduce bias into the survey estimates. However, the extent of any bias depends upon the magnitude of the undercoverage and the extent of the difference between the characteristics of those people in the coverage population and those of the in-scope population.
Potential sources of undercoverage include:
Information on the potential sources of undercoverage is provided in the Explanatory notes of the 2014–15 NATSISS summary publication.
Rates of undercoverage
Undercoverage rates can be estimated by calculating the difference between the sum of the initial weights of the sample and the population count. If a survey has no undercoverage, then the sum of the initial weights of the sample would equal the population count (ignoring small variations due to sampling error).
The overall undercoverage rate for the 2014–15 NATSISS is approximately 62% of the in-scope population at the national level. This rate varies across the states and territories, as shown in the table below. This is a relatively large level of undercoverage when compared to other ABS surveys. For example, the estimated undercoverage rate for the Monthly Population Survey for private dwellings is on average 17%, and the non-response rate is around 7%. There was also an increase in undercoverage compared to the 2008 NATSISS, which had an estimated undercoverage rate of 53%.
Table 5.2.1. 2014–15 NATSISS undercoverage, by state/territory
Errors in processing
Errors may also occur during data processing, between the initial collection of the data and final compilation of statistics. These may be due to a failure of computer editing programs to detect errors in the data, or during the manipulation of raw data to produce the final survey data files; for example, in the course of deriving new data items from raw survey data or during the estimation procedures or weighting of the data file.
To minimise the likelihood of these errors, a number of quality assurance processes were employed, including:
These documents will be presented in a new window.