Australian Bureau of Statistics
Celebrating the International Year of Statistics 2013

Rate the ABS website
ABS Home > Statistics > By Catalogue Number
4363.0.55.001 - Australian Health Survey: Users' Guide, 2011-13  
Latest ISSUE Released at 11:30 AM (CANBERRA TIME) 07/12/2012   
   Page tools: Print Print Page Print all pages in this productPrint All RSS Feed RSS Bookmark and Share Search this Product  
Contents >> Data quality and interpretation of results >> Interpretation of results

INTERPRETATION OF RESULTS

As noted above, a range of factors have impacted on the quality of the data collected. The ABS has sought to minimise the effects of these factors through various means in the development and conduct of this survey, however, only sampling error can be quantified to allow users of the data to adjust for possible errors when using/interpreting the data. Information is not available from the survey to enable the effects of other issues affecting the data to be quantified. The relative importance of these factors will differ between topics, between items within topics, and by characteristics of respondents.

Comments have been included in individual topic descriptions in this publication to alert users of the data to the more significant issues likely to affect results for that topic, or items within it. These notes reflect ABS experience of past health and other surveys, feedback from users of data from those surveys, and ABS and other research on survey methods and response patterns, as well as information from survey testing and validation. However, they are indicative only, and do not necessarily reflect all factors impacting results, nor the relative importance of those factors.

Against this background, the following general comments are provided about interpreting data from the surveys.

  • The surveys aims to provide statistics which represent the population or component groups of the population. It does not aim to provide data for analysis at the individual level. While errors of the types noted above may occur in individual respondent records, they will have little impact on survey estimates unless they are repeated commonly throughout the respondent population.
  • The survey data are mainly self-reported and may differ from data sources that have different collection methodologies (e.g. administrative data), however, the AHS is able to provide dimensions of the data (e.g. population group, related health characteristics, use or non use of other health services) and cross-classifications (e.g. self-assessed health by alcohol risk level) which are not available from administrative sources.
  • Some survey topics, such as alcohol consumption, have some known data quality issues. While this means the data should be interpreted with care, the information is still considered valuable for certain uses. For example, while the overall levels of alcohol consumption described by the survey should be interpreted with caution, the data is still considered useful in describing consumption patterns across days of the week, types of drink consumed, relative levels of consumption across population groups, and alcohol consumption in relation to other risk behaviours or characteristics. It is also useful for monitoring changes in the levels and patterns of consumption over time. Notes regarding any known data quality issues are contained in the individual topic descriptions in this User's Guide.
  • Although various reference periods are used throughout the survey for different topics (e.g. current, usual, last week, last 2 weeks, last 4 weeks) the survey essentially provides a 'point in time' picture of the health of the population and of population sub-groups. That is, the survey provides information about the prevalence of characteristics, not the incidence of those characteristics or of changes in characteristics (except in terms of differences between surveys). As the surveys were conducted over two separate 12 month periods, the results are essentially an average over that period, that is, representative of a typical week, fortnight, etc in that period.

Comparability with 2007-08

Understanding the comparability of data from the 2011-12 NHS with data from previous NHSs is important to the use of those data and interpretation of apparent changes in health characteristics over time. While the 2011-12 NHS is deliberately the same or similar in many ways to the 2007-08, 2004-05 and 2001 NHSs (and in part to the 1995 NHS), there are important differences in sample design and coverage, survey methodology and content, definitions and classifications between the surveys. These differences will affect the degree to which data are directly comparable between the surveys, and hence the interpretation of apparent changes in health characteristics over the 2007-08 to 2011-12 period.

Throughout the topic descriptions and in other parts of this publication, comments have been made about the changes between surveys and their expected impact on the comparability of data. These are general comments based on results of testing, ABS experience in survey development, and preliminary examination of data from the 2011-12 survey. They should not, therefore, be regarded as definitive statements on comparability, and they may omit the types of findings which might result from a detailed analysis of the effects of all changes made.

The following table summarises key differences in the general survey characteristics of the 2007-08 to 2011-12 surveys:

Survey characteristic

2011-12 NHS
2007 - 08 NHS

Collection method
Personal interview with adult respondents; proxy interview for children less than 15 years
Same
Personal interview with children aged 15-17 years with parental consent; otherwise interview by proxy
Same
Questionnaires
Single CAI instrument, incorporating household, adult and child components
Same
Sample coverage
Private dwellings only
Same
Urban and rural areas
Same
All States and Territories; additional sample in NT
All States and Territories
Sample design/size
One child aged 0-17 years, one adult per dwelling
Same
Fully responding H'holds = 15566
Fully responding H'holds = 15792
Final sample = 4951 children
Final sample = 5009 children
15476 adults, 20427 persons
15779 adults, 20788 persons
Enumeration period
March 2011 to March 2012
August 2007 to June 2008
Collection methodology
CAI questionnaire
CAI questionnaire
Automated coding with inbuilt trigram coders, supported by manual and CAC systems for some modules
Automated coding, supported by manual and CAC systems
Main output units
Person
Person



Sample design/size

The overall sample of households was about 1.4% lower in 2011-12 than in 2007-08, with the same proportion of people in each household enumerated. This resulted in the total sample of persons in the 2011-12 survey being only slightly below that of 2007-08. The impact of the lower sample size on the RSEs of the NHS 2011-12 will be minimal.

However, information on the following topics was collected in both NHS and NNPAS:

  • cardiovascular disease;
  • diabetes/high sugar levels;
  • kidney disease;
  • Physical measurements (height, weight and waist);
  • Body Mass Index;
  • smoking;
  • fruit, vegetables and salt consumption;
  • blood pressure; and,
  • female life stages.

Results from the combined NHS and NNPAS sample (core) will be calculated from a larger sample of approximately 34,000 respondents. These estimates will be available in mid 2013 and it is expected that the larger sample will enable finer detailed estimates to be produced with higher accuracy.

Differences in the reliability of estimates between surveys should be considered in interpreting apparent changes between the surveys. It is recommended that apparent changes are significance tested (see 'Testing for statistically significant differences', in the Data Quality section of this chapter).

Through the weighting process, survey estimates at the state by part of state by sex by broad age group will be the same or very similar to the benchmark populations. However, because the characteristics of the sample are not identical to those of the benchmark population, some records will receive higher or lower weights than others. As this will vary between surveys, it is a factor to consider in comparing 2011-12 with 2007-08 data, but the impact on comparability is expected to be small. Sample and population figures for the two surveys appear in the following table:


2011-12 NHS(a) 2007-08 NHS(b)


% of adults in sample
% of adults in population
% of adults in sample
% of adults in population

Males
47.7
49.8
47.1
49.3
Females
52.3
50.2
52.9
50.7
18-24
7.9
12.8
8.6
12.7
25-34
17.5
18.8
16.9
18.3
35-44
19.2
18.4
20.4
19.3
45-54
18.5
17.7
18.0
18.3
55-64
16.5
15.1
15.9
14.9
65-74
11.3
9.9
10.8
9.2
75 and over
9.0
7.3
9.3
7.3
Persons
100.0
100.0
100.0
100.0

(a) Benchmark population as at 31 October 2011.
(b) Benchmark population as at 31 December 2007.


Partial enumeration of households

Prior to 2001, all persons in sampled dwellings were included in NHSs, and only records from fully responding households were retained on the data file. This meant that results could be compiled at household, family and income unit level, in addition to person level. The 2001, 2004-05, 2007-08 and 2011-12 surveys, however, sub-sampled persons in households (one adult and one child 0-17 years NHS; one adult and one child 2-17 years NNPAS), therefore complete enumeration occurs only in a minority of households.

As basic demographic characteristics, and some other items (e.g. number of daily smokers in household) were collected from the selected adult about all household members, some household, family and income unit characteristics of the respondent are available as person level items. This information enables a greater degree of analysis in relation to the person and the household; for example; number of children aged 0-14 in households containing smokers. Person level information is only available for persons enumerated in the survey (i.e. one adult and one child), not all people in the household.

Comparability of long term conditions data with 2007-08

There are a number of issues affecting comparability of data for long-term medical conditions between the 2007-08 and 2011-12 surveys.

  • Some modules contain additional questions and or have had questions removed. This is detailed below and also at the end of each module under the comparability with 2007-08 sub-heading;
  • The 2011-12 survey made use of trigram coders where a question contained a list of possible answers plus an 'other' category. In the 2007-08 survey, these other categories were office coded.
  • In 2011-12, a specific Kidney disease module was included in the survey for the first time. Therefore, kidney disease data is not directly comparable to previous surveys given the change in collection methodology.

These issues are discussed in general terms below. Further discussion of issues related to particular conditions is contained in the relevant condition sections of this Users' Guide.

Methodological issues

The question methodology for long-term conditions in 2011-12 was similar to that of 2007-08: a combination of direct questions, general questions supported by prompt cards (either showing examples of conditions, or a list of conditions from which respondents are asked to select), and open ended questions. Differences between the two surveys are outlined below.
  • Respondents use of medication was not collected in modules relating to specific medical or mental health conditions in the 2011-12 survey (with some exceptions, including mental and behavioural conditions, asthma, and some diabetes medication). Respondents were asked to provide details of all medications that they were taking regardless of what condition they were taking the medicine for. In the 2007-08 NHS survey respondents were asked about what medications they were taken for each specific medical or mental health condition.
  • The following health workers were added to the list of choices on the prompt card in the "Healthy Lifestyles" module which were not included in the 2007-08 survey:
      • Aboriginal Health Workers;
      • alcohol and drug workers
      • audiologist/audiometrist
      • herbalist
      • hypnotherapist; and,
      • speech therapist/pathologist
  • Two additional categories that a person could ask their GP about were added to the prompt card which were not included in the 2007-08 survey:
      • family planning; and,
      • safe sexual practices.
  • An additional way in which persons could be categorised as having a: "mild core activity limitation" was added to the 2011-12 survey, which was not in the 2007-08 survey. The question was included in the disability module and asked people if they could do all of the following:
      • easily walk 200 meters;
      • walk up and down stairs without a handrail
      • easily bend to pick up an object from the floor
      • use public transport without difficulty, help or supervision.
  • The age group of respondents who were asked questions relating to levels of pain was changed from persons aged 15 years and older in 2007-08 survey to persons aged 18 years and over in the 2011-12 survey.
  • Additional questions were added to the alcohol module which were not included in the 2007-08 survey so that respondents use of alcohol could be assessed against the 2009 NHRMC guidelines (see alcohol for details).
  • Additional questions were added to the exercise module which were not in the 2007-08 survey on the amount of walking respondent did to get from place to place.

A further factor which may affect comparability is that the reported prevalence of illness is complex and dynamic, and is a function of respondent knowledge and attitudes, which in turn may be affected by the availability of health services and health information, public education and awareness, accessibility to self-help, etc. For example, a public education program has been running in Australia over a number of years aimed to raise public awareness and public acceptance of mental health disorders. Consequently, respondents may be more willing to talk about or report feelings of anxiety or depression than in previous years.

Recording of condition details

Provision made to record conditions information (a combination of selected categories and text responses) was similar in the 2011-12 and 2007-08 surveys. Interviewers were encouraged in both surveys to add supplementary information if they felt this would help clarify responses or assist with coding. The 2011-12 survey utilised trigram coders to assist interviews to code responses which were not on prompt cards or pick lists in addition to recording the respondents answers in a text field. Text fields were cross-referenced with trigram coding to ensure the response answer was correctly coded during processing.

Survey content

The following tables summarise the main differences in content between the 2011-12 and 2007-08 surveys:

SURVEY CONTENT POPULATION CHARACTERISTICS

Topics covered
2007-08 NHS
2011-12 NHS
Main items available from 2011-12
Comments on main differences between 2007-08 and 2011-12

General demographics
X  
X  
Sex; age; marital status (registered & social); Indigenous status; country of birth; year of arrival in Australia; main language spoken at home; proficiency in English; family type; household size, composition, type; Income unit type; Location.  
Same content in 2011-12 as in 2007-08.  
Education
X  
X  
Whether attending school; highest year of school completed; whether has non-school qualification; level of highest educational attainment; level of highest non-school qualification; whether currently studying full or part time; field of study of qualification obtained.  
Same content in 2011-12 as in 2007-08.  
Labour force
X  
X  
Labour force status; status in employment; occupation, industry and industry sector of main job; hours worked; duration of unemployment; shift work.  
Same content in 2011-12 as in 2007-08. New item: whether ever served in the Australian Defence Force 
Income
X  
X  
Personal income - Level; equivalised sources and main source; type of pension/benefit received; Household income level, equalised  
Same content in 2011-12 as in 2007-08.  
Housing
X  
X  
Dwelling type; number of bedrooms; household tenure; landlord type.  
Same content in 2011-12 as in 2007-08. New item: whether has landline phone. 
Private health insurance/health cards
X  
X  
Whether has PHI; type of cover; time covered by PHI; reasons having/not having PHI; duration with PHI; Whether has DVA or other Govt concession card: type of card.  
Same content in 2011-12 as in 2007-08.  


SURVEY CONTENT LONG-TERM HEALTH CONDITIONS

Topics covered
2007-08 NHS
2011-12 NHS
Main items available from 2011-12
Main differences between 2007-08 and 2011-12

Arthritis and Osteoporosis
X  
X  
Type of arthritis; age first told arthritis; types of other specific actions taken for arthritis and osteoporosis; visits to health professionals; bone density checked and types of actions taken.
Same content in 2011-12 to 2007-08. 2011 - 12 survey did not ask medication question specific to Arthritis and Osteoporosis, medication use was obtained from a separate module.
Asthma
X  
X  
Whether asthma worse or out of control; whether attended emergency; whether has written asthma action plan; visits to health professionals; and whether discussed self management. and types of actions taken. 
Similar content in 2011-12 to 2007-08. 2011 - 12 survey did not ask medication question specific to asthma in the asthma module, medication use was obtained from a separate module.
Cancer
X  
X  
Cancer status; type of cancer; visits to health professionals.  
Similar content in 2011-12 to 2007-08.  
Cause of reported long-term conditions
X  
X  
Whether condition result of an injury; where injury occurred; age injury occurred and allergies to food or drugs. 
Similar content in 2011-12 to 2007-08. New items: whether had food or drug allergies.  
Cardiovascular conditions
X  
X  
Types of condition; blood pressure taken and by whom; cholesterol/blood pressure checks in last 12 months and 5 years and whether aspirin taken
Similar content in 2011-12 to 2007-08.  2011 - 12 survey did not ask medication question specific to cardiovascular conditions, medication use was obtained from a separate module.
Diabetes/ high sugar levels
X  
X  
Types of diabetes; types of meds used in last 2 weeks; type of other actions taken to manage condition; whether screened for diabetes in the last 3 years; age first told; whether condition interferes with usual activity; whether has diabetes-related sight problems; period since last visited optometrist/eye specialist; whether discussed self-management.  
Similar content in 2007-08 to 2004-05. New items: Whether parents have diabetes/high sugar levels.  
Disability
 
X  
Disability status; type of disability; main type of disability.  
Similar content in 2011-12 to 2007-08. New items: assessment on the extent of disability.
Kidney disease
X  
X  
Kidney disease status; use of dialysis
Specific kidney disease module included in 2011-12; kidney disease status obtained from general long-term condition module in 2007-08.
Mental health condition
X  
X  
Type of conditions and stressors and the frequency of these conditions occurring; number of times consulted health professional.
Similar content in 2011-12 to 2007-08; 2011-12 survey did not ask medication question specific to mental health, medication use was obtained from a separate module.
Mental well-being
X  
X  
Psychological distress (K10).  
Same content in 2011-12 as 2007-08; 2011-12 survey did not ask medication question specific to mental health, medication use was obtained from a separate module. 
Bodily pain
 
X  
Bodily pain in last 4 weeks; whether interfered with work.  
Same content in 2011-12 as 2007-08.
Self assessed health
X  
X  
SF1 - Rate health  
Same content in 2011-12 as 2007-08.
Status of condition
X  
X  
Status for each condition reported  
Same content in 2011-12 as 2007-08.


SURVEY CONTENT HEALTH RISK BEHAVIOURS

Topics covered
2007-08 NHS
2011-12 NHS
Main items available from 2011-12
Comments on main differences between 2004-05 and 2007-08

Alcohol consumption
X  
X  
Period since last drank; days consumed alcohol in last week; quantity of alcohol by type of drink consumed in last week (max 3 days); alcohol risk level; graduated frequency; how consumption changed since this time last year.  
Same content in 2011-12 as 2007-08. New item: New questions on number of times respondent had 11, 7, 5 and 3 standard drinks in a day in the last 12 months.  
Body mass
X  
X  
Body mass index and measured height, weight and waist measurements.  
Similar content in 2011-12 as 2007-08, except there were no self-reported measurements and no hip measurements. 
Dietary habits
X  
X  
Type of milk usually consumed and fat content of milk; usual daily intake of vegetables & fruit and use of salt and whether it is iodised.  
Similar content to 2011-12 as 2007-08, with new questions for salt use.  
Exercise
X  
X  
Type, frequency and duration of exercise in last week; exercise level; whether walked for transport; number of times walked and total duration; time spent sitting at work and home.  
Similar content in 2011-12 as 2007-08, except respondents are only asked about exercise in the last week rather than also in the last 2 weeks.  
Smoking
X  
X  
Smoker status; number of smokers in household, age started/stopped smoking regularly.  
Same content in 2011-12 as 2007-08.
Breastfeeding
X  
Whether was breatfeed; age commenced and ended and whether feed other fluids including milk, formula, soy, juice, softdrink or water.
Not collected in 2007-08  





Previous PageNext Page


Bookmark and Share. Opens in a new window


© Commonwealth of Australia 2013

Unless otherwise noted, content on this website is licensed under a Creative Commons Attribution 2.5 Australia Licence together with any terms, conditions and exclusions as set out in the website Copyright notice. For permission to do anything beyond the scope of this licence and copyright terms contact us.