4450.0 - Supplementary Disability Survey, 2016 Quality Declaration 
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 30/06/2016  First Issue
   Page tools: Print Print Page Print all pages in this productPrint All

EXPLANATORY NOTES

Introduction

1 In 2001, at the United Nations International Seminar on the Measurement of Disability, there was broad agreement on the need for population measures for international comparison. To address this, the United Nations Washington Group (WG) on Disability Statistics developed a series of six questions, known as the WG Short Set on Functioning, which were designed to identify people with functional limitations at greater risk of experiencing reduced or restricted participation in society.

2 The WG Short Set is not expected to identify every person with a disability. Its focus is on six domains covering the most basic activities: seeing, hearing, walking, cognition, self-care and communication.

3 The WG Short Set was the basis for the Supplementary Disability Survey (SDS) which was conducted by the ABS for the first time in 2016 using a sample of respondents from the 2015 Survey of Disability, Ageing and Carers (SDAC). Results from the SDS will be used for international comparison and are expected to inform future research. The results are not comparable with disability prevalence figures produced in other ABS surveys, such as the SDAC.

Scope and coverage of the survey

4 The SDS used a sub-sample of households from the 2015 SDAC and was therefore subject to the same scope, inclusions and exclusions. For more information, see the Disability, Ageing and Carers Australia: First Results, 2015 (cat. no. 4430.0.10.001) Explanatory Notes.

5 Further scope exclusions for the SDS were:

  • children under five years of age
  • people living in cared- accommodation establishments, such as hospitals and nursing homes
  • people who had left the household since the time of the 2015 SDAC interview.

Sample design

6 The sample for the SDS was taken from the household component of the SDAC, which included private dwellings and self-care retirement villages. The person responsible for providing household level information (Any Responsible Adult or ‘ARA’) that was enumerated in the 2015 SDAC between September and November 2015 was asked whether they would consent to participating. The SDS interviews were conducted from 31 January to 1 March 2016. The ARA answered the WG Short Set for all members of their household.

7 The initial sample for the SDS included approximately 7,600 households. After exclusions due to sample loss and non-response, the final SDS sample was 6,213 households comprised of 13,837 people.

Data collection and processing procedures

8 Information was collected by trained ABS Interviewers who conducted computer-assisted telephone interviews with respondents who agreed to participate in the SDS.

9 Basic demographic information from the 2015 SDAC was used to confirm household members with the respondent. Anyone who had left the household since the time of the 2015 SDAC interview was removed from the SDS.

10 Respondents were then asked about the level of difficulty each member of their household aged five years and over may have doing certain activities because of a health problem.

11 After collection, a small number of households were excluded due to incomplete or incorrect data.

Weighting, benchmarking and estimation

12 Weighting is a process of adjusting results from a sample survey to infer results for the in-scope total population. To do this, a weight is allocated to each sample unit; for example, a household or a person. The weight is a value which indicates how many population units are represented by the sample unit.

13 The first step in calculating weights for each person in the SDS was to assign an initial weight, which was based on a person's final weight from the 2015 SDAC. An adjustment was made to these weights to account for the sample size of the SDS in comparison to the 2015 SDAC sample.

14 The initial person weights were then calibrated to align with independent estimates of the population of interest, referred to as 'benchmarks'. Calibrating weights to population benchmarks helps to compensate for over or under-enumeration of particular categories of persons and ensure that the survey estimates conform to the distribution of the population rather than to the distribution within the sample itself.

15 Two sets of benchmarks were used in the SDS weighting process: estimated resident population (ERP) benchmarks and person level benchmarks from the 2015 SDAC household survey.

16 The estimated resident population benchmarks were the same as those used in the 2015 SDAC weighting process, but excluded children under five years of age. Benchmark cells were defined by cross classifying sex by age and area of usual residence. The benchmarks used the estimated resident population living in private dwellings in each state and territory, excluding people living in very remote areas of Australia and in discrete Aboriginal or Torres Strait Islander communities as at 30th September 2015. These estimates do not (and are not intended to) match estimates for the total Australian population obtained from other sources.

17 Person level benchmarks from the 2015 SDAC survey were used to reduce the impact of response bias in the SDS sample. The benchmark cells were defined by summing the weights of 2015 SDAC disability status, state and age groups. Calibrating to these benchmarks adjusts for differences in non-response observed in the SDS sample, compared to the full 2015 SDAC sample.

18 As a result of the weighting process, totals for some population groups will not exactly match totals published in the 2015 SDAC first release Disability, Ageing and Carers Australia: First Results, 2015 (cat. no. 4430.0.10.001).

Reliability of estimates

19 All sample surveys are subject to sampling and non-sampling error.

20 Sampling error is the difference between estimates derived from a sample of persons and the value that would have been produced if all persons in scope of the survey had been included. Indications of the level of sampling error are given by the Relative Standard Error (RSE) and 95% Margin of Error (MOE). For more information, see the Technical Note - Data quality.

21 In this publication, estimates with an RSE of 25% to 50% are flagged to indicate that the estimate has a high level of sampling error relative to the size of the estimate, and should be used with caution. Estimates with an RSE over 50% are also flagged and are generally considered too unreliable for most purposes.

22 Margins of Error are provided for proportions to assist users in assessing the reliability of these data. The proportion combined with the MOE defines a range which is expected to include the true population value with a given level of confidence. This is known as the confidence interval. This range should be considered by users to inform decisions based on the proportion.

23 Non-sampling error may occur in any data collection, whether it is based on a sample or a full count such as a census. Non-sampling errors occur when survey processes work less effectively than intended. Sources of non-sampling error include non-response, errors in reporting by respondents or in recording of answers by Interviewers, and errors in coding and processing data.

24 Non-response occurs when people are unable to or do not respond, or cannot be contacted. Non-response can affect the reliability of results and can introduce a bias. The magnitude of any bias depends on the rate of non-response and the extent of the difference between the characteristics of those people who responded to the survey and those who did not.

25 The following methods were adopted to reduce the level and impact of non-response:
  • follow-up of respondents if there was initially no response
  • weighting to population and survey benchmarks to reduce non-response bias.

Interpretation of Results

Measuring disability

26 Disability is a difficult concept to measure because it is complex, dynamic and multidimensional. The internationally-accepted conceptual framework for classifying disability is the World Health Organisation’s International Classification of Functioning, Disability and Health (ICF). The ICF describes disability in terms of a biopsychosocial model, meaning that “disability and functioning are viewed as the outcomes of interactions between health conditions (diseases, disorders and injuries) and contextual factors”. For more information, see http://www.who.int/classifications/icf/en/.

27 Whilst the ICF provides a common language for disability, the way that these concepts are applied in surveys or censuses varies depending on the purpose of the collection and how the results will be applied.

Measuring disability using the Washington Group question modules

28 The UN Washington Group has designed a number of survey instruments to measure disability that align with the ICF framework. These measures are intended to produce comparable results between countries and include:
  • The WG Short Set on Functioning which was developed primarily for adults and is suitable for use in censuses and sample based national surveys.
  • The WG Extended Question Set on Functioning which expands on the WG Short Set to provide a more comprehensive measure of disability in population surveys or as a supplement to specialty surveys.
  • The WG Module on Child Functioning and Disability, which is an additional set of questions currently in development to measure disability among people under the age of eighteen.

29 After processing the SDS, early analysis of the data showed that the results were best output for people aged 18 years and over. This is consistent with the population that is intended to be most accurately captured by the WG Short Set.

Washington Group Short Set on Functioning

30 The WG Short Set aims to identify people in the population who are at greater risk of experiencing limited or restricted participation in society than the general population . As noted above, the questions cover six functional domains or basic actions:
  • seeing
  • hearing
  • walking
  • remembering or concentrating
  • self-care
  • communicating.

31 For each functional domain, four responses are available:
  • no, no difficulty
  • yes, some difficulty
  • yes, a lot of difficulty
  • cannot do at all.

For more information, see the Questionnaire in the Downloads tab.

32 With these response categories, the WG Short Set questions fulfil two specific data needs. They describe disability data as a continuum of functioning based on graded responses from no difficulty to unable to do at all, thereby capturing the full spectrum of functioning from mild to severe. They also define a cut-off (or a set of cut-offs) to calculate disability prevalence for six basic actions and to disaggregate outcome indicators (e.g. access to education, employment). This is done by disability status to determine if people with disability are participating in society at equal levels to those without disability.

33 The most commonly used cut-off includes everyone with at least one functional domain that is coded as a lot of difficulty or cannot do at all.

34 The WG Short Set was designed to identify the majority, but not all people with limitations and disability. The WG have recognised that the WG Short Set “may not meet all needs for disability statistics, nor will it replicate a survey of the population that can collect information across a wider range of disability domains”. The WG Short Set, for example, was not designed to cover people with intellectual or psychological functioning difficulties or young children. (For more information, see http://www.cdc.gov/nchs/data/washington_group/meeting8/NSO_report.pdf).

35 The purpose of the WG Short Set is to produce internationally comparable data on disability. The six questions are therefore at a broad level that can be translated across cultures and included in censuses. When compared to the WG’s more comprehensive Extended Question Set, whose purpose is to identify more people with disability and more limitations than the Short Set, the limitations mentioned above become more prominent.

Comparison of the SDS methods with the Survey of Disability, Ageing and Carers

36 The Survey of Disability, Ageing and Carers (SDAC) has been collected by the Australian Bureau of Statistics (ABS) since 1981 and was enumerated again in 1988, 1993, 1998, 2003, 2009, 2012 and 2015. The survey provides a comprehensive overview of disability, ageing and carers in Australia and includes measures of prevalence, level of support required by people with disability, older people and their carers.

37 The SDS and SDAC both measure disability using the ICF framework. However, due to methodological differences, direct comparisons between results of the two surveys should not be made. The ABS is currently investigating the relationship between the SDS and the 2015 SDAC Disability Identification module. The detailed analysis will be released on the ABS website in early 2017.

38 Key differences in the collection methodology are included in the table below:


Table 1 Key differences in survey methodology- Supplementary Disability Survey and Survey of Disability, Ageing and Carers


2016 Supplementary Disability Survey
2015 Survey of Disability Ageing and Carers

Age
People aged five years and over
People of all ages

Scope
Households only
Households and establishments (e.g. hospitals, nursing home, aged care hostels, cared components of retirement villages and other ‘homes’, such as group homes for people with disability).

Interview method
Computer-assisted telephone interviews
Computer-assisted personal interviews

Respondent
An adult responsible for providing information on behalf of the household.
Initial questions asked of an adult responsible for providing information on behalf of the household, followed by personal interviews for those in the target populations (with disability, older people and carers). Proxy interviews conducted for those unable to self -report.

Number of questions
Six questions, referred to as WG Short Set, designed to identify difficulties experienced with six functional domains.
An extensive set of questions designed to identify disability and the underlying conditions causing disability. A large number of these questions are used to establish if a person is restricted by disability and is in need of assistance.



International comparison


39 When comparing the disability prevalence rates from the SDS with other countries, it is important to be aware of the different approaches used, including:
  • Using a different cut-off to determine disability by lowering the threshold to include some difficulty as well as a lot of difficulty or cannot do at all.
  • Removing certain questions, adding extra questions or replacing the four possible responses with a dichotomous Yes/No response option.

For more information on the work of the WG see: http://www.cdc.gov/nchs/washington_group/index.htm.

40 The ways that the SDS differed from the standard WG approach are listed in the table below:


Table 2 Key differences in survey methodology- Washington Group Short Set and Supplementary Disability Survey


Washington Group Short Set
2016 Supplementary Disability Survey

Age
Designed for the adult population
Collected for people aged five years and over; however, data was published for people aged 18 years and over to be consistent with the WG target population.

Interview method
Face-to-face
Computer-assisted telephone interviews

Respondent
Individual (self-response)
An adult responsible for providing information on behalf of the household.

Collection type
Census or population survey
Population survey conducted as voluntary follow up to SDAC.

Question format
Questions asked in numerical order for each respondent before proceeding to next person.
Questions for each functional domain asked of every person in the household before proceeding to the next functional domain.



Classifications


41 Geography data were classified according to the Statistical Geography: Volume 1 - Australian Statistical Geography Standard (ASGS), 2011 (cat. no. 1270.0.55.001).

42 Education data were classified according to the Australian Standard Classification of Education (ASCED), 2001 (cat. no. 1272.0).

Confidentiality

43 The Census and Statistics Act, 1905 requires that statistical output shall not be published or disseminated in a manner that is likely to enable the identification of a particular person or organisation. This requirement means that the ABS must take care and make assurances that any statistical information about individual respondents cannot be derived from published data.

44 To minimise the risk of identifying individuals in aggregate statistics, a technique known as perturbation is used to randomly adjust cell values. Perturbation involves a small random adjustment of the statistics and is considered the most satisfactory technique for avoiding the release of identifiable statistics while maximising the range of information that can be released. These adjustments have a negligible impact on the underlying pattern of the statistics. After perturbation, a given published cell value will be consistent across all tables. However, adding up cell values to derive a total will not necessarily give the same result as published totals.

Rounding

45 Estimates presented in this publication have been rounded.

46 Proportions presented in this publication are based on unrounded estimates. Calculations using rounded estimates may differ from those published.

Acknowledgements

47 ABS publications draw extensively on information provided freely by individuals, businesses, governments and other organisations. Their continued cooperation is very much appreciated; without it, the wide range of statistics published by the ABS would not be available. Information received by the ABS is treated in strict confidence as required by the Census and Statistics Act, 1905.

48 The SDS was a conducted as a consultancy service for the Australian Department of Foreign Affairs and Trade (DFAT).

49 The ABS gratefully acknowledges and thanks DFAT and the Washington Group on Disability Statistics for their support, advice and expertise.

Products and Services

50 The 2016 SDS publication, including data cubes and these Explanatory Notes, are available free of charge on the ABS website <www.abs.gov.au>.

Related publications

51 Other ABS publications which may be of interest are shown under the 'Related Information' tab of this release.

52 Current publications and other products released by the ABS are listed on the ABS website. The ABS also issues a daily Release Advice on the website which details products to be released in the week ahead.

Disability, Ageing and Carers, Australia: First Results - 4430.0.10.001 - 2015
Intellectual Disability, Australia - 4433.0.55.003 - 2012
Psychological Disability - 4433.0.55.004 - 2012
Aboriginal and Torres Strait Islander People with a Disability - 4433.0.55.005 - 2012
Disability and Labour Force Participation - 4433.0.55.006 - 2012
Unmet Need for Formal Assistance - 4433.0.55.007 - 2012
Young People with Disability - 4427.0 - 2012
Autism in Australia - 4428.0 - 2012
Disability, Ageing and Carers, Australia: Summary of Findings - 4430.0 - 2012
Microdata: Disability, Ageing and Carers, Australia - 4430.0.30.002 - 2012
Disability, Ageing and Carers, Australia: Additional data cubes - 4430.0.55.009 - 2012
National Health Survey: Summary of Results - 4364.0 - 2007-2008 (Reissue)


References

World Health Organization, 2002, Toward a Common Language for Functioning, Disability and Health

Madans, J., Loeb, M. and Altman, B. 2001, Measuring disability and monitoring the UN Convention on the Rights of Persons with Disabilities, BioMed Central

Washington Group National Center for Health Statistics, 2010, Development of an Internationally Comparable Disability Measure for Censuses

Washington Group National Center for Health Statistics, 2010, Disability Information from Censuses

Washington Group National Center for Health Statistics, 2011, The Measurement of Disability: Recommendations for the 2010 Round of Censuses