4714.0.55.002 - National Aboriginal and Torres Strait Islander Social Survey: Data Reference Package, 2002  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 23/06/2004   
   Page tools: Print Print Page Print all pages in this productPrint All

RELIABILITY OF THE ESTIMATES

1 Since the estimates in this publication are based on information obtained from a sample of persons, they are subject to sampling variability. That is, they may differ from those that would have been produced had all Indigenous persons aged 15 years or over been included in the survey. One measure of the likely difference is given by the standard error (SE), which indicates the extent to which an estimate might have varied by chance because only a sample of persons was included. There are about two chances in three that the sample estimate will differ by less than one SE from the number that would have been obtained if all persons had been surveyed, and about 19 chances in 20 that the difference will be less than two SEs.

2 Another measure of the likely difference is the relative standard error (RSE), which is obtained by expressing the SE as a percentage of the estimate.
RSE% = (SE / estimate) x 100

3 Space does not allow for the separate indication of the SEs and/or RSEs of all the estimates in this publication. However, RSEs for all these estimates are available free-of-charge in the National Aboriginal and Torres Strait Islander Social Survey: Data Reference Package, 2002 (cat. no. 4714.0.55.002) on the ABS web site <www.abs.gov.au>.

4 In the tables in this publication, only estimates (numbers, percentages and means) with RSEs of less than 25% are considered sufficiently reliable for most purposes. However, estimates with larger RSEs have been included and are preceded by an asterisk (e.g. *3.4) to indicate they are subject to high SEs and should be used with caution. Estimates with RSEs greater than 50% are preceded by a double asterisk (e.g. **2.1) to indicate that they are considered too unreliable for general use.

5 To assist users of this publication to ascertain the approximate levels of reliability of estimates throughout this publication, a table of SEs and RSEs for certain estimates of population counts appears at the end of the Technical Note. These values do not give a precise measure of the SEs or RSEs for a particular estimate, but will provide an indication of their magnitude.


CALCULATING STANDARD ERRORS FOR POPULATION ESTIMATES

6 An example of the calculation and use of SEs in relation to estimates of numbers of persons is as follows. Consider the estimate of the number of persons (aged 15 years or over) who hold a non-school qualification, which is 73,500 (table 7). Since this estimate is between 50,000 and 75,000, the SE will be between 2,340 and 2,700 (as shown in the SE table), and can be approximated by interpolation using the following formula:
SE = lower SE + ((size of estimate - lower size) / (upper size - lower size)) x (upper SE - lower SE)
SE = 2,340 + ((73,500 - 50,000) / (75,000 - 50,000)) x (2,700 - 2,340)
SE = (approximately) 2680
Therefore, there are about two chances in three that the value that would have been produced if all persons had been included in the survey would have fallen within the range 70,820 to 76,180, and about 19 chances in 20 that the value would have fallen within the range 68,140 to 78,860.


CALCULATING STANDARD ERRORS FOR PROPORTIONS AND PERCENTAGES

7 Proportions and percentages formed from the ratio of two estimates are also subject to sampling errors. The size of the error depends on the accuracy of both the numerator and the denominator. For proportions where the denominator is an estimate of the number of persons in a group and the numerator is the number of persons in a sub-group of the denominator group, the formula to approximate the RSE is given by:
Equation: RSE (x / y) = square root of [RSE (x)] squared - [RSE (y)] squared

8 Consider the example given above of the number of persons who held a non-school qualification. Of these, 59.1% (or approximately 43,400) identified with a clan, tribal or language group (table 7). As already noted, the SE of 73,500 is approximately 2,680, which equates to an RSE of 3.6%. The SE and RSE of 43,400 are approximately 2,210 and 5.1% respectively. Applying the formula above, the estimate of 59.1% will have an RSE of:
RSE = [RSE(43,400)]2 - [RSE(73,500)]2
= SQRT([5.1]2 - [3.7]2)
= 3.5%

9 This gives a SE for the proportion (59.1%) of approximately 2.1 percentage points (0.035 x 59.1). Therefore, if all persons had been included in the survey, there are two chances in three that the proportion that would have been obtained is between 57.0% and 61.2% and about 19 chances in 20 that the proportion is within the range 54.9% to 63.3%.


RELATIVE STANDARD ERRORS FOR MEANS

10 Estimates of means shown throughout this publication are subject to sampling error. RSEs for these estimates are available free-of-charge in the National Aboriginal and Torres Strait Islander Social Survey: Data Reference Package, 2002 (cat. no. 4714.0.55.002) on the ABS web site <www.abs.gov.au>.

Comparison of estimates

11 Published estimates may also be used to calculate the difference between two survey estimates. Such an estimate is subject to sampling error. The sampling error of the difference between two estimates depends on their SEs and the relationship (correlation) between them. An approximate SE of the difference between two estimates (x-y) may be calculated by the following formula:
Equation: SE (x - y) = square root of [SE (x)] squared + [SE (y)] squared

12 While the above formula will be exact only for differences between separate and uncorrelated (unrelated) characteristics of subpopulations, it is expected that it will provide a reasonable approximation for all differences likely to be of interest in this publication.


SIGNIFICANCE TESTING

13 Significance testing has been undertaken for the comparison of estimates in the following tables:

  • 1 - between remote and non-remote populations
  • 4 - between remote and non-remote populations and Indigenous and non-Indigenous populations
  • 5 - between remote and non-remote populations and Indigenous and non-Indigenous populations
  • 6 - between 1994 NATSIS and 2002 NATSISS populations.

14 The statistical significance test for any of the comparisons between estimates was performed to determine whether it is likely that there is a difference between the corresponding population characteristics. The standard error of the difference between two corresponding estimates (x and y) can be calculated using the formula in paragraph 11.This standard error is then used to calculate the following test statistic:
Equation: [x-y]/SE(x-y)

15 If the value of this test statistic is greater than 1.96 then there are 19 chances in 20 that there is a real difference in the two populations with respect to that characteristic. Otherwise, it cannot be stated with confidence that there is a real difference between the populations.

16 The selected tables in this publication that show the results of significance testing are annotated to indicate whether or not the estimates which have been compared are significantly different from each other with respect to the test statistic. In all other tables which do not show the results of significance testing, users should take account of RSEs when comparing estimates for different populations.


NON-SAMPLING ERROR

17 The imprecision due to sampling variability, which is measured by the SE, should not be confused with inaccuracies that may occur because of imperfections in reporting by respondents and recording by interviewers, and errors made in coding and processing data. Inaccuracies of this kind are referred to as non-sampling error, and they occur in any enumeration, whether it be a full count or a sample. Every effort is made to reduce non-sampling error to a minimum by careful design of questionnaires, intensive training and supervision of interviewers, and efficient operating procedures.


AGE STANDARDISATION

18 For this publication the direct age standardisation method was used. The standard population used was the total estimated resident population of Australia as at 30 June 2001. Estimates of age-standardised rates were calculated using the following formula:
Equation: C direct = Sum over all age categories a (Ca*Psa)

19 where Cdirect = the age-standardised rate for the population of interest, a = the age categories that have been used in the age standardisation, ca = the estimated rate for the population being standardised in age category a, and Psa = the proportion of the standard population in age category a. The age categories used in the standardisation for this publication are 18-19 years, and then five-year age groups to 65 years or over.


CALCULATING STANDARD ERRORS


Standard errors of Indigenous Persons estimates
Remote
Non-remote
Australia
Size of estimate
Standard error
Relative standard error
Standard error
Relative standard error
Standard error
Relative standard error
200
110
57
110
54
100
52
500
180
37
210
43
200
41
1,000
260
26
340
34
320
32
1,500
320
22
440
29
420
28
2,000
380
19
520
26
500
25
2,500
420
17
590
24
570
23
3,000
470
16
660
22
630
21
3,500
510
14
720
20
690
20
4,000
540
14
770
19
740
19
4,500
580
13
820
18
790
18
5,000
610
12
870
17
840
17
7,000
730
10
1,020
15
1,000
14
10,000
870
9
1,210
12
1,190
12
15,000
1,080
7
1,450
10
1,430
10
20,000
1,250
6
1,630
8
1,630
8
30,000
1,550
5
1,910
6
1,930
6
40,000
1,800
5
2,120
5
2,150
5
50,000
2,020
4
2,280
5
2,340
5
75,000
2,500
3
2,590
3
2,690
4
100,000
. .
. .
2,800
3
2,950
3
150,000
. .
. .
3,110
2
3,330
2
200,000
. .
. .
3,320
2
3,600
2
250,000
. .
. .
. .
. .
3,800
1
. . not applicable

Number of Indigenous Persons, Estimates with relative standard errors of 25% and 50%
Size of estimate
Remote
Non-Remote
Australia
no.
no.
no.
RSE of 25%
1,110
2,220
2,000
RSE of 50%
260
280
240
Standard and Relative Standard Errors for Non-Indigenous Estimates
Standard error
Relative standard error
Size of estimate
no.
%
4,000
2,100
52
4,500
2,250
50
5,000
2,390
48
6,000
2,660
44
8,000
3,140
39
10,000
3,550
36
20,000
5,160
26
30,000
6,330
21
40,000
7,320
18
50,000
8,150
16
100,000
11,200
11
200,000
15,000
8
300,000
17,700
6
400,000
19,600
5
500,000
21,500
4
1,000,000
27,000
3
2,000,000
34,000
2
5,000,000
45,000
1
10,000,000
50,000
1
Standard and Relative Standard Errors for 1994 Indigenous Estimates
STANDARD ERROR
RELATIVE STANDARD ERROR
Size of estimate
no.
%
200
110
56
500
230
46
1,000
370
37
1,500
480
32
2,000
570
29
2,500
660
26
3,000
730
24
3,500
790
23
4,000
850
21
4,500
910
20
5,000
960
19
7,000
1,130
16
10,000
1,330
13
15,000
1,580
11
20,000
1,770
9
30,000
2,050
7
40,000
2,250
6
50,000
2,400
5
75,000
2,680
4
100,000
2,870
3
150,000
3,110
2
200,000
3,270
2
250,000
3,380
1