TECHNICAL NOTE 2 DATA QUALITY


RELIABILITY

1 The estimates in this release are based on information obtained from a sample survey conducted by the Australian Bureau of Statistics (ABS). The reliability of statistics derived from any collection may be affected by a range of factors that are independent from the methodology used. These factors result in non-sampling error. Sample surveys are also subject to inaccuracies that arise from the fact that a sample was selected rather than conducting a census. This type of error is called sampling error.


Sampling error

2 Sampling variability occurs when a sample, rather than the entire population, is surveyed. It reflects the difference between estimates based on a sample and those that would have been obtained had a census been conducted. One measure of the likely difference is given by the standard error, which indicates the extent to which an estimate might have varied by chance because only a sample of units was included.

3 There are about two chances in three that a sample estimate will differ by less than one standard error from the figure that would have been obtained if all businesses had been included in the survey, and about 19 chances in 20 that the difference will be less than two standard errors.

4 Another measure of sampling variability is the relative standard error (RSE), which is obtained by expressing the standard error as a percentage of the estimate to which it refers. The RSE is a useful measure in that it provides an immediate indication of the sampling error in percentage terms, and this avoids the need to refer also to the size of the estimate. Selected data item RSEs at the aggregate level for Australia are shown in the table below. Detailed RSEs can be made available on request.

Relative standard errors for key aggregates
		Private and public trading sector	General government sector
		%	%
Businesses/organisations at end June		2.2	0.0
Employment at end June		1.6	2.2
Owner/drivers on contract		11.9	9.3
Income
	Income from waste services	1.5	1.1
	Income from sales of recyclable or recoverable material	2.7	3.0
	Income from energy generated from waste	0.0	0.0
	Other sources of income	3.1	1.5
	Total income	0.8	1.3
Expenses
	Wages and salaries	1.2	2.6
	Contract and subcontract expenses for waste management services	6.0	1.8
	Fees for the treatment/processing and/or disposal of waste	3.2	3.4
	Other expenses	1.1	1.4
	Total expenses	0.8	1.4
Waste quantities
	Disposed at landfills	2.6	5.9
	Recovered or reprocessed (all facilities)	2.8	3.0

5 To illustrate the above, the estimate of total income from waste services in 2009-10, for the private and public trading sector, was $5,149.1 million. The RSE of this estimate is shown as 1.5%, giving a standard error of approximately $77.2 million. Therefore, there are two chances in three that, if all units had been included in the survey, an estimate in the range of $5,071.9 million to $5,226.3 million would have been obtained. Similarly, it implies that there are nineteen chances in twenty (i.e. a confidence interval of 95%) that the estimate would have been within the range of $4,994.7 million to $5,303.5 million.

6 The size of the RSE may be a misleading indicator of the reliability of operating profit before tax. It is possible for an estimate legitimately to include positive and negative values, reflecting the financial performance of individual businesses/organisations. In this case, the aggregated estimate can be small relative to the contribution of individual businesses/organisations, resulting in a standard error which is large relative to the estimate.

7 The size of the RSE may also be a misleading indicator of the reliability of some of the estimates for percentages and counts. These estimates appear in tables 13, 14, 15, 21 and 22. The published estimate represents the proportion of total businesses with a certain characteristic, for example, operating a Green waste recycling facility. Using the same data and methodology, an estimate of the proportion of total businesses without that characteristic could also be produced. These two estimates would have identical standard errors however their RSEs would differ. Specifically, the RSE for the smaller estimate will be larger than the RSE for the larger estimate.

Non-sampling error

8 Error other than that due to sampling may occur in any type of collection, whether a full census or a sample, and is referred to as non-sampling error. It can arise from inadequacies in available sources from which the population frame was compiled, imperfections in reporting by providers, errors made in collection such as in recording and coding data and errors made in processing data. It also occurs when information cannot be obtained from all businesses selected.

9 Although it is not possible to quantify non-sampling error, every effort was made to reduce it to a minimum. Collection forms were designed to be easy to complete and assist businesses to report accurately. Efficient and effective operating procedures and systems were used to compile the statistics. The ABS compared data from different ABS (and non-ABS) sources relating to this (the waste) industry, to ensure consistency and coherence.

10 Differences in accounting policy and practices across businesses and industries can lead to some inconsistencies in the data used to compile the estimates. Although much of the accounting process is subject to standards, there remains a great deal of flexibility available to individual businesses in the accounting policies and practices that they adopt.

11 The above limitations are not meant to imply that analysis based on these data should be avoided, only that the limitations should be borne in mind when interpreting the data presented in this publication. This publication presents a wide range of data that can be used to analyse business and industry performance. It is important that any analysis be based upon the range of data presented rather than focusing on one variable.

QUALITY INDICATORS

12 In the 2009-10 survey of waste management services, there was an overall response rate of 93% from all businesses/organisations that were surveyed and found to be operating during the reference period (89% for businesses in Australia New Zealand Standard Industrial Classification (ANZSIC) subdivision 29 and 98% for organisations in the general government sector). Data were imputed for the remaining 7% of operating businesses/organisations. Imputed responses contributed 5.3% to the estimate of total income for businesses in ANZSIC subdivision 29 and 2.4% for the general government sector.


DATA COMPARABILITY

Comparison with other ABS statistics

Australian Industry (cat no. 8155.0)

13 Key annual industry data for ANZSIC subdivision 29 Waste Collection, Treatment and Disposal Services are published in Australian Industry (cat. no. 8155.0). There are important differences between statistics published in Australian Industry and Waste Management Services and users should exercise caution when making comparisons between the two sets of estimates. A key difference is Australian Industry includes non-employing units that in total contribute less than 2% to total turnover in ANZSIC subdivision 29, whereas these are excluded from the Waste Management Services Survey. Another key difference is that Waste Management Services includes data from general government sector organisations in relation to waste management activities, whereas these general government sector units are excluded from ANZSIC subdivision 29 in Australian Industry.

14 Australian Industry presents annual summary statistics at the ANZSIC division and subdivision levels. It shows the relative performance of each ANZSIC division and subdivision, and allows patterns of change of growth to be analysed across particular segments of the Australian economy.

15 Waste Management Services supplements Australian Industry statistics with a detailed examination of the structure and performance of waste management services businesses and the waste management activity of the general government sector for the reference year of the survey. As such, the survey is not designed to monitor change over time.


Counts of Australian Businesses, including Entries and Exits (cat. no. 8165.0)

16 Estimates of the number of businesses operating in Australia can be derived from a number of sources within the ABS. They may relate to a particular point in time or may be presented as an average annual figure. However, these estimates will not always show the same results. Variations will occur because of differing data sources, differing scope and coverage definitions between surveys, as well as variations due to sampling and non-sampling error. More information about business counts can be found in the information paper A Statistical View of Counts of Businesses in Australia (cat. no. 8162.0).

17 The Waste Management Services survey is not designed to provide high quality estimates of numbers of businesses for any of the output classifications and the number of businesses in this publication are only included to provide contextual information for the user. A more robust source of counts of Australian businesses is available from Counts of Australian Businesses, including Entries and Exits, Jun 2003 to Jun 2007 (cat. no. 8165.0).

Historical comparisons

18 This survey was not designed to support accurate estimates of change over time. There have been major changes in the statistical units, frame, scope, industry classification and estimation methodology between the 2009-10, 2002-03 and 1996-97 Waste Management Services surveys. These include:

• major changes between the 1993 and 2006 editions of ANZSIC. In 2002-03 and 1996-97, waste management services were categorised at the class level (ANZSIC class 9634) based on the 1993 edition of ANZSIC. The 2009-10 survey was based on the 2006 revision of ANZSIC, in which waste management services were expanded to be categorised at the subdivision level (ANZSIC subdivision 29).
• the 2002-03 survey excluded business with recycling as their primary activity (see Explanatory Notes in the 2002-03 publication). These were included in 1996-97.
• the 2002-03 questionnaire did not have questions relating to solid landfill waste quantities. The 2002-03 collection used administrative by-product data from state and territory government departments and Environment Protection Authorities (EPAs) for quantities of waste received and disposed of at landfills in their regulated areas. This data was collected directly in the 2009-10 and 1996-97 questionnaires and relates only to those landfills in-scope of waste management services.
• in 2002-03 the data for local government was collected separately in the Local Government Environment and Natural Resources Survey.
• changes to the statistical unit as a result of the introduction of The New Tax System on 1 July 2000.
• the introduction of generalised regression estimation methodology for the private and public trading sector instead of number raised estimation.
• the introduction of the new Australian Equivalents to International Financial Reporting Standards (AEIFRS) from 1 January 2005, resulting in changed definitions applying to some items in the financial accounts of Australian businesses.

19 Consequently, estimates in this issue are not directly comparable with those in the 2002-03 and 1996-97 Waste Management Services publications.