The effects of perturbation on counts in Census data products
Making data accessible to people with different levels of experience in Census concepts and statistical knowledge is a key principle guiding our efforts in the design of Census data products. The ABS aims to provide users with access to information relevant to their needs and in suitable formats, so that Census data can be easily used to inform decision-making in many different contexts.
It’s important when accessing and interpreting Census data from our products that the information is used appropriately, as there are some common pitfalls that can arise when working with the data after it has been extracted. In particular, the effects of perturbation on the data need to be understood.
The process of perturbation
Perturbation includes the suppression of small counts so individual information cannot be determined. This is why you’ll never see counts of 1 or 2 in Census output. As perturbation is applied independent of the size of a count, any individual count or total in Census data products will be no more than a very small number away from the unperturbed value.
It is because of this process of perturbation to protect individual information that the tables in Census data products are not additive. The ABS has not changed the size or characteristics of these small adjustments – they remain the same as for the 2011 Census.
Confidentiality and protecting your data explains the perturbation process and how it protects individual respondent information.
Getting the most out of Census data
To best use Census data, the ABS recommends using Census data products to source the information that you need, rather than deriving data or aggregating counts after extracting information from Census data products. Deriving new statistics after information has been extracted from ABS data products involves summing over many instances of perturbation, meaning the derived statistics could differ significantly from the original counts.
Any derivation of new statistics (such as totals, recodes, data manipulation or custom geographic structures) should be undertaken within TableBuilder, so perturbation is applied only once to the output.
Loading customised variable groups or geographic structures to TableBuilder
The following instructions describe how a custom geographic structure or data recode can be loaded to TableBuilder to ensure perturbation is only applied once to the data. Before following the instructions below, please see creating and managing custom recodes in TableBuilder.
1. Create a custom recode in the custom data tab of TableBuilder, specific to the variable grouping required.
2. Download the file and save it in a text editing program. Do not convert to DOS format.
3. Close and reopen the file in a spreadsheet program like MS Excel and update the group name column to reflect the custom recode groups. Note: when the changes are uploaded the system will automatically create all the custom recode groups as individual recodes.
4. After making the changes, sort the spreadsheet file by group name and values.
5. Save the file in your spreadsheet program as a .csv
6. Upload the .csv file into Census TableBuilder.
Print Page
Print All