Assuring Census Data Quality

We have learnt and improved as a result of the events surrounding the 2016 Census. We are committed to collecting, processing and disseminating quality Census data that is used to shape Australia’s future.

What Happened1

DDoS Attacks
On Census night, 9 August 2016, the online form suffered a series of outages due to distributed denial of service (DDoS) attacks. Australians accessing the online form did not cause a system failure, submission rates were within expectations and load capacity.

Hardware Failure
An attempt to restore the system during the fourth DDoS attack led to the failure of one of our supplier’s routers, compounding network issues.

Monitoring Systems
Network performance monitoring systems indicated there was unusual outbound traffic from our systems. It was later determined this was a system monitoring error and there was no unusual outbound traffic from the system.

ASD Report
The Australian Signals Directorate (ASD) reported the incident was a DDoS attack and it did not result in any unauthorised access to, or extraction of, any personal information.

What We Did

Online Form Closed
At the time, we were unable to determine if the unusual outbound traffic from our systems was malicious or benign. We took decisive action to close the online form to protect the privacy of the Australian public

Pro-privacy Approach
In a statement on 11 August 2016, the Australian Privacy Commissioner (and Acting Australian Information Commissioner) said that our decision to shut down the online form was, in the circumstances, a pro-privacy precaution and was satisfied that personal information was not inappropriately accessed, lost or mishandled.

Online Form Reopened
We reopened the online form to the Australian public, 1 day 18 hours and 44 minutes after we took the decision to close the online form. During this time we implemented protocols to lessen the risk of any further outages.

Sorry Australia
We apologised to the Australian public. Within a few days of reopening the online form, we were back on track with the expected responses from Australian households. Thanks to their overwhelming support and participation, we received a total of 4.9 million online submissions and 3.5 million paper forms.

How You know Census Data Is High Quality

High Participation
The preliminary response rate for the 2016 Census is very high and comparable to the response to the 2006 and 2011 Censuses. The final Census response rate will be published on 27 June 2017, with the first release of 2016 Census data.

Post-Enumeration Survey
We significantly increased the sample size (more than 40,000 households) for the Post-Enumeration Survey, which is conducted after every Census to measure Census quality and produce high quality population estimates.

Data Quality Checks and Investigations
We undertake extensive and rigorous data quality assurance work. We review, monitor, and assess data quality through a number of steps, including analysing time series data and comparing answers within the form and with other data sources to identify fake and erroneous data.

Independent Assurance Panel
We established an independent panel of eminent Australian and international statisticians, academics, state government representatives and members of the Australian Statistics Advisory Council (ASAC) to independently review and assure the quality of statistical outputs from the 2016 Census. Their report will be published on 27 June 2017.

We will publish data quality statements and fact sheets about the quality for each Census variable, including non-response rates and results of data quality investigations, in order to explain the quality of the information to data users.

What's Being Done

Organisational Transformation
We are undergoing a major transformation at the ABS, recognising that we need to continue delivering high quality statistics including Census data, while taking advantage of new information opportunities, strengthening our partnerships, and driving high performance.

Special Adviser to the Prime Minister
The Special Adviser to the Prime Minister on Cyber Security undertook a review of the events surrounding the outage of the online Census form. We accepted and are implementing the recommendations.

Australian Senate
The Senate Economics References Committee conducted an inquiry into our preparation, administration and management of the 2016 Census. The Government tabled its response to the findings of the Committee, addressing the recommendations raised in their report.

Australian Federal Police
The Australian Federal Police is investigating the incident to attempt to identify who is responsible for the DDoS attacks.

What's Next

11 April 2017
A preview of the 2016 Census results published on the ABS website giving insight into what makes the ‘typical’ Australian at the national and state/territory level, showing what has changed over time.

27 June 2017
The first comprehensive Census dataset released including national, State/Territory and capital city data for selected key person, family and dwelling characteristics including age, sex, religion, language and income. Community level Census data will also be available. Census data quality statements and the Independent Assurance Panel report will also be released at this time.

October 2017
Detailed Census data on employment, qualifications and population mobility, including journey to work and internal migration, released.

December 2017
Further five per cent sample from the 2016 Census added to the Australia Census Longitudinal Dataset, bringing together data from the 2006, 2011 and 2016 Censuses to create a research tool for exploring how Australian society is changing over time.

1Source: Review of the events surrounding the 2016 eCensus: Improving institutional cyber security culture and practices across the Australian government—Alastair MacGibbon, Special Adviser to the Prime Minister on Cyber Security—Department of the Prime Minister and Cabinet—13 October 2016