1542.0 - Statistical Quality Incident Response Plan, Jun 2012
Latest ISSUE Released at 11:30 AM (CANBERRA TIME) 26/06/2012 First Issue
|Page tools: Print Page Print All RSS Search this Product|
WHAT IS A QUALITY INCIDENT?
A quality incident occurs when the quality of the data are called into question. This could occur because an anomaly has been identified with data already released in the public domain; or a potential error that could have a negative impact on the outputs and agency is identified in data that are not yet in the public domain.
Throughout this paper the term quality incident is used to mean that either the quality of the data or the processes that produce the data are being called into question (or both), as well as covering the situation when a definite error is identified in the data.
IDENTIFY A QUALITY INCIDENT
A quality incident that warrants a QIRP response is one where the consequences of the potential risk being realised are severe for either the organisation or the public. A severe situation signals extreme circumstances and is a signal to 'call a QIRP' immediately, postpone the release of the data if need be, and commence investigations into the quality incident.
In order to initiate a quality incident response plan a quality incident needs to be identified. There could be a variety of triggers that indicate that there is or is about to be a quality incident. Some of the obvious triggers that the Australian Bureau of Statistics (ABS) has noted are:
Each of the above factors by themselves may not be enough to call a quality incident, but rather a combination of some of them may lead to a quality incident being identified and a quality incident response plan being invoked.
The ABS Data Quality Framework is a useful tool that may help when trying to identify a quality incident. The ABS uses the seven dimensions of the ABS Data Quality Framework; institutional environment, relevance, timeliness, accuracy, coherence, interpretability, and accessibility; to assess, compare and declare the quality of final statistical outputs. However, the ABS Data Quality Framework can also be used as a checklist and applied to intermediate outputs at earlier stages of the statistical cycle.
When an anomaly with the data is identified, it may be useful to create a list of questions relating to each dimension of the ABS Data Quality Framework to help determine if the anomaly is actually a quality incident. By doing so, a determination can be made as to whether the quality incident is severe enough to call a QIRP. Some questions that might be useful to ask in order to determine if a quality incident has occurred are the following:
More information on the dimensions of the ABS Data Quality Framework and its uses can be found in the information paper ABS Data Quality Framework, May 2009, (cat.no 1520.0) or alternatively in the Data Quality Online tool on the National Statistical Services website (http://www.nss.gov.au/dataquality/).
Back to top
WHEN SHOULD A QUALITY INCIDENT RESPONSE PLAN BE INITIATED?
It is important that a quality incident response plan is only used when it is really required. A key point of a contingency plan is to evoke a powerful response to a problem that needs to be dealt with before proceeding with business as usual.
Using a contingency plan too often, or when it is not necessary, may decrease the effectiveness of this response, and waste time and resources, as well as risk undermining the self-confidence of team members. However, it is critical that quality issues are not overlooked for too long or worse still, missed altogether. Hence, it is important to be able to identify quality incidents that require a quality incident response plan.
An assessment of the potential quality incident should be made to determine its impact, if it is realised, on the reputation of the organisation and also on the quality of the data. A quality incident that has potentially severe consequences to the organisation or to users of data should signal that a Quality Incident Response Plan is required. The initial assessment of the potential quality incident may provide enough information for senior managers to be satisfied that the data are correct and hence, a quality incident response plan process will not be invoked.
At the Australian Bureau of Statistics, any quality incident that could cause a loss of reputation, credibility, or impact negatively on provider or user confidence due to the publication of misleading data, would instigate the initiation of a quality incident response plan to deal with the issue. For example, if the Labour Force estimates prior to release revealed that one or more of the headline indicators behaved in an unexpected way, given the previous months' trend and everything else that is known about the current economic situation, then a quality incident response plan would be invoked as a way of further quality assuring the data. This is because the consequence of the data being incorrect and released to the public would not only have a detrimental effect on the reputation of the ABS but may also impact negatively on the economy. A quality incident that is discovered after the data have been released publicly would warrant an immediate quality incident response plan.
For those quality incidents that are identified prior to the public release of data, a risk assessment of the quality incident could be undertaken to assist in determining whether a quality incident response plan is required. Each organisation has its own risk assessment matrix to assist in decision making. However, a risk assessment matrix that may be of use is the ABS Statistical Risk Management Framework . This can be found in the appendix of the Quality Management of Statistical Risk Using Quality Gates information paper.
Knowing when a potential quality incident is serious enough that it requires a quality incident response plan to be invoked is an important part in managing statistical risk. A quality incident response plan assists in determining what the causes are of the potential quality incident and how to resolve them.
Back to top
These documents will be presented in a new window.