3317.0.55.001 - Information Paper: External Causes of Death, Data Quality, 2005  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 11/04/2007  First Issue
   Page tools: Print Print Page Print all pages in this productPrint All

ABS Quality Framework and Causes of Death Collection


The ABS uses a quality framework as the conceptual basis for the data quality assurance procedures undertaken across all ABS data collections. Its dimensions include accuracy, timeliness, integrity, relevance, accessibility, interpretability and coherence. Each of these dimensions is related and the ways that they interact can also impact on data quality. A brief discussion of these aspects of quality, as they relate to cause of death statistics, is included below. The ABS Quality Framework informs the discussion in this current paper and will continue to inform the future quality efforts for this collection.


RELEVANCE

The relevance of statistical information reflects the degree to which it meets the real needs of clients. Relevance is concerned with whether the available information sheds light on the issues most important to users. Relevance of data must be tested against the requirements of any particular user of the statistics. Users need to make a determination of the relevance of the dataset, taking into account all of the aspects of quality, for the purpose to which the statistics are being used.

Issues for causes of death data

  • The primary objective of the owner of the source data can differ from the information needs of the statistical users. Registrars of Births, Deaths and Marriages and Coroners have legislative and administrative obligations to meet, as well as being the source of statistics. As a result, the population covered by the source data, the time reference period for some data, and the data items available in the registration system, may not align exactly with the requirements of users of the statistics.
  • There can be differences between the defined scope of the population (i.e. every death occurring in Australia) and the actual coverage achieved by the registration system. Levels of registration can be influenced by external factors and coverage achieved will be influenced by the steps taken by the owners of death registration systems to ensure all deaths are registered. For example, a death certificate may need to be produced in order to finalise certain other legal requirements e.g. finalisation of a person's estate.
  • There are eight different registration systems within Australia. Each jurisdiction's registration system, whilst similar in many ways, also have a number of differences. These can include the types of data items collected and the definition of those data items, and business processes undertaken within Registries of Births, Deaths and Marriages including coding and quality assurance practices.


ACCURACY

The accuracy of statistical information is the degree to which the information correctly describes what it was designed to measure. It may be described in terms of major sources of error that potentially cause inaccuracy. Quality standards used to achieve administrative goals can be different to quality standards needed to produce statistics, although the application of sound quality assurance principles by the administrative program will usually result in quality suitable for statistical use.

Non-sample errors are the main influence on accuracy in datasets such as this which are a complete census of the population rather than a sample. Non-sample error arises from inaccuracies in collecting, recording and processing the data. The most significant of these errors are: misreporting of data items; deficiencies in coverage; non-response to particular questions; and processing errors. Every effort is made to minimise error by working closely with data providers, the careful design of forms, training of processing staff, and efficient data processing procedures.

Issues for causes of death data

The main sources of non-sample error are:
  • completeness of an individual record at a given point in time ( e.g. incomplete causes of death information due to non-finalisation of coronial proceedings)
  • completeness of the dataset e.g. impact of registration lags, processing lags and duplicate records
  • extent of coverage of the population (whilst all deaths are legally required to be registered some cases may not be registered for an extended time, if at all)
  • lack of consistency in the application of questions or forms used by data providers. For example, sometimes old forms are used before being replaced with new forms and so there could be a period of overlap when a mixture of questions is used, or different questions are asked on death registration forms on the same subject. For occupation, the questions include main occupation during working life, usual occupation, and current occupation
  • particular data items which would be useful for statistical purposes may not be collected by jurisdictions where that item is not essential for administration purposes
  • question and ‘interviewer’ biases given that information for death registrations is supplied about the person by someone else. For example, Indigenous origin as reported by a third party can be different from self reported responses on a form. Forms are often not subject to the same best practice design principles as statistical questionnaires, and respondent and/or interviewer understanding is rarely tested
  • level of specificity and completeness in coronial reports or doctor's findings on the Medical Certificate of Cause of Death will impact on the accuracy of coding
  • errors can occur in coding of the causes of a death to ICD-10. Consistency between mortality coders is a contentious issue with literature suggesting only a 50% concordance between coders at the very detailed (four digit) level (McKenzie et. al., 2001).


TIMELINESS

The timeliness of statistical information refers to the delay between the end of the reference period to which the information pertains, and the date on which the information becomes available.

Issues for causes of death data
  • A balance needs to be maintained between accuracy (completeness) and timeliness, taking account of the different needs of users.
  • Causes of death statistics are released with a view to ensuring that they are fit for purpose when released. Supporting documentation for causes of death statistics are published and should be considered when interpreting the data to enable the user to make informed decisions on the relevance and accuracy of the data for the purpose the user is going to use those statistics.
  • The availability of information from the administrative system to support processing will depend on the reasons for the system, timeliness requirements imposed by legislation or the system, resourcing of the administrative system.
  • To meet user requirements for timely data it is often necessary to obtain information from the administrative source before all information for the reference period is available (e.g. finalisation of coronial proceedings).


COHERENCE

The coherence of statistical information reflects the degree to which it can be successfully brought together with other statistical information within a broad analytical framework and over time. Coherence encompasses the internal consistency of a collection as well as its comparability both over time and with other data sources. The use of standard concepts and classifications promotes coherence.

Issues for causes of death data
  • Use of the supporting documentation released with the statistics is important for assessing coherence within the dataset and when comparing the statistics with data from other sources. For example, differences in the scope of collections, classifications used, point in time at which the analysis or coding occurred, availability of information for coding purposes, purpose for which the data/information is being produced, and confidentiality protocols may be different for ABS data and other sources of causes of death information.
  • Changes to questions, scope etc. over time can affect the consistency of data collected over the period, even when the source of the data is the same. These changes can be the result of legislative or program objective changes.
  • The completeness or quality of older versus newer data can also impact on comparisons across time or domains.
  • Statistical concepts for questions are not always suited to the administrative purpose or the means of collection.


ACCESSIBILITY

The accessibility of statistical information refers to the ease with which it can be referenced by users. This includes the ease with which the existence of information can be determined, as well as the suitability of the form or medium through which the information is being accessed.

Issues for causes of death data
  • Often an administrative source can provide the basis for statistical information which has a different nature and focus to the source's principal administrative purpose. There may be a reduced focus or availability of funding within the program to ensure the accessibility of information for non-administrative uses.
  • Each jurisdiction has its own legislation governing death registration as well as that governing the coronial process. Jurisdictions also have privacy legislation which governs the accessibility of the statistics.
  • The ABS observes strict confidentiality protocols as required by the Census and Statistics Act (1905). This may restrict access to data at a very detailed level which is sought by some users.


INTERPRETABILITY

The interpretability of statistical information reflects the availability of the supplementary information (metadata) necessary to interpret and utilise it appropriately, including concepts, classifications and measures of accuracy. In addition, interpretability includes the appropriate presentation of data to aid in the correct interpretation of the data.

Issues for causes of death data
  • Information on some aspects of statistical quality may be hard to obtain as information on the source data has not been kept over time. This is related to the issue of the administrative rather than statistical purpose of the collection of the source data.
  • Changing business rules over time and/or across data sources can affect consistency and hence interpretability of statistical output.
  • An increasing amount of information about the quality of causes of death statistics has been published in recent years. This paper is an example. The section of this paper Initiatives to Address External Causes of Death Data Quality Issues provides information about future initiatives in this area.