1504.0 - Methodological News, Sep 2016  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 29/09/2016   
   Page tools: Print Print Page Print all pages in this productPrint All

QUADRATIC OPTIMISATION FOR TABLE BALANCING IN OFFICIAL STATISTICS

ABS and our international counterparts often encounter the problem of adjusting tables or time series of data to achieve internal consistency. For example, National Accounts and other statistical outputs frequently require "balancing" or "benchmarking": adjusting estimates to satisfy internal consistency constraints and/or to reconcile information from multiple sources. This balancing must be done in a way that avoids excessive changes to the data.

In recent years, several agencies have adopted a weighted-least-squares (WLS) method for table balancing, with ABS now looking to do the same. This approach relies on defining an "objective function" that measures the overall impact of adjustments to the data, both in terms of point-in-time estimates and in year-to-year movements, and then finding the balanced solution that minimises this objective - this may be thought of as the least disruptive adjustment possible, under the balancing constraints specified.

The objective function may reflect our knowledge about the quality of the inputs. For example, we may be more willing to accept large adjustments to one component of the table if its source is considered less reliable, and this is reflected in an objective function that makes that component cheaper to adjust relative to others deemed more reliable. One published approach is to construct multiple objective functions, each designed for a specific purpose, and then add these together to create an overall objective.

Current implementations of these methods require a subjective approach to weighting and to the form of the objective function. It is not always clear what the best choices might be, and different agencies have adopted different approaches.

ABS Methodology Advisory Committee (MAC) Paper #157 presents a theoretical framework that may allow a more objective approach to these questions. It shows that simple WLS objective functions are equivalent to maximum likelihood estimators (MLEs) under certain assumptions about the nature of measurement errors. This interpretation suggests a less subjective approach to setting weights, one that may be largely automated if given sufficient input data. It also implies that the approach of adding objective functions together may not be ideal, and offers an alternate way to handle this problem: it may be preferable to create a combined error model and then construct a single MLE-based objective function corresponding to this model.


Further Information
For more information, please contact Geoffrey Brent (methodology@abs.gov.au).

The ABS Privacy Policy outlines how the ABS will handle any personal information that you provide to us.