THE MEASUREMENT STRATEGY FOR REGISTER SHOCKS
As a result of the reform of the Australian tax system in 2002 and the creation of the Australian Business Register (ABR) which is maintained by the Australian Tax Office (ATO), ABS surveys moved to new frames for business surveys. The ABS Business Register now comprises of ABS maintained (large, complex) units from the old register, and simple and middle units from the ABR. This move to a new frame resulted in changes in estimates caused by changes to the units model (structural changes to businesses) resulting in population changes within particular ANZSIC classes which affected the scope of many collections.
A generalised methodology for measuring the impact on estimates was developed. The methodology was developed for sub-annuals and annual collections. The basis of the methodology required the creation of a new basis frame and the 'imputation' of data on the new basis for a units on the survey frame for which actual data was not available. A method of backcasting old series was developed by Time Series. In practice it was expected that individual collections would need to fine tune the general strategy to suite their own circumstances.
In respect of the measurement strategy the main challenges/difficulties were:
- that surveys conducted on a less than annual frequency need some understanding of the changes to their series over time. The standard approach had been to ignore these effects because the surveys in general are conducted infrequently. In the end specific assessments of bridging for ad-hoc collections was required;
- that the measurement frame and associated concordances files were not perfect (nor were they expected to be) and it took time to understand and assure ourselves of the final version. Main issues were with duplication between the simple population in the new ATO maintained population and the ABS maintained population, public and private units (especially for surveys with limited scope) and resolving specific treatments for split Type of Activity Unit caused by the new business structure arrangements in the real world;
- most measurement outcomes were not able to be signed off in advance of understanding what the new publication estimates would be, as originally planned. In the end the final quality assurance process for each collection was done as part of the sign-off process for each publication. This enabled survey managers to convince themselves that the estimates on the old and new basis were coherent after the measured effects were taken into account.
A newly formed SSB Methodological Panel recently reviewed the measurement methodologies that could be employed for managing major shocks to series. It was an attempt to consider alternatives in light of the Tax Reform experiences.
The mass imputation method worked reasonably well for tax reform stream 2. However, improvements and research have been identified that should be progressed. The Stream 2 impact measurement required a new basis estimate to be calculated using old basis sample data. This was done by a mass imputation of all units on a new basis measurement frame. An issue in mass imputation methods is how to calculate standard errors.
The measurement strategy issues were considered as a time series problem. Given that the purpose of a measurement strategy is to link the new series to the old at the changeover point, there were three options if it was not possible to estimate impacts due to sampling error, (e.g. due to a higher than usual rotation rate). A three step process is more appropriate, noting that some of the steps may be skipped depending on the specific issue.
- Estimate the impact using data and frames from 3 months before the changeover point. This impact estimate and/or backcast series would be released at the same time as the first new basis statistics.
- An improved impact estimate can be made using the data from the first new basis survey. This impact would be released after the first new basis statistics, (e.g. 1 or 2 months after).
- Once 3 cycles of new basis data are available, an even better impact estimate can be made using time series methods.
Another area worth investigating is how to re-weight an old-basis sample to get new basis estimates. While this method has many of the problems faced in tax reform stream 2, it is considered possible to get estimates which could be mapped over time prior to crossing over to a new basis. This has potential for less radical changes than tax reform stream 2 where business were changing structure.
Overall, it is considered feasible to develop improved methods for managing statistical impacts.
For more information, please contact Paul Sutcliffe on (02) 6252-6759.
Email: p.sutcliffe@abs.gov.au.