Using SUTSE Models to Measure Statistical Impact for the Job Vacancies Survey
As an important element of the ABS transformation program, best practice guidelines for Statistical Impact Measurement (SIM) have been developed by Methodology Division. One aspect of SIM is the use of state space models to measure and manage statistical impact on time series. The ABS has previously communicated on the use of state space models for measuring statistical impacts. In preparation for onboarding to new data acquisition functionality, state space models in the form of Seemingly Unrelated Time Series Equations (SUTSE) were developed for the Job Vacancies Survey (JVS). The use of SUTSE models for the JVS was possible because it is one of the few collections that have a related time series (the ANZ Job Ads Series) that can be used to improve the predictive power of the SIM model to detect a possible impact.
Several SUTSE models were developed for SIM in JVS, with the bivariate SUTSE model incorporating an estimated trend relationship between JVS and the ANZ series, as well as an estimated relationship between the irregular/noise components of the two series, being developed as the main SIM model for JVS. Additionally, two other SUTSE models were also developed for comparison purposes. The first alternative model incorporates only an estimated trend relationship between the JVS and ANZ series with no estimated relationship for the irregular/noise components, while the second alternative model assumes no estimated relationship for neither the trend nor the irregular components. The models were applied on the assumption that the impact was to be a level shift in the time series.
With all of the three models, the Minimum Detectable Impact (MDI) indicator is used to decide whether the impact is deemed statistically significant by the model with specified significance level and power. For the JVS SUTSE model, it was set to detect a 5% level shift in the JVS series (which is about one published standard error) in the first quarter after transition with power of 50%, and a 10% level shift (which is about two published standard errors) in the first quarter after transition with power close to 98%.
The SUTSE models have been applied to measure the statistical impact on the JVS time series for both the May and August 2019 cycles (the first two cycles for the collection under new data acquisition functions). In addition to the SUTSE models, regular time series diagnostics including Regression ARIMA forecasts were applied to assess whether a statistical impact was present in the JVS time series. The results from both cycles have indicated no statistically significant impact on the JVS series from the transformation process. The application of the models have been valuable for adding an extra layer of quality assurance to the JVS data during the transitioning period. Looking forward, it is envisaged that these models will be re-deployed in future JVS cycles where significant changes are introduced to the survey, for example when a new sample design is introduced. These models also have potential to be used as a regular data confrontation tool during the quality assurance phase of each cycle.
For more information, please contact Lyndon Ang Methodology@abs.gov.au
The ABS Privacy Policy outlines how the ABS will handle any personal information that you provide to us.
Print Page
Print All