|Page tools: Print Page Print All|
UPDATE ON RECOMMENDATIONS FROM INDEPENDENT TECHNICAL REVIEW
The independent technical review noted that "The LFS response rates have fallen from about 96% to 93% between the beginning of 2013 and September 2014. Analysis of the final 2-4% of responses in earlier years indicated that these last responses were not significantly different from the initial response. As a result, it was concluded that a drop in the response rates from 95-96% to about 93-95% should not have a significant effect on the LFS estimates."
The analysis referred to in the above quote from the independent technical review was undertaken by the ABS prior to a decision to refine, from February 2014, collection procedures for the LFS. The ABS has had increasing difficultly contacting persons selected in the survey due to changes in lifestyles. This had led to significant increases in costs which were not sustainable. Achieving the previous response rates which typically ranged from 95 to 97% each month had become increasingly difficult. As well as having cost implications, this often resulted in increased collection time exposing the statistical process to risks due a smaller window for producing results while maintaining the timeliness of the publication.
As part of a broader program to enhance the cost-effectiveness, efficiency and timeliness of ABS response follow-up strategies, while maintaining the high quality of its statistics, the ABS refined several aspects of its contact with householders including reducing the LFS enumeration period to 21 days. As a result, the LFS response rate was expected to range from 93 to 95% each month compared to previous typical response rates of 95 to 97%.
While the independent technical review concluded that "There is no indication that the lower response rates are biasing the LFS results; rather, if they are having an effect then it is likely to show up as increased noise in the estimates", the review recommended further analysis be undertaken on the response rates. The summary results of analysis into the impact of lower response rates due to a shorter enumeration period are presented in this article.
Prior to the reduction in response rate target (and related enumeration changes) in February 2014 the ABS investigated the impact of reducing the enumeration period from 25 to 21 days. The investigation selected several years of monthly LFS responses back to 2006 and assessed the impact on key survey estimates at the national and state level. Those who responded to the survey on day 22 or beyond were dropped from the dataset and key estimates recalculated. The investigation concluded that under a 21 day collection period there would be, on average, a 1.7 percentage point drop in the response rate and there would not be any significant impact on the tested estimates. The expected change for each of the key estimates at the national and state levels was minimal and of a magnitude much less than one standard error. The results of the investigation are available on request.
The ABS has conducted further analysis to understand the impact on key labour force estimates since February 2014 of missing late respondents due to lower response rates induced by a shorter enumeration period. There were three important areas of analysis.
First, the ABS examined whether the shorter enumeration period had altered non-response behaviour among households selected in the LFS. In particular, whether there was more "learned" non-response as a result of the shorter enumeration period (i.e. a household responded when first selected in the sample but stopped responding as their time in the sample progressed). Participation in the survey was monitored for individual respondents to identify if response patterns changed before and after the implementation of the shorter enumeration period. The analysis concluded that while there was a small increase in "learned" non-response it was unlikely to be large enough to influence the key labour force estimates.
Second, the ABS investigated whether missing late respondents due to response rates lower than the target 93 to 95% would impact key labour force estimates. A national response rate outside this target has only occurred once since the enumeration period changed in February 2014; this was in August 2014 when the national response rate was 92.1%. An increasing proportion of late respondents were removed to study the impact on the labour force estimates. The results indicated that while a minor decrease of 2 percentage points below 93% would not significantly impact the key labour force estimates, a much larger decrease in the response rate would bias the level of the employment estimates downward and the not in the labour force estimates upwards.
Finally the ABS investigated whether household response patterns over the enumeration period differed before and after February 2014. This analysis found there were slightly more responses early in the enumeration period but this was consistent across all labour force statuses. There is no evidence to invalidate previous investigations which found that shortening the enumeration period would not significantly impact estimates.
The data used in the pre-February 2014 investigations was prior to e-form self enumeration being offered to all LFS rotation groups from April 2014. However, the analysis described above suggests that late and non-responding units have similar characteristics before and after the e-form offer, indicating there is no evidence of interaction between the shorter enumeration period and e-form take up.
The following chart shows the monthly response rates achieved at the national level since January 2013. The response rates achieved in 2013 are higher than those for 2014 and 2015 reflecting that the reduction in response rate target and related enumeration changes were introduced in February 2014. The response rate for each August is typically lower than other months which is attributed to the supplementary survey conducted with the LFS each August. The response rates for the Australian LFS remain high by international standards.
These documents will be presented in a new window.