RESPONSE PROPENSITY MODELLING WITH MACHINE LEARNING
The difficulties in achieving and maintaining response rates are leading National Statistical Offices (NSOs) to focus on potential respondents and how to direct data collection efforts most efficiently to maximise the quality of outputs. This area of research is known as adaptive and responsive design and complements efforts to improve the level of response via improving the provider experience. ABS is currently seeking to identify a line of investigation regarding the use of machine learning for response propensity modelling.
For household surveys and before standard workload enumeration, we want to predict the propensities on how a geographic area will respond to different collection protocols. The propensities will inform decisions on protocols to use and the number and location of interviewers needed. Finally, after standard workload enumeration, we want to predict the likelihood of getting responses from those that have not yet responded and use this to assign follow-up protocols.
For business surveys we want to apply response propensity modelling before the Intensive Follow Up (IFU) stage to identify units that will respond without reminder letters or calls. This is a cost saving initiative, referred to as a "Gold Provider" strategy in previous ABS work.
During the IFU, we want to identify units least likely to respond regardless of how many additional reminders we make. These units would be de-prioritised in the IFU, allowing effort to concentrate on units that are more likely to deliver a response.
It is proposed that the Random Forests Machine Learning method be investigated. This method appears to be well suited to our problem because it deals with small sample sizes relative to the number and type of predictors as well as behaviour that involves factor interactions not known by the researcher at the time the model is estimated. The aggregation across trees is expected to generate more stable estimates compared to those generated from any single tree.
For more information, please contact Carl Mackin Methodology@abs.gov.au