The environment in which statistical agencies operate is changing. New opportunities to access and interrogate big data sets are becoming available, increasing the potential to provide new insights into matters of importance. The statistical landscape is becoming more complex, expectations of decision makers are growing, and National Statistical Offices (NSOs) are being challenged to deliver the best possible statistical program in more efficient and innovative ways. To remain relevant and to capitalise on these opportunities of the dynamic information environment, the Australian Bureau of Statistics (ABS) has undertaken a research program to enhance the Australian Consumer Price Index (CPI). This publication focuses on one component of that program, that being to maximise the use of transactions data to compile the Australian CPI (ABS 2015).
While transactions data provide opportunities to enhance the CPI, it also creates methodological challenges for price index practitioners. To date only a handful of countries have implemented transactions data for use in their CPI, each using different methods and practices. This is in part due to a lack of consensus amongst leading NSOs on the best aggregation method when utilising large volumes of data, as well as differences in circumstances of each NSO in producing their CPI.
This publication presents the case for change to some of the methods used to compile the Australian CPI. The methodological changes outlined in this publication will enhance the accuracy of the CPI by utilising big data sources. This publication examines the emerging academic literature relating to big data methods and supplements this literature review with ABS research. This publication concludes by outlining a path towards implementation. Of note, some technical challenges remain unresolved. User and stakeholder input is sought to contribute to the resolution of these challenges. A final ABS publication will be released on this topic in mid-2017 which will outline the precise methods to be implemented in the CPI.
The ABS would like to acknowledge the input and advice provided by Professor Jan de Haan of Statistics Netherlands and Delft University of Technology during the preparation of this publication.