6401.0.60.003 - Information Paper: Making Greater Use of Transactions Data to compile the Consumer Price Index, Australia, 2016  
Latest ISSUE Released at 11:30 AM (CANBERRA TIME) 29/11/2016  First Issue
   Page tools: Print Print Page Print all pages in this productPrint All RSS Feed RSS Bookmark and Share Search this Product

MULTILATERAL METHODS


BACKGROUND

2.1 Multilateral methods possess a number of desirable qualities, both theoretical and practical, to produce temporal price indexes from big data sources. The theoretical qualities include maintaining transitivity when reweighting and chaining frequently; while, from a practical perspective, automated processes allow a greater sample of products to be used to produce price indexes.

2.2 The international price statistics community has reached a consensus that multilateral methods are the most appropriate approach to produce temporal price indexes when using big data (UNECE 2016). At this point in time no specific multilateral method has received international endorsement. Over recent years researchers have been proposing several multilateral methods for producing temporal indexes. As described by de Haan (2015, p.1), "in spite of all the research that has been done, so far only a handful of countries have actually implemented scanner data into their CPI [...] using different methods and practices." This reticence in implementation and divergence in methods and practice stems from a number of factors, including a lack of consensus amongst researchers or leading NSOs about the best method to use, and the circumstances in which each NSO produces their CPI.

2.3 The difference between the multilateral methods themselves is in the aggregation approach each takes. For instance, the TPD method uses a regression based approach that estimates price change over time by measuring the statistical relationship between prices, products and time. The GK and Quality Adjusted Unit Value (QAUV) methods in general use a unit value approach to estimate price change and so necessitate product homogeneity to be able to standardise the quantities into common units. The difference between the QAUV_TPD and GK is in the approach to standardising the quantities - these will be discussed in this section. The GEKS method applies a geometric mean of the ratios of all bilateral indexes.

2.4 This section describes the multilateral methods the ABS is investigating for future implementation in the CPI. The choice of index formula, in part, depends on the transactions datasets that are available to the NSO. The transactions data used in this publication(footnote 1) contain the following generic data fields: city, date, SKU, product description, proprietor classifications, quantity and value sold of each product. Due to the lack of detailed product characteristics available in the data, this publication focuses on a selection of popular matched-model multilateral methods including the TPD, GK, QAUV and GEKS methods (footnote 2) .


TPD

2.5 The TPD method uses a regression approach that is similar to hedonic based methods previously used in the Australian CPI - it uses the statistical relationship between prices, products and time to directly estimate price change over time. The TPD method is a technique adapted by Aizcorbe, Corrado and Doms (2003) from the Country Product Dummy (CPD) method for spatial comparisons.

2.6 The TPD model is estimated by pooling together data for a specified window length (T+1) and modelling the log of price against time and product binary indicators. The TPD model is expressed as:

Equation : Represents the TPD model as a formula(2.1)

where,

Represents the log of price for item i in period t= log of price for item Equation: Represents item i in period Represents period t.

Represents the intercept term= intercept term

Represents the time parameter corresponding to time period t= time parameter corresponding to time period Represents period t

Represents the time dummy variable, equal to 1 if the price observation pertains to period t and 0 otherwise= time dummy variable, equal to 1 if the price observation Represents the price of item i in period t pertains to period Represents period t and 0 otherwise

Represents product parameter corresponding to product i= product parameter corresponding to product Represents product i

Represents the product dummy variable, equal to 1 if the price observation pertains to item i and 0 otherwise= product dummy variable, equal to 1 if the price observation Represents the predicted price of product i from period t pertains to item Equation: Represents item i and 0 otherwise

Represents the error term= error term

2.7 The time effect Represents the time parameter corresponding to period t reflects the overall price level in period Represents period t relative to a reference period 0, while the product effect Represents the product parameter corresponding to product i reflects the typical price of product Equation: Represents item i relative to the reference product N. Transforming equation 2.1, the predicted prices are Represents the predicted prices in the TPD for all products belonging to an EC and Represents the predicted prices in the TPD for all products belonging to an EC for all products belonging to an EC. Using the ratio of these predicted prices, the price index can be directly estimated from the modelled time effect parameters as follows:

Represents the price index estimated from the modelled time effect parameters (2.2)

where,

Represents the price movement between periods 0 and t= price movement between periods 0 and Represents period t

Represents the predicted price of product i from period 0= predicted price of product Represents item i from period 0

Represents the predicted prices of product i from period t= predicted price of product Represents item i from period Represents period t

2.8 The model specified in equation 2.1 does not produce a weighted price index using ordinary least squares estimation. In order to produce a weighted price index, expenditure shares for each product Represents product i and time period Represents period t are derived and used in fitting a weighted least squares version of equation 2.1, such that the following sum is minimised:

Represents the weighted sum of squared residuals(2.3)

where,

Represents the weighted sum of squared residuals= weighted sum of squared residuals

Represents teh set of all price observations in the window= set of all price observations in the window

Represents the expenditure share of product i relative to other products sold in time period t= expenditure share of product Equation: Represents item i relative to other products sold in time period Represents period t

Represents the residual error term of price oberservation Pti= residual error term of price observation Represents the predicted prices of product i from period t


QAUV AND GK

2.9 The QAUV and GK methods both appeal to the notions of homogeneity and unit values by expressing the quantity of products into common units, then calculating a unit value across all products. The QAUV and GK have different methods of standardising quantities, these are discussed in more detail below.

2.10 The QAUV method can be described as constructing an implicit price index using information on value and quantity available from transactions datasets. The idea behind the QAUV method is to separate products into relatively homogeneous groupings, and calculate implicit price indexes by dividing a value index by a quantity index. This is expressed as:

Represents the unit value index between period 0 and t(2.4)

where,

Represents the unit value index between periods 0 and t= unit value index between periods 0 and Represents period t

Represents the value index between periods 0 and t= value index between periods 0 and Represents period t

Represents the quantity index between period 0 and t= quantity index between periods 0 and Represents period t

2.11 While Represents the quantity index between periods 0 and t in equation 2.4 is constructed across a group of homogeneous products, it is likely that some differences in quality between individual products remain. In order to produce a standardised quantity index, each item within the homogeneous grouping can be standardised with respect to an arbitrary base item, then aggregated to produce a quantity aggregate. As a result, the standardised quantity index can be expressed as:

Respresents the standardised quantity index(2.5)

where,

Represents the standardised quantity index= standardised quantity index

Represents the adjustment factor comparing item i to base item b= adjustment factor comparing item Equation: Represents item i to base item Represents base item, b

Represents quantity of item i in period t= quantity of item Equation: Represents item i in period Represents period t

Represents quantity of item i in period 0= quantity of item Equation: Represents item i in period 0

Represents sample of products from period t= sample of products from period Equation: Represents period t

Represents sample of products from period 0= sample of products from period 0

Substituting the quantity index derived in equation 2.5 into equation 2.4, the QAUV method can be expressed as:

QAUV method described by substituting the quantity index derived in 2.5 into equation 2.4(2.6)

2.12 The expression in equation 2.6 does not explicitly describe a suitable method to calculate the adjustment factors Represents the adjustment factors, or weighted average of deflated prices of product i.This publication investigates two methods for calculating adjustment factors. Firstly, the method proposed by Chessa (2016) is investigated which is a temporal adaption of the GK method used to construct spatial price indexes, where adjustment factors are estimated using an iterative approach based on two sets of simultaneous equations. The adjustment factor is the quantity weighted average of deflated prices and can be expressed as:

Represents the weighted average of deflated prices of product i(2.7)

where,

Represents the weighted average of deflated prices of product i= weighted average of deflated prices of product Represents product i

Represents the quantity share of product i in time period= quantity share of product Represents product i in time period Represents time periodRepresents time periodEquation: Tupper

Represents price of product i in time period zeT= price of product Represents product i in time period Represents time periodRepresents time periodRepresents time period

Represents price index in time period= price index in time period Represents time periodRepresents time periodRepresents time period

The quantity share component from equation 2.7 is derived as:

Represents quantity of of product i in period z(2.8)

where,

Represents quantity of of product i in period z= quantity of product Equation: Represents item i in period Equation: z

Represents the total quantity of product i over all time periods = total quantity of product Equation: i over all time periods Equation: sEquation: eEquation: Tupper

In this publication, we denote the use of quality adjustment factors as defined above for the QAUV method as GK.

2.13 An alternative option for calculating adjustment factors is the TPD approach outlined in de Haan (2015). The TPD method uses the regression model specified earlier in equation 2.1, which estimates the predicted prices Represents predicted prices for all products and Represents predicted prices for all products for all products. In order to express quantities in constant units, the predicted prices for all products can be compared to a base item, such that the adjustment factor becomes the following:

To express quantities in constant units, the predicted prices for all products can be compared to a base item, such that this equation becomes the adjustment factor (2.9)

where,

Represents the product parameter corresponding to product i= product parameter corresponding to product Equation: Represents item i

Represents the product parameter corresponding to product b= product parameter corresponding to product Represents product b

The TPD approach is denoted in a way to form adjustment factors for QAUV as QAUV_TPD.


GEKS

2.14 The GEKS method takes the geometric mean of the ratios of all bilateral indexes (calculated using the same index number formula) between a number of entities. For spatial indexes these entities are generally countries, while for price comparisons across time, the entities are time periods.

2.15 The bilateral index formula chosen for this publication is the Törnqvist index which can be expressed as:

Represents the bilateral index formula which is the Tornqvist index between periods 0 and t (2.10)

where,

Represents the Tornqvist index between period 0 and t= Törnqvist index between periods 0 and Equation: Represents period t

Represents price of item i in period t= price of item Equation: Represents item i in period Equation: Represents period t

Represents price of item i in period 0= price of item Equation: Represents item i in period 0

Represents average expenditure share of item i across periods 0 and t= average expenditure share item Equation: Represents item i across periods 0 and Equation: Represents period t

Represents number of matched items between periods 0 and t= number of matched items between periods 0 and Equation: Represents period t

2.16 The GEKS is calculated as the geometric mean of the ratios of all matched-model bilateral indexes Represents Törnqvist index between periods l and t and Represents Törnqvist index between periods l and 0 where each period Represents period l is taken in turn as the base (de Haan 2015). The GEKS method can be expressed as:

Represents the GEKS index between periods 0 and t (2.11)

where,

Represents the GEKS index between periods 0 and t= GEKS index between periods 0 and Represents period t

Represents the Tonqvist index between periods l and t= Törnqvist index between periods Represents period l, which is taken in turn as teh base and Represents period t

Represents the Tonqvist index between periods l and 0= Törnqvist index between periods Represents period l, which is taken in turn as the base and 0

In this publication we denote the use of the Törnqvist bilateral formula to form GEKS indexes. This is referred to as GEKS indexes in the following sections.

1 See Attachment 1 of ABS (2015) for a full list of ECs where transactions data is available. <back
2 See de Haan, Willenborg and Chessa (2016) for a more exhaustive list of multilateral methods available for temporal aggregation. <back