MAINTAINING THE RELEVANCE OF THE CONSUMER PRICE INDEX


INTRODUCTION

11.1 At the expenditure class (published) level the ABS uses a fixed basket of goods and services in order to measure the price change in the Consumer Price Index (CPI) excluding any quality or quantity changes. However, as consumer expenditure patterns change over time in a dynamic economy, the fixed basket used in the CPI runs the risk of becoming unrepresentative and can lead to bias. There are a number of different types of bias that may affect price indexes, outlined in Price index theory of this manual. The ABS applies significant effort to address these biases. Some aspects, such as quality change, have been addressed in Price collection of this manual. This chapter includes the strategies the ABS uses to minimise the effect of substitution bias on the CPI and an estimation of one type of bias, the upper-level substitution basis.

11.2 When referring to the use of a 'fixed basket', it is important to note that it is not the expenditure weights (price x quantity) that are fixed, but the quantities of the goods and services included in the basket. The price component of the expenditure weights are price updated each quarter (see Consumer price index calculation in practice of this manual for information on price updating). It is also important to note that the items within the CPI samples (what is actually priced) are regularly updated to reflect changes to consumer preferences. The term 'fixed basket' is used as an analogy to help users understand the concept of measuring pure price change.


LIMITATIONS OF FIXED BASKET PRICE INDEXES

11.3 The production of a price index by reference to a fixed basket of goods and services has several advantages. Firstly, the concept is easy to understand; price the same basket of goods and services at two different periods, and compare the total price of the basket. Secondly, by fixing both the goods and services within the basket and their quantities, the resulting values provide a measure of pure price change that is free from compositional change. In application, this process is more complex than the basket analogy would suggest. In practice, the transactions occurring in the market place are frequently changing. This observation reveals a dilemma, namely how can a price index use a fixed basket to measure pure price change and at the same time remain both contemporary and representative of the market?


ABS STRATEGY FOR REVIEWING AND MAINTAINING PRICE INDEXES

11.4 The ABS has a policy of continual assessment of the samples of consumer goods and services that it uses in the CPI. Essentially there are three levels of maintaining representation of an index:

(i) Sample maintenance - ongoing updating and replacement of specifications, respondents, and weights for the prices collected in the CPI, which ensures that the structure of respondent samples and specifications remains relevant.
(ii) Sample review - a complete reassessment of the sample used to represent all products in the commodity classification; covering companies, products, pricing procedures and relative weights based on consumer expenditure. The end product of the sample review may be a new or revised sample (respondents, specifications and collection methods), the confirmation of the existing sample or a change to the index structure below the Expenditure Class (EC) level.
(iii) Index reviews - periodic reviews of the overall index structure and the price collection methodology and updates to the weighting pattern. The ABS currently updates expenditure weights at the EC level every six years using data from the Household Expenditure Survey (HES). However, subject to stakeholder feedback, beginning in the December quarter 2017, the CPI will be re-weighted at the EC level on an annual basis using Household Final Consumption Expenditure (HFCE) data from the National Accounts.

ITEM SUBSTITUTION, INDEX FORMULAS AND THE FREQUENCY OF CPI WEIGHT UPDATES

11.5 Item substitution occurs when households react to changes in relative prices by choosing to reduce purchases of goods and services showing higher relative price change and instead buy more of those showing lower relative price change.

11.6 Under these circumstances, a fixed-basket Laspeyres index will overstate the price change of the whole basket as it does not take account of changes in the substitutions that consumers make in response to relative price changes. For example, if the price of beef were to increase more than the price of chicken, one would expect consumers to purchase more chicken and less beef. As a fixed-base index would continue to price the original quantities of beef and chicken, the price change faced by consumers would be overstated.

11.7 Item substitution bias is due to changes in the pattern of household consumption which takes place over time as a result of both demand and supply changes. The longer the time period between weight revision periods, the more time there is for consumers to substitute towards or away from goods and services in reaction to relative price changes and as a result of changes in income. Similarly, supply conditions (and therefore the availability, or otherwise, of certain goods and services) can change substantially over the period in which the weights are fixed.

11.8 Like most CPIs, the Australian CPI is calculated using a fixed-base modified Laspeyres index formula (known as Lowe index(footnote 1) ) which keeps quantities fixed between major revisions but allows prices to vary. A Laspeyres (or in most cases a Laspeyres-type) index measures the change in the cost of purchasing the same basket of goods and services in the current period as was purchased in a specified base period. The weights reflect expenditures from a historical period, the base period. See Price index theory of this manual for more detail.

11.9 The ABS currently updates expenditure weights at the EC level every six years using data from the HES. However, subject to stakeholder feedback, beginning with the December quarter 2017, the Australian CPI will be re-weighted annually for the inter-HES years using HFCE data from the National Accounts. The use of expenditure aggregates from the National Accounts is an internationally recommended approach in situations where the time interval between household surveys is large.

11.10 The ability to re-weight the Australian CPI more frequently will have significant benefits to the user community. These include more representative expenditure weights that more accurately reflect consumer spending patterns, addressing stakeholder concerns following the 16th Series CPI review, coherence across macro-economic statistics and, improved alignment with international standards. For more information on these changes see the Information paper: Increasing the Frequency of CPI Expenditure Class Weight Updates, July 2016 (cat. no. 6401.0.60.002).

11.11 There is a family of indexes called superlative indexes. Superlative indexes make use of both beginning-of-period and end-of-period information on both prices and quantities (expenditures), thereby accounting for substitution across items. However, in order to construct a superlative index both price and quantity (expenditure) data are required for both periods under consideration.

11.12 Superlative indexes for the entire CPI basket of goods and services can only be produced retrospectively once the required weighting data are available(footnote 2) . Given that current period expenditure data for households is not available on a sufficiently timely basis (generally not available until 12 months after the reference period), a superlative formula cannot be used in the routine production of the CPI, which is why statistical agencies rely on fixed baskets. Most, if not all, statistical agencies use a Laspeyres-type index. The requirement for current period quantity data in real time is the reason a superlative index is an impractical option for statistical offices for the compilation of the CPI.


ESTIMATION OF THE UPPER LEVEL SUBSTITUTION BIAS

11.13 The ABS has constructed a retrospective superlative-type index to provide an estimation of potential item (upper level) substitution bias in the fixed-weight Australian CPI. While there are five main sources of bias in CPIs (described further in this manual in Price index theory), this analysis focuses on one type only - upper level item substitution bias - and therefore the results in the analysis should not be taken to equate to the total bias in the CPI, which will be the cumulative impact of all sources of bias. This analysis can only be conducted retrospectively when new HES data are available - currently every six years.

11.14 Superlative indexes allow for substitution as they make use of weights for both the earlier and later periods under consideration (basically averaging across historical and current expenditures to derive a ‘representative’ set of weights for the period) whereas the Laspeyres index uses only base period weights.

11.15 The estimate of upper level substitution bias has been made at relatively high levels of aggregation. The analysis is calculated based on the amount of consumer substitution between expenditure classes as this is the lowest level for which reliable weighting information (from the HES and other alternative data sources) is available and this is the level at which the underlying quantity weights remain fixed between CPI reviews. Thus, the analysis captures substitution from one expenditure class to another, e.g. from beef and veal to poultry, but not within a given expenditure class, e.g. from beef to veal. The substitution within an expenditure class is called lower level substation bias which is minimised through regular sample maintenance, sample reviews and choice of index formulas.

11.16 Two superlative indexes have been constructed and linked together to form one continuous series. The first index was constructed on the 14th series CPI basis between the June quarter 2000 and the June quarter 2005, and the second index was constructed on the 15th series CPI basis between the June quarter 2005 and the June quarter 2011.

11.17 Using the expenditure class at the weighted average of eight capital cities level, i) Laspeyres-type, ii) Paasche-type, and iii) superlative Fisher-type indexes have been calculated at the All groups CPI level.(footnote 3) The indexes have all been calculated with the base period June quarter 2000 = 100.0. The Fisher index is regarded as the best practical approximation of a 'true' (or 'ideal') price index, being the geometric average of the Laspeyres and Paasche indexes.

11.18 The Laspeyres-type index is equivalent to the published All groups CPI re-referenced to the June quarter 2000. There may be some differences in the movements compared to the All groups CPI due to rounding.

11.19 The Paasche-type and Fisher-type indexes were a retroactively modelled analytical series and are not replacing the published Australian Consumer Price Index which is designed to measure price inflation for the household sector as a whole.

11.20 The Paasche-type and superlative Fisher-type indexes were constructed using the same structure as the All groups CPI as published at the time to allow for direct comparison. The indexes from the June quarter 2000 to the June quarter 2005 were derived using the 14th series classification consisting of 88 expenditure classes. The index numbers from the June quarter 2005 to the June quarter 2011 were derived using the 15th series classification consisting of 90 expenditure classes.

11.21 Using these indexes, an estimate of upper level substitution bias in the CPI was obtained by subtracting the superlative (Fisher-type) index from the All groups CPI (Laspeyres-type) index. For the Paasche-type index, to estimate current period weights each quarter, the ABS applied a linear model between the re-weighting periods (June quarter 2000 - June quarter 2005 and June quarter 2005 - June quarter 2011).


ANALYSIS OF THE UPPER LEVEL SUBSTITUTION BIAS

11.22 The analysis found the total upper level substitution bias of the All groups CPI (as measured by the difference between the Laspeyres-type index and the Fisher-type index) was 3.6 percentage points after 11 years due to the inability of the fixed-base index to take account of the item substitution effect. The All groups CPI, calculated using a fixed-weight direct Laspeyres-type index increased by a total of 41.3% from June quarter 2000 to June quarter 2011. The retrospective superlative index, calculated using the Fisher-type index, increased by 37.7% over the same period.

11.23 To estimate the average annual upper level substitution bias, the indexes can be expressed as Compound Annual Growth Rates (CAGR).

Laspeyres_CAGR

= ((I_L,JQ11 / I_L,JQ00) ^(1/11) - 1) * 100
= ((141.3/100.0) ^(1/11) - 1) * 100
= 3.19%

Fisher_CAGR

= ((I_F,JQ11 / I_F,JQ00) ^(1/11) - 1) * 100
= ((137.7/100.0) ^(1/11) - 1) * 100
= 2.95%

11.24 The average annual upper level substitution bias was calculated as Laspeyres_CAGR - Fisher_CAGR = 3.19% - 2.95% = 0.24%. The CPI for the period June quarter 2000 to the June quarter 2011 was potentially upwardly biased by 0.24 of a percentage point per year on average due to the inability to take account of the upper level item substitution effect. These results are consistent with studies by other national statistical agencies.

11.25 The results show that the longer the period between re-weights, the larger the potential upper level item substitution bias effect on the index. Table 11.1 illustrates that the average annual substitution bias increases at a faster rate the longer the period between re-weights. The re-weighting periods in this analysis were June quarter 2000 and June quarter 2005.

11.1 Average Annual item substitution bias(a)
	Time since re-weight	LaspeyresCAGR - FisherCAGR
	(b)1 year	0.16
	2 years	0.08
	3 years	0.12
	4 years	0.15
	5 years	0.22
	(c)6 years	0.25
	Annual average between June quarter 2000 and June quarter 2011	0.24
(a) This takes the average of the average annual item substitution bias for the period June quarter 2000 - June quarter 2005 and the period June quarter 2005 - June quarter 2011.
(b) This figure includes the banana price increase in March 2006 which was a result of cyclone Larry.
(c) The six year average annual item substitution bias is only based on the index numbers for June quarter 2005 to June quarter 2011.

11.26 The result for 1 year since re-weight was caused by the introduction of the GST and cyclone Larry and can be considered atypical. Excluding this, it can be seen that the average annual item substitution bias increases over time and also increases at a faster rate, especially after the fourth year. This finding is consistent with the Statistics New Zealand (SNZ) analysis which showed that item substitution bias is considerably greater when NZ CPI weights are updated at six-yearly rather than three-yearly intervals.(footnote 4)

11.27 While there are five main sources of bias in CPIs, this analysis focuses on one type only - upper level item substitution bias - and therefore the results in the analysis should not be taken to equate to the total bias in the CPI, which will be the cumulative impact of all sources of bias.


CHOOSING AN INDEX NUMBER FORMULA

11.28 As different index number formulas produce different results, the ABS has to decide which formula to use. The usual way is to evaluate the performance of a formula against a set of desirable mathematical properties or tests. This is called the axiomatic approach. This approach is certainly useful however a few practical issues need to be considered, such as: the relevance of the tests for the application at hand; the importance of a particular test (some tests are more important than others); and even if an index formula fails a test, how close in practice will the index likely be to the best measure?

11.29 The range of tests developed for index numbers has expanded over the years. Diewert (1992) describes twenty tests for weighted index formulas, and Diewert (1995) provides seventeen tests for equally weighted (or elementary) index formulas, and attributes the tests to their authors. It is beyond the scope of this chapter to describe all the tests, but several important ones are outlined below. Many of the tests apply to both types of formulas.

• Time reversal. This test requires the index formula to produce consistent results whether it is calculated from period 0 to period 1 or from period 1 to period 0. More specifically, if the price observations for period 0 and period 1 are changed around then the resulting price index should be the reciprocal of the original index.
• Circularity (often called transitivity). This is a multi-period test (essentially a test of chaining). It requires that the multiplication of the price index obtained by going from period 0 to period 1 and from period 1 to 2 is the same as going directly from period 0 to period 2.
• Commensurability. This test requires that if the units of measurement of the item are changed (e.g. from kilograms to tonnes), then the price index should not change.

11.30 The Fisher Ideal index formula passes the tests on time reversal and commensurability; whereas the Laspeyres and Paasche only pass the test of commensurability. The Lowe index used in the Australian CPI passes the circularity and commensurability tests.

11.31 Regarding the three equally weighted price index formulas discussed in this manual in Price index theory, the arithmetic mean of price relatives (APR) fails the time reversal and circularity tests, the relative of average prices (RAP) fails the commensurability test, but the geometric mean (GM) approach passes all tests.

11.32 Although the equally weighted GM appears to have considerable appeal as an elementary index formula, there are some situations in which it produces an undesirable result. The GM cannot handle zero prices which might occur, for example, if the government introduced a policy to subsidise fully a particular good or service. In addition, the GM may not produce acceptable movements when a price falls sharply. For example, consider a price sample of two items, each selling for $10 in one period, with the price of one of the items falling to $1 in the second period. The GM produces an index of 31.6 for the second period (assuming it was 100 in the first period), a fall of around 68%. Because the GM (implicitly) maintains equal expenditure shares in each period, it effectively gives a larger weight to lower prices.(footnote 5)

11.33 The GM formula has become more widely accepted in official circles for compiling consumer price indexes. For example, Canada switched to using GMs in the late 1980s; the United States introduced the GM formula for items making up about 61% of its CPI in January 1999; and Australia began introducing the formula in the December quarter 1998. (However, where there is a likelihood of a zero price occurring in the sample or there are barriers to item substitution, it is inappropriate to use the GM, then the ABS generally uses the RAP formula instead.) Furthermore, the GM formula is prescribed by the European Union for calculation of price sample means in its Harmonised Indices of Consumer Prices (HICP).

11.34 There is another aspect to indexes that is worth considering, although it is not rated as a test in the literature. In most countries the CPI is produced at various levels of aggregation. Typically there are three or more levels between the lowest published level, and the total of all goods and services. In practice, it is desirable that the same result is obtained whether the total index is compiled directly from the lowest level or in a staged way using progressively higher levels of aggregation. This property is known as 'additivity'. Diewert (1978) shows that the fixed weighted Laspeyres and Paasche indexes may be aggregated consistently, and the Fisher and Törnqvist indexes are very close approximations of one another(footnote 6) .

1 Consumer Price Indices; An ILO Manual, by Ralph Turvey et al (ILO, Geneva 1989). <back
2 Transactions data has access to timely quantity data, however this is currently only for a subset of the CPI basket, primarily the Food and non-alcoholic beverages group (see Use of transactions data in the Australian CPI of this manual for more details). <back
3 For a description of the indexes, refer to Price index theory of this manual. <back
4 Analytical retrospective superlative index based on New Zealand’s CPI: 2014, (Statistics New Zealand, 2014), available at http://www.stats.govt.nz/browse_for_stats/economic_indicators/CPI_inflation/cpi-retrospective-superlative-index-2014/retrospective-superlative-index.aspx. <back
5 The RAP and APR formulas both give an index of 55.0 in this case. <back
6 The aggregation property of the Laspeyres and Paasche indexes allows them to be broken down into points contributions which is very useful for analysing the relative significance of items in the index, and their contributions to changes in the aggregate index. However, Diewert (2000) has a way to decompose superlative indexes. <back