6461.0 - Consumer Price Index: Concepts, Sources and Methods, 2016  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 21/02/2017   
   Page tools: Print Print Page Print all pages in this productPrint All

PRICE INDEX THEORY


OVERVIEW

4.1 Price indexes in one form or another have been constructed for several centuries, and are commonly used in everyday life. However, the complexities of price indexes are not always fully appreciated or understood. This chapter provides an overview of the theory and practices that underpin the construction of price indexes.(footnote 1)

4.2 The chapter commences by describing the concept of a price index as a single-number representation of information about many prices before discussing the relationship between indexes of prices, quantities and expenditures.

4.3 Two levels of construction of price indexes are described. At the lowest level is the construction of an index for a narrowly defined commodity from price observations. The other is the aggregation of these basic or elementary aggregate indexes across a range of commodities. Various mathematical formulas for constructing these indexes are discussed including problems for prices statisticians in selecting the most appropriate methodology. The advantages and disadvantages of the various formulas are discussed, along with criteria to guide decisions on the most appropriate formula.

4.4 The chapter concludes with a discussion of issues that arise in price index construction, including changes in observation numbers, quality adjustments, the inclusion of new products and index number bias.

4.5 This chapter will focus on traditional price index methods, however in the past ten years there have been significant developments in new price index construction methods involving transactions (scanner) data. These methods are discussed in more detail in this manual in Use of transactions data in the Australian CPI.


THE CONCEPT OF A PRICE INDEX

Comparing prices

4.6 There are many situations where there is a need to compare two (or more) sets of price observations. For example, a household might want to compare prices today with some earlier period; a manufacturer would be interested in comparing prices between markets to determine where to sell its output, or to compare price movements between two time periods with movements in its production costs; and economists and market analysts need to be able to compare prices between countries and over time to assess and forecast a country’s economic performance.

4.7 In some situations, the price comparisons might only involve a single commodity. Here it is simply a matter of directly comparing the two price observations. For example, a household might want to assess how the price of shampoo today compares with the price at some previous time for the same item.

4.8 In other circumstances, the required comparison is of prices across a range of commodities. For example, a comparison of clothing prices might be required. There is a wide range of clothing types and thus prices to be considered (e.g. toddlers’ jump suits, women’s fashion skirts, boys’ shorts, men’s suits). Although comparisons can readily be made for individual or identical clothing items, this is unlikely to enable a satisfactory result for all clothing in aggregate. A method is required for combining the prices across this diverse range of items allowing for the fact that they have many different units or quantities of measurement. This is where price indexes play an extremely useful role.


The basic concept

4.9 A price index allows the comparison of two sets of prices either over time (temporal indexes) or regions (spatial indexes) for a common item or group of items. In order to compare the sets of prices, it is necessary to designate one set the reference set and the other the comparison set.(footnote 2) The reference price set is used as the base (or first) period for constructing the index, and by convention in Australia is always given an index value of 100. For example, suppose for a single item the average of prices in the first set was $15 and for the second set was $30. Then, designating the first set as the reference set gives an index of 200.0 (30/15x100) for the comparison second set. Designating the second set as the reference set gives an index of 50.0 (15/30×100) for the comparison first set.

4.10 The most common price index is a comparison between sets of prices at two times (temporal indexes). The times can be adjacent (this month and previous month) or many periods apart (this year and ten years earlier). Typically the method is to nominate one set of prices as the reference prices and to revalue the quantities (or basket) of items purchased in the base period by prices in the second (or comparison) period. The ratio of the revalued comparison period basket to the value of the reference period basket provides a measure of the price change between the two periods. This simple revaluation, however, does not take account of any changes or substitutions that may be made in quantities consumed in response to changes in relative prices between the two periods. Nor does it allow for any change in tastes between the two periods. These changes to the preferences of consumers are significant in the choice of index methodology.

4.11 Handling quantity changes that occur in response to changes in relative prices is fundamental to price index construction. Changes in the relative importance of items in the basket of goods and services can have a significant effect on index movements.

4.12 Another objective of price indexes is to determine levels of household expenditure that are equivalent between two cities, say Darwin and Hobart. To do this, a spatial price index is required which allows the price levels in the two cities to be compared. This can be done by specifying a basket (i.e. quantities) of goods and services, and pricing this basket in both cities. The ratio of the total price of the basket in each city gives a measure of price relatives.

4.13 The composition of the basket would depend on the comparison required. For example, suppose the household was considering relocating from Darwin to Hobart and desired to be no worse off in terms of the overall basket of goods and services it could purchase. The reference basket should then comprise the quantities of each item currently purchased by the household in Darwin. Alternatively, if the household were in Hobart and considered relocating to Darwin, then it would specify the reference basket as the quantities of goods and services being purchased in Hobart.

4.14 The composition of the basket reflects the consumption preferences of the subject, in this case the household. It will reflect the household’s preferences under the prices and income prevailing in its current situation. Ideally, what would be required is some indication of how the household’s tastes or preferences might change between locations. Clearly the household could choose a different mix of items in Hobart than in Darwin, reflecting differences in relative prices between the cities, climate and other factors. The objective, though, is the same: to measure the relative expenditures in the two cities for which the household is equally satisfied (or indifferent).


REFINING THE CONCEPT

4.15 The remainder of this chapter focuses on the comparison of prices over time (temporal indexes). Expenditure on an individual item is the product of price and quantity, that is:

Equation: Expenditure on an individual item is the product of the price and quantity of that item in the time period it was observed

where e is expenditure, p is price, q is quantity and the subscript t refers to the time periods at which the observations are made.

4.16 Consider the expenditures on the same commodity in two different times periods. Changes in these expenditures can reflect changes in the price, changes in the quantity, or a combination of both price and quantity changes. For example, suppose the price of Granny Smith apples at a particular market is $2.00 per kg in period one, and it rises to $2.50 per kg in period two. The change in the price of apples between these two periods is obtained from the ratio of the price in the second period to the price in the first period; that is, $2.50/$2.00 = 1.25 or an increase of 25% in the price. If a consumer bought exactly the same quantity of apples in the two periods, the expenditure on Granny Smith apples would rise by 25%. However, if the amount purchased in the first period was 10 kg, and the amount purchased in the second period was 12 kg, the quantity would also have risen by a factor of 12/10 = 1.20 or 20%. In these circumstances, the total expenditure on apples increases from $20 in the first period (10 kg at $2.00 per kg), to $30 in the second period (12 at $2.50 per kg), an increase in expenditure of $10 or 50%. The ratio of the current expenditure to the previous expenditure is the product of the change in price and the change in quantity (1.25 x 1.20 = 1.50).

4.17 The ratio between the price in the current period and the price in the reference period is called a price relative. A price relative shows the change in price for one item only (e.g. the pricing of Granny Smith apples at one particular fruit market).

In terms of the formula in equation 4.1:

      Equation: expenditure on individual item one(expenditure in period 1) = Equation: price of individual item one($2.00) x Equation: quantity of individual item one(10 kg) = $20, and
      Equation: expenditure on individual item two(expenditure in period 2) = Equation: price of individual item two($2.50) x Equation: quantity of individual item two(12 kg) = $30
      where: Equation: price of individual item oneis the price per kg in period 1; Equation: quantity of individual item oneis the quantity in period 1;
      Equation: price of individual item oneis the price per kg in period 2 and Equation: quantity of individual item twois the quantity in period 2.

The ratio between the prices in the two periods, Equation: price of individual item oneand Equation: price of individual item one($2.50/$2.00 = 1.25) is the price relative.(footnote 3)

4.18 It is only necessary to have observations on two of the three components of equation 4.1 to analyse contributions to change in the expenditure. Using the apple example, suppose observations were only available on expenditure and price. The expenditure could be divided by the price to estimate the quantity (or the movements in expenditure and price could be used). Alternatively, if only expenditure and quantity data were available, expenditure could be divided by quantity to derive what's known as the 'unit (price) value'.

4.19 Now consider the case of price and quantity (and expenditure) observations on many commodities. The quantity measurements can have many dimensions, such as kilograms, tonnes, or even units (e.g. number of motor cars), and the quantities and prices of items are likely to show different movements between periods. Answers are required to questions such as these: what is the change over time in the quantity of commodities, and what is the contribution of price changes to changes in the expenditure on the bundle of commodities over time? Answering these questions is the task of index numbers: to summarise the information on sets of prices and quantities into single measures to assist in understanding and analysing changes.

4.20 In essence, an index number is an average of either prices or quantities compared with the corresponding average in a base period. The problem is how to calculate the average.

4.21 More formally, the price index problem is how to derive an index of price (Equation: Price Index) and an index of quantity (Equation: Quantity Index) such that the product of the two is the change in the total value of the items between the base period (0) and any other period (t), that is

Equation: Formula to calculate the total value of the items between the base period (0) and any other period (t),

where Equation: 4_Vtis the value of all items in period t and Equation: 4_V0is their value in period 0 (base period). Based on equation (4.1), can be represented as:

Equation: Formula to show that the value of all items in a period is equal to the sum of the product of prices and quantities of those items in that period.

that is, the sum of the product of prices and quantities of each item denoted by subscript i. The summation range (i=1..N) is not shown in order to make the formula more readable.


MAJOR INDEX FORMULAS

4.22 In presenting index number formulas, a simple starting point is to compare two sets of prices (sometimes called bilateral indexes). Consider price movements between two periods, where the first period is denoted as period 0 and the second period as period t (period 0 occurs before period t). To calculate the price index, the quantities need to be fixed at the same period in time. The initial question is what period should be used to determine the basket (or quantities). There are several possibilities.

(i) The quantities of the first (or earlier) period. This approach answers the question how much would it cost in the second period, relative to the first period, to purchase the same basket of goods and services that was purchased in the first period. Estimating the cost of the basket in the second period's prices simply requires multiplying the quantities of items purchased in the first period by the prices that prevailed in the second period. A price index is obtained from the ratio of the revalued basket to the total price of the basket in the first period. This approach was proposed by Laspeyres in 1871, and is referred to as a Laspeyres price index Equation: Laspeyres price index . It may be represented, with a base of 100.0, as:

Equation: The Laspeyres price index is derived from the ratio of the revalued basket using prices from period t, to the total value of the basket in period 0.

(ii) The quantities of the second (or more recent) period. This approach answers the question how much would it have cost in the first period, relative to the second period, to purchase the same basket that was purchased in the second period. Estimating the cost of purchasing the second period's basket in the first period simply requires multiplying the quantities of items purchased in the second period by the prices prevailing in the first period. A price index is obtained from the ratio of the total price of the basket in the second period compared to the total price of the basket valued at the first period's prices. This approach was proposed by Paasche in 1874, and is referred to as a Paasche price index Equation: Törnqvist price index. It may be represented, with a base of 100.0, as:

Equation: The Paasche price index is derived from the ratio of the total value of the basket in period t compared to the revalued basket from period 0 using prices quantities from period t..

(iii) A combination (or average) of quantities in both periods. This approach tries to overcome some of the inherent difficulties of using a basket fixed at either time period. In the absence of any firm indication that either period is the better to use as the base or reference, then a combination of the two is a sensible compromise. In practice this approach is most frequent in:
      a) the Fisher Ideal price index,(footnote 4) which is the geometric mean of the Laspeyres and Paasche indexes:
          Equation: The Fisher Ideal price index, is the geometric mean of the Laspeyres and Paasche indexes
      and
      b) the Törnqvist price index, which is a weighted geometric mean of the price relatives where the weights are the average expenditure shares in the two periods, that is:
          Equation: The Törnqvist price index, is a weighted geometric mean of the price relatives where the weights are the arithmetic average shares of total values in the two periods.

where Equation: This expression is the arithmetic average of the expenditure shares for the ith  item in the two periods.is the average of the expenditure shares for the ith item in the two periods.

The Fisher Ideal and Törnqvist indexes are often described as symmetrically weighted indexes because they treat the weights from the two periods equally.

4.23 The Laspeyres and Paasche formulas are expressed above in terms of quantities and prices. However, in practice, quantities might not be observable or meaningful (e.g. consider the quantity dimension of legal services, public transport, and education). Thus in practice, the Laspeyres formula is typically estimated using expenditure shares to weight price relatives - this is numerically equivalent to the formula (4.4) above.

4.24 To derive the price relatives form of the Laspeyres index, multiply the numerator of equation 4.4 by Equation: Ratio of price of item i from period 0, divided by itself, which gives a value of 1.and rearrange to obtain:

Equation: Rearranging Laspeyres price index after using price from period 0 to divide by 1.

where the term in parentheses represents the expenditure share of item i in the reference (or, more commonly labelled, base) period. Let:

Equation: expression of expenditure share of item i from period 0.

then the Laspeyres formula may be expressed as:

Equation: This is the price relative form of the Laspeyres index derived as the product of the expenditure share of item i  in the reference period and the price relative for the ith item in that same period

where Equation: 4_10_ais the price relative for the ith item.

4.25 In a similar manner, the Paasche index may be constructed using expenditure weights. In equation 4.5, multiply the denominator by Equation: This is the ratio of the price for the ith item in period t relative to the same price, which equal 1.and rearrange terms to obtain:

Equation: Construction of the Paasche index using expenditure weights

which may be expressed as:

Equation: Restatement of the Paasche index as the inverse of the sumproduct of the expenditure share of item i in period t to the ratio of the inverse of the price relative

which is the inverse of a ‘backward’ Laspeyres index (i.e. a Laspeyres index going from period t to period 0 using period t expenditure weights).(footnote 5)

4.26 The important point to note here is that if price relatives are used, then value (or expenditure) weights must also be used. On the other hand, if prices are used directly rather than in their relative form, then the weights must be quantities.

4.27 An example of creating index numbers using the above formulas is presented in Table 4.1. For the purposes of this exercise, a limited range of the types of commodities households might purchase is used. The quantities that these items would typically be measured in may vary. There are likely to be differences in price behaviour of the commodities over time. Further, the quantities of these items households purchase may vary over time in response to changes in prices (of both the item and other items) and household incomes.

4.28 Differences that might arise in price changes (and, by implication expenditure patterns) are illustrated by the following:
  • prices of high labour content items, such as services like a haircut, will tend to show steady trends over time relative to other items;
  • prices of high technology goods, such as tablets, tend to decline over time, either absolutely or relative to other items, reflecting productivity and technological advances;
  • prices of some items, such as fresh fruit, are affected by climatic and seasonal influences and so have volatile price movements; and
  • prices of some items might at times be influenced by changes in taxation rates (e.g. tobacco).

4.29 Price changes influence, to varying degrees, the quantities of items households purchase. For some items, such as basic food stuffs, the quantities purchased may show little change in response to price changes. For other items, the quantities households purchase may change by a smaller or greater proportionate amount than the price change.(footnote 6)

4.30 The examples in Table 4.1 reflect some of these possibilities.

4.31 In Table 4.1 the different index formulas produce different index numbers, and thus different estimates of the price movements. Typically the Laspeyres formula will produce a higher index number than the Paasche formula in periods after the base period, with the Fisher Ideal and the Törnqvist of similar magnitude falling between the index numbers produced by the other two formulas. In other words the Laspeyres index will generally produce a higher (lower) measure of price increase (decrease) than the other formulas and the Paasche index a lower (higher) measure of price increase (decrease) in periods after the base period.(footnote 7)

4.32 With the recent ability of National Statistical Offices to access transactions (scanner) data for use in their CPIs, new index construction methods have been developed to make use of the available price and quantity data in each period. These methods borrow heavily from existing methods in the production of spatial price indexes, in particular multilateral indexes. These multilateral index methods are being adapted for the purpose of producing temporal price indexes, which can be used in the production of the CPI. The use of transactions data and multilateral index methods are discussed further in this manual in Use of transactions data in the Australian CPI.


Generating index series over more than two periods

4.33 Most users of price indexes require a continuous series of index numbers at specific time intervals. There are two options for applying the above formulas when compiling a price index series.

(i) Select one period as the base and separately calculate the movement between that period and each required period. This is called a fixed base or direct index.

(ii) Calculate the period-to-period movements and chain these (i.e. calculate the movement from the first period to the second, the second to the third with the movement from the first period to the third obtained as the product of these two movements).

4.34 The calculation of direct and chained indexes over three periods (0, 1, and 2) using observations on three items, is shown in Table 4.2. The procedures can be extended to cover many periods.

4.1 COMPILING PRICE INDEXES OVER TWO PERIODS

Item
Price ($)
Quantity
Expenditure ($)
Expenditure shares
Price relatives

Period 0

White fresh bread loaves
2.90
2 000
5 800
0.3932
1.0000
Apples kg
5.50
500
2 750
0.1864
1.0000
Beer litres
8.00
200
1 600
0.1085
1.0000
LCD TV units
1 200.00
2
2 400
0.1627
1.0000
Jeans units
55.00
40
2 200
0.1492
1.0000
Total
14 750
1.0000

Period t

White fresh bread loaves
3.00
2 000
6 000
0.4220
1.0345
Apples kg
4.50
450
2 025
0.1424
0.8182
Beer litres
8.40
130
1 092
0.0768
1.0500
LCD TV units
1 100.00
3
3 300
0.2321
0.9167
Jeans units
60.00
30
1 800
0.1266
1.0909
Total
14 217
1.0000

Index number

Index formula
Period 0
Period t
Laspeyres no.
100.0
98.5
Paasche no.
100.0
97.6
Fisher no.
100.0
98.1
Törnqvist no.
100.0
98.0

Note: Any discrepancies between totals and sums of components are due to rounding.


4.35 The following illustrate the index number calculations:

Laspeyres

= (0.3932 x 1.0345) + (0.1864 x 0.8182) + (0.1085 x 1.0500) + (0.1627 x 0.9167) + (0.1492 x 1.0909) x 100
= 98.51

Paasche

= 1 / ((0.4220 / 1.0345) + (0.1424 / 0.8182) + (0.0768 / 1.0500) + (0.2321 / 0.9167) + (0.1266 / 1.0909)) x 100
= 97.62

Fisher

= (98.51 x 97.62)1/2
= 98.06

Törnqvist is best calculated by first taking the logs of the index formula

= 1/2 x (0.3932 + 0.4220) x ln (1.0345)
+ 1/2 x (0.1864 + 0.1424) x ln (0.8182)
+ 1/2 x (0.1085 + 0.0768) x ln (1.0500)
+ 1/2 x (0.1627 + 0.2321) x ln (0.9167)
+ 1/2 x (0.1492 + 0.1266) x ln (1.0909)
= -0.0199

and then taking the exponent multiplied by 100

= e-0.0199 * 100
= 98.04

4.2 CONSTRUCTING PRICE INDEX SERIES

Item
Period 0
Period 1
Period 2

Price ($)

1
10
12
15
2
12
13
14
3
15
17
18

Quantity

1
20
17
12
2
15
15
16
3
10
12
8

Index number

Index formula
Laspeyres
Period 0 to 1
100.0
114.2
Period 1 to 2
100.0
112.9
chain
100.0
114.2
128.9
direct
100.0
114.2
130.2
Paasche
Period 0 to 1
100.0
113.8
Period 1 to 2
100.0
112.3
chain
100.0
113.8
127.8
direct
100.0
113.8
126.9
Fisher
Period 0 to 1
100.0
114.0
Period 1 to 2
100.0
112.6
chain
100.0
114.0
128.3
direct
100.0
114.0
128.5



4.36 In this example, the Laspeyres Chain Index for period 2 is calculated as follows:

(114.2/100) * (112.9/100) * 100
= 128.9

The Paasche Chain Index for period 2 is calculated as follows:

(113.8/100) * (112.3/100) * 100
= 127.8

And the Fisher Chain Index for period 2 is calculated as follows:

(114/100) * (112.6/100) * 100
= 128.3

OR

(128.9 * 127.8)1/2
= 128.3

4.37 An index formula is said to be 'transitive' if the index number derived directly is identical to the number derived by chaining. In general, no weighted index formula will be transitive because period-to-period calculation of the index involves changing the weights for each calculation. This can be seen in Table 4.2 where in period 2 the direct Laspeyres (130.2) is different to the chain Laspeyres (128.9) due to the different quantities. The index formulas in Table 4.2 will only result in transitivity if there is no change in the quantity of each item in each period or if all prices show the same movement. In both these unlikely cases, all the formulas (Laspeyres, Paasche and Fisher) will produce the same result.

4.38 The direct Laspeyres formula has the advantage that the index can be extended to include another period's price observations when available, as the weights are fixed at some earlier base period. On the other hand, the direct Paasche formula requires both current period price observations and current period weights before the index can be calculated.


Setting the CPI basket of goods and services in practice

4.39 The households’ expenditures on all consumer goods and services in the Consumer Price Index (CPI) basket is mainly sourced from information derived from the Household Expenditure Survey (HES). However, the results from the HES are not available until approximately 12 months after the end of the survey. The Laspeyres index requires either quantities or expenditure in the base period which would mean the CPI would be unable to be calculated on these expenditures until approximately 16 months after the HES is completed.

4.40 The CPI is a quarterly survey which means the ABS must continue to calculate the CPI on the old expenditures until the new expenditures are available. When the new expenditures are available, a statistical office can then recalculate the CPI based on the new weights. However, this will lead to revisions to previously published CPI estimates which is not desirable for any contract indexation. The alternative is to use a class of price indexes called a Lowe index which defines the index as the percentage change, between the periods compared, in the total cost of purchasing a fixed basket of quantities. Most statistical offices make use of some kind of Lowe index in practice.

4.41 To calculate a price index, any set of quantities could be used. These do not have to be restricted to quantities or expenditures purchased in one period and could be arithmetic or geometric averages of the quantities of multiple periods. For the Australian CPI, the quarterly percentage change from the June quarter 2011 onwards is mainly based on the HES which was collected in respect of the financial year 2009-10. Prior to this, the CPI from the June quarter 2005 was based on the HES which was collected in respect of the financial year 2003-04. For a complete listing of the historical CPI weighting patterns see Consumer Price Index: Historical Weighting Patterns, 1948-2011 (cat. no. 6431.0).

4.42 The period whose quantities are actually used in a CPI is described as the weight reference period. In the 16th series this generally refers to the HES which is 2009-10 and it will be denoted as period b. Period 0 is the price reference period which is the June quarter 2011 in the 16th series CPI. The Lowe index using the quantities of period b can be written as follows:

Equation: 4_13_a

where Equation: 4_13_b

4.43 Similar to the Laspeyres index described earlier, the Lowe index can be calculated as either the ratio of prices and quantities, or as an arithmetic weighted average of the price relatives. The expenditures refer to quantities in period b (e.g. 2009-10) and prices in period 0 (e.g. June quarter 2011). Lowe indexes are widely used for CPI purposes.

4.44 The Laspeyres and Paasche indexes are two special cases of the Lowe price index. When the quantities are those of the price reference period, that is when b=0, the Laspeyres index is obtained. When quantities are those of the other period, that is when b=t, the Paasche index is obtained.


Unweighted, or equally weighted indexes

4.45 In some situations, it is not possible or meaningful to derive weights in either quantity or expenditure terms for each price observation. This is typically so for a narrowly defined commodity grouping in which there might be many sellers (or producers). Information might not be available on the total volume of sales of the item or for the individual sellers or producers from whom the sample of price observations is taken. In these cases, it seems appropriate not to weight, or more correctly to assign an equal weight, to each price observation. It is a common practice in the CPI in many countries that the price indexes at the lowest level (where prices enter the index) are calculated using an equally weighted formula, such as an arithmetic mean or a geometric mean.

4.46 Suppose there are price observations for n items in period 0 and period t. Then three approaches(footnote 8) for constructing an equally weighted index are as follows.
      (i)Calculate the arithmetic mean of prices in both periods and obtain the relative of the current period’s average to the base period’s average (i.e. divide the current period’s average by the base period’s average). This is the relative of the arithmetic mean of prices (RAP) approach, also referred to as the Dutot formula:
          Equation: 4_14
      (ii) For each item, calculate its price relative (i.e. divide the price in the current period by the price in the base period) and then take the arithmetic average of these relatives. This is the arithmetic mean of price relatives (APR) approach, also referred to as the Carli formula:
          Equation: 4_15
      (iii) For each item, calculate its price relative, and then take the geometric mean(footnote 9) of the relatives. This is the geometric mean (GM) approach, also referred to as the Jevons formula:
          Equation: 4_16

4.47 Although these formulas apply equal weights, the implicit basis of the weights differs. The geometric mean applies weights such that the expenditure shares of each observation are the same in each period. In other words, it is assumed that as an item becomes more (less) expensive relative to other items in the sample the quantity declines (increases) with the percentage change in the quantity offsetting the percentage change in the price. The RAP formula assumes equal quantities in both periods. That is, the RAP assumes there is no change in the quantity of an item purchased regardless of either its price movement or that of other items in the sample. The APR assumes equal expenditures in the base period with quantities being inversely proportional to base period prices.

4.48 The following are calculations of the equal weight indexes using the data in Table 4.2. Setting period 0 as the base with a value of 100.0, the following index numbers are obtained in period t:

RAP formula:

APR formula:

GM formula:

4.49 Theory suggests that the APR formula will produce the largest estimate of price change, the GM the least and the RAP a little larger but close to the GM.(footnote 10) Empirical examples generally support this proposition,(footnote 11) although with a small sample as in the example above, substantially different rankings for the RAP formula are possible depending on the prices.

4.50 The behaviour of these formulas under chaining and direct estimation is shown in Table 4.3 using the price data from Table 4.2. The RAP and GM formulas are transitive, but not the APR.

4.3 LINKING PROPERTIES OF EQUAL WEIGHT INDEX(a)

Formula
Period 0
Period 1
Period 2

Relative of average prices (RAP)

period 0 to 1
100.0
113.5
period 1 to 2
100.0
111.9
chain
100.0
113.5
127.0
direct
100.0
113.5
127.0

Average of price relatives (APR)

period 0 to 1
100.0
113.9
period 1 to 2
100.0
112.9
chain
100.0
113.9
128.6
direct
100.0
113.9
128.9

Geometric mean (GM)

period 0 to 1
100.0
113.8
period 1 to 2
100.0
112.5
chain
100.0
113.8
(b)128.0
direct
100.0
113.8
(b)128.1

(a) Uses the same price data as in Table 4.2.
(b) Difference in calculated index is due to rounding.



Unit values as prices

4.51 A common problem confronted by index compilers is how to measure the price of items in the index whose price may change several times during an index compilation period. For example, in Australia petrol prices change almost daily at many outlets, but the CPI is quarterly. Taking more frequent price readings and calculating an average is one approach to deriving an average quarterly price. A more desirable approach, data permitting, would be to calculate unit values and use these as price measures.(footnote 12) Unit values are obtained by dividing expenditure by a quantity (e.g. the total expenditure of petrol sold in a particular period divided by the number of litres sold will give a unit value per litre for the price of petrol over the period). Unit values can be used to measure price changes only for similar (homogeneous) products.

4.52 For example, suppose outlet X sells chocolate bars in weights of 50g, 80g and 100g. Further, suppose the outlet keeps records of the value of sales of these chocolate bars in aggregate and the number of each size of chocolate bar sold. It is then possible to calculate the total quantity of chocolate sold in grams. Dividing the expenditure on chocolate by the total quantity in grams produces a unit value that could be used as the price measure for chocolate.

4.53 The advent of transactions (scanner) data from retail outlets is making the construction of unit values more feasible. Transactions data provide information about both values and quantities at the point of sale, and so enable the collection of a large number of unit values at fine levels. In effect, these data would remove any need for the unweighted index formulas discussed above (at least for those items where unit values are available). For more detail on the use of transactions data and unit values see Use of transactions data in the Australian CPI of this manual.


RESOLVING EXPENDITURE AGGREGATES

4.54 It is appropriate at this point to re-examine the decomposition of an expenditure aggregate into price and quantity components introduced in equation 4.1. It is important to know the form of the quantity index when a particular form of the price index is used (and vice versa) to ensure the accurate decomposition of the value change.

4.55 A value is the product of a price and a quantity (in its simplest form, the price of a single item multiplied by 1 is the value of the item). It follows that changes in the value of expenditure on an item from period to period are the result of changes in the prices or quantities or both. If any two of the value, price or quantity are known, the third can be derived (i.e. E = P x Q, where E = expenditure, P = price and Q = quantity), e.g. Q=E/P. The calculation is straightforward when a single item is involved. However, in the case of an expenditure total that is the sum of several items, breaking up that expenditure into its price and quantity components becomes more complicated.

4.56 Price indexes provide a means of removing the effects of price changes from changes in expenditure so that the underlying changes in quantity can be identified. In the Australian National Accounts, price indexes are widely used in the process of estimating changes in volumes of expenditure, production etc. The process of using price indexes in this way is known as price deflation, with the index termed a deflator. The form of price index (current or fixed weighted) will determine the resulting index of quantity change.

4.57 The change in an expenditure or value aggregate between period 0 and t may be expressed as:

Equation: The change in an expenditure aggregate between period 0 and t may be expressed as the sum product of prices and quantities in period t divided by the sum product of prices and quantities in the base period (period 0).

4.58 Multiplying the right-hand side of equation (4.17) by Equation: This is a Paasche quantity index using price from period t anf quantities from periods t and 0.allows the equation to be expressed as:

Equation: This is an example of a Laspeyres decomposition. The change in an expenditure aggregate between two periods is the product of a Laspeyres price index and a Paasche volume index.

where the first term on the right-hand side of the equals sign is a Laspeyres price index and the second is a Paasche volume index.(footnote 13) This is referred to as the Laspeyres decomposition. In other words, if an index of value change is deflated by a base- period-weighted price index, then the index of quantity change is a current-period-weighted quantity index.

4.59 An alternative decomposition of the change in the expenditure aggregate is obtained by multiplying the right-hand side of (4.17) by Equation: The sum product of prices from period 0 and quantities from period t, divided by itself, which has a value of 1.which produces:

Equation: This is an example of a Paasche decomposition. The change in an expenditure aggregate between two periods is the product of a Paasche price index and a Laspeyres volume index.

where the first term on the right-hand side of the equals sign is a Paasche price index and the second is a Laspeyres volume index. This is referred to as the Paasche decomposition. In other words, if an index of value change is deflated by a current-period-weighted price index, then the index of quantity change is a base-period-weighted quantity index.

4.60 A similar decomposition can also be undertaken for the Fisher Ideal index. By taking the geometric average of the alternative Laspeyres and Paasche decomposition of value change (right-hand sides of equations (4.18) and (4.19)) it can be shown that value change is the product of Fisher Ideal price and quantity indexes.


SOME PRACTICAL ISSUES IN PRICE INDEX CONSTRUCTION

Handling changes in price samples

4.61 All the index formulas discussed above require observations on the same items in each period. In some situations it may be necessary to change the items or outlets included in the price sample or, if weights are used, to re-weight the price observations. Examples of changes in a price sample include:
  • a respondent goes out of business;
  • the sample needs to be updated to reflect changes in the market shares of respondents;
  • to introduce a new respondent; or
  • to include a new item.

4.62 It is important that changes in price samples are introduced without distorting the level of the index for the price sample. This usually involves a process commonly referred to as splicing. Splicing is similar to chaining except that it is carried out at the level of the price sample. An example of handling a sample change is shown in Table 4.4, for equally weighted indexes assuming a new respondent is introduced in period t. A price is also observed for the new respondent in the previous period t-1. The inclusion of the new respondent causes the geometric mean to fall from $5.94 to $5.83. The index should capture the effect of respondent 4's price movement between period t-1 and t without capturing this recorded price change due to the inclusion of a new respondent.

4.4 CHANGE IN SAMPLE - INTRODUCING A NEW RESPONDENT

Price
Price relative
Respondent
Period 0
Period t-1
Period t
Period 0
Period t-1
Period t

Observations in period t-1

1
4.00
5.50
6.00
1.000
1.375
1.500
2
4.50
4.50
5.00
1.000
1.000
1.111
3
5.00
5.50
7.00
1.000
1.100
1.400
Geometric mean (GM)
4.48
5.14
5.94
1.000
1.148
1.326

Observations in period t

1
4.00
6.00
6.50
1.000
1.500
1.625
2
4.50
5.00
5.50
1.000
1.111
1.222
3
5.00
7.00
7.00
1.000
1.400
1.400
4
-
5.50
6.00
1.000
1.326
1.447
GM (all items)
5.83
6.22
1.000
1.326
1.416
GM (matched sample)
5.94
6.30

- nil or rounded to zero (including null cells)


4.63 In the case of the APR and GM formulas, the process involves:
  • setting the previous period price relative for period t for the new respondent (4) equal to the average of the price relatives of the three respondents included in period t-1 (1.326); and
  • applying the movement in respondent 4’s price between period t-1 and t to derive a price relative for period t (6.00/5.50 x 1.326=1.447).

4.64 For these two formulas, the average of the price relatives is effectively the index number, so the GM index for period t-1 is 132.6 and for period t is 141.6.

4.65 In the case of the RAP formula, the method is similar, but prices are used instead of price relatives. The RAP formula uses the arithmetic mean of prices (not the arithmetic mean of the price relatives). The index for RAP can be calculated from the period-to-period price movements:
  • between the base period and period t, the movement in the average price was 1.333 (6.00/4.50) without the new respondent;
  • between period t-1 and t, the movement in the average price was 1.063 (6.25/5.88) including the new respondent in both periods; and
  • thus the index for period t is 141.7 (1.333 x 1.063 x 100).


Temporarily missing price observations

4.66 In any period, an event may occur that makes it impossible to obtain a price measure for an item. For example, an item could be temporarily out of stock or the quality is not up to standard (as may occur with fresh fruit and vegetables because of climatic conditions).

4.67 There are a few options available to deal with temporarily missing observations. These include:

(i) repeat the previous period’s price of the item;

(ii) impute a movement for the item based on the price movement for all other items in the sample; or

(iii) use the price movement from another price sample.

4.68 Approach (ii) is equivalent to excluding the item, for which a price is unavailable in one period, from both periods involved in the index calculation. It strictly maintains the matched sample concept.

4.69 An example of imputing using the first two approaches for the equally weighted formula is provided in Table 4.5. The example assumes that there is no price observation from respondent B in period 2.

4.5 IMPUTATION OF MISSING PRICE OBSERVATIONS

Respondent
Period 0
Period 1
Period 2
Period 3

Price ($)

A
10.00
11.00
12.00
13.00
B
12.00
13.00
-
12.00
C
15.00
15.50
14.50
17.00
D
14.00
13.50
15.00
18.00

Price relatives

A
1.000
1.100
1.200
1.300
B
1.000
1.083
-
1.000
C
1.000
1.033
0.967
1.133
D
1.000
0.964
1.071
1.286

Impute using previous period's price

Price for respondent B
12.00
13.00
13.00
12.00
Imputed relative for B (e.g. 13.00/12.00)
1.083

Indexes

RAP
100.0
103.9
106.9
117.6
APR
100.0
104.5
108.0
118.0
GM
100.0
104.4
107.7
117.3

Impute using average price movement for other items in sample

RAP
Arithmetic mean price of A, C and D
13.33
13.83
Imputed price for B (e.g. 13.00x(13.83/13.33))
13.49
Index
100.0
103.9
107.8
117.6
APR
Arithmetic mean of relatives of A, C and D
1.032
1.079
Imputed relative for B (e.g. 1.083x(1.079/1.032))
1.132
Index
100.0
104.5
109.3
118.0
GM
Geometric mean of relatives of A, C and D
1.031
1.075
Imputed relative for B (e.g. 1.083x(1.075/1.031))
1.129
Index
100.0
104.4
108.8
117.3

- nil or rounded to zero (including null cells)



HANDLING CHANGES IN GOODS AND SERVICES

Quality change

4.70 A price index by definition measures what can be described as pure price change; that is, it is not distorted by changes in quality. The concept of a good or service within a price index is important in determining whether an item has changed (i.e. new or a modification) compared to the previous period. Under the usual index compilation practices, if the change in price of the item fully or partly reflects a change in quality, then for index purposes an adjustment is necessary to account for that quality change. If it is a new item, then that item must be introduced into the index by linking (or splicing).

4.71 There are two main approaches to treating goods and services for the purposes of compiling a price index. The conventional or goods approach is to treat each good and service as a separate item; for example, a distinction might be made between red and green apples. The alternative approach could be termed a characteristics approach that takes commodities and tries to identify the component characteristics or attributes which are valued by the consumer. For example, the characteristics of an apple which households value might be its taste, nutritional content plus the ability to consume without having to perform any food preparation. The outcome is that consumers satisfy their hunger.(footnote 14)

4.72 Strict adherence to a goods approach where each good and service is treated as a separate item would see frequent linking in response to any change in the specifications of individual items priced. Frequent linking is undesirable as each link is effectively a break in the series and can introduce bias. Any observed difference in price between two items at the same point in time would be treated as quality change. In a consumer price index these adjustments should be based, as far as possible, on the value of the quality change to the consumer (user value). In this respect, use of only differences in observed prices or manufacturing cost (resource cost) data to value quality change may be misleading.(footnote 15)

4.73 The characteristics approach provides a conceptual basis for describing quality change. In the context of price indexes, quality can be thought of as embracing all those attributes or characteristics of an item on which the consumer places some value.(footnote 16) Take apples as an example. Consumers will value them for nutritional content as well as taste and absence of blemishes and bruising. The price index will be biased unless an apple of the same quality is priced each period. For some items quality change over time is not a major issue (e.g. the quality change in apples might only reflect differences in growing conditions between seasons), but for other items quality changes are very important (e.g. the increase in power and speed of laptops, and changes in safety and fuel efficiency of motor vehicles). In practice the ABS uses observable characteristics to adjust for quality where possible (e.g. size or weight).

4.74 The characteristics approach has not been used so far as the sole basis for constructing a consumer price index. However, it is the foundation of the so-called hedonic technique for estimating pure prices for commodities.(footnote 17) The hedonic technique is now being used by some countries in their CPIs for some types of consumer goods.(footnote 18) Essentially the hedonic approach involves estimating a relationship between a commodity’s price and the characteristics that it contains (e.g. for laptops, a relationship might be estimated between the price of the computer and its processing power (chip type and speed), amount of Random Access Memory (RAM), hard disk size, etc. over a range of computers). This effectively imputes a price for each characteristic that can be used to adjust prices as specifications change.(footnote 19)

4.75 Although intuitively appealing, the hedonic technique is difficult to apply in practice. It requires a lot of information and the careful selection of attributes that would be appropriate in a household utility function (e.g. if performance is one characteristic of a motor vehicle that consumers desire, would engine power or acceleration speed or some other parameter be the best measure of it). In addition, there are issues such as the functional form to be used and weighting.(footnote 20) Nevertheless, the hedonic technique does provide a tool that may assist in identifying the characteristics of commodities that influence their price, and it does provide a basis for adjusting for quality change.

4.76 Changes to goods or services that are perceived to have little or no increase in user value should be treated as a price change. This can also be the case for government mandated changes such as energy rating standards for newly constructed dwellings. For more information on quality change see Quality change and new products of this manual.


Prices of services

4.77 The CPI includes a range of services ranging from medical, insurance, childcare to gardening and hairdressing. Prices are generally collected for a fixed service such as a procedure, set of tasks or period of time (e.g. 4 hours of child care). For services that are not directly observable each period to constant quality such as real estate charges, regression modelling techniques are used to derive a final price. Quality changes for such services are very difficult to measure. For example, with a female haircut and colour, it is difficult to capture quality change such as improved ingredients or staff training over time. Generally any observed price changes are recorded as actual price change for services.


New goods

4.78 Prices statisticians are often confronted with the problem of determining when a new item on the market is a new good for index construction purposes. A completely new good is not easily included in an existing price collection because there is no product category to which it can be readily classified. In these cases, it may eventually require its own separate recognition within the index rather than being a part of an existing product group.

4.79 The use of a hedonics or characteristics approach may assist in defining new goods. For example, the hedonics approach might suggest that DVDs are not actually new goods, but rather a better bundling of sound and images and other characteristics that people value (such as a more durable medium).

4.80 The difficulty of new goods is that they often show substantial falls in price once they gain market acceptance (sometimes after improvements in quality), and the supply of the goods expand. There are two problems here. The first is that the traditional fixed-weighted index does not allow for the introduction of new goods until weights are updated. The second is that if the new good is not included until some time after establishing a significant market share, then the initial phase of falling prices is missed.

4.81 It has been suggested (Hicks (1940), and Fisher and Shell (1972)) that, in a cost-of-living framework, new goods should be valued at their demand reservation price. This price is the intercept of the demand curve with the price axis, essentially the price at which no units of the good would be sold. However, procedures to estimate reliably the demand reservation price have yet to be established.


BIAS IN PRICE INDEXES

4.82 Some of the issues about bias have been covered in this manual. However, it is useful to bring these matters together to consider further some of the practical issues involving price indexes, especially considering a major inquiry into the issue was held in the United States in 1996.(footnote 21)

4.83 A price index may be described as biased if it produces estimates which depart from a notionally true or correct measure. In the case of consumer price indexes, the true measure is usually taken to be the cost-of-living index, as it allows for the substitutions in consumption that consumers make in response to changes in relative prices. As it is impractical to construct a true cost-of-living index, official agencies are forced into second-best solutions.

4.84 The following types of bias, typically upwards, have been described by Diewert (1996).
      (i) Elementary index bias, which results from the use of inappropriate formulas for compiling index numbers at the elementary aggregate level;
      (ii) Substitution bias, which arises from using formulas at levels above the elementary aggregates which do not allow for substitution in response to changes in relative prices;
      (iii) Outlet substitution bias, which occurs when consumers shift their purchases from higher cost outlets to lower cost outlets for the same commodity;
      (iv) Quality adjustment bias, which arises from inadequate adjustment for quality changes; and
      (v) New-goods bias, which arises largely from the failure to include new goods when first introduced into the market.

4.85 Although it is almost impossible to eliminate these sources of bias, some measures can be taken to minimise them.
      (i) Use appropriate formulas in compiling elementary aggregate indexes, in particular use of the GM formula where appropriate or the RAP formula.
      (ii) Use a superlative index formula rather than the Laspeyres, if current-period weighting data can be obtained on time. More frequent updating of weights in the Laspeyres formula is also suggested, although changing weights alone does not have a significant effect in the short to medium term unless the change in the weighting pattern is significant.(footnote 22) Other options might be to use formulas that allow substitution or assumptions about substitution between commodity groupings to be entered.
      (iii) Closely monitor and update price samples to reflect changes in the outlets from which households purchase. For example, there is clearly a need to plan for the inclusion in consumer price indexes of purchases from outlets operating exclusively online.
      (iv) Make greater use of the hedonic technique to adjust for quality change and to determine comparable items.
      (v) Include new goods into the CPI as soon as possible. For a fixed-weighted index such as Laspeyres, there would also be a need to update the fixed weights to allow for the inclusion of the new goods if they are substituting for all goods in general, or to adjust the weights within a commodity grouping if the new good is substituting for specific items. For example, one could argue that CDs were a new good, but as they were substituting for records and tapes they could be introduced into the commodity grouping for records and tapes, and weights between these items adjusted accordingly.


CONCLUSION

4.86 Price index theory guides prices statisticians as to the best practices and formulas to use in compiling price indexes in order to produce reliable price measures. However, the highly desirable must be balanced against the practical. It would be highly desirable to use a superlative index formula such as the Fisher Ideal, but this is often not possible because of data problems and issues with timeliness.

4.87 There is much more to a price index than which formula to use. Also important is the determination of what items are to be included in the index, that is the index domain. This subject is covered next in Coverage and classifications of this manual.

1 For a detailed discussion of price index theory and internationally recommended practices, see Consumer Price Index Manual, Theory and Practice, 2004 (International Labour Office). <back
2 This is the terminology used by Pollak (1971). <back
3 In this example, the price relative shows the change in price between two times. If, instead of two different periods we looked at the price between two different markets in the same period, the price relative would show the difference between the prices in the two markets in the same period. <back
4 The use of the geometric mean of the Laspeyres and Paasche indexes was first proposed by Pigou in 1920, and given the title "ideal" by Fisher (1922). <back
5 For further discussion of forward and backward Laspeyres and Paasche price and quantity indexes, refer to Chapter 2 of Allen (1975). <back
6 Economists measure the change in the quantity of an item in response to a change in price (or income) by elasticities, which are measured as the ratio of the percentage change in the quantity to the percentage change in price (or income). An item is price inelastic if the percentage change in the quantity is less than the percentage change in price. It has unit elasticity if the percentage changes are the same, and is price elastic if the percentage change in the quantity is greater than the percentage change in price. If an item is price inelastic, the change in expenditure will be in the same direction as the change in price (i.e. if price increases, then expenditure also increases). If the item has unit elasticity, then expenditure is unchanged. If the item is price elastic, the change in expenditure will be in the opposite direction to the price change (i.e. if price increases, then expenditure decreases). <back
7 The relationship between the Laspeyres and Paasche indexes holds while there is a normal relationship (negative correlation) between prices and quantities; that is, quantity declines if price increases between the two periods, and vice versa. <back
8 The implicit weights applied by the three formulas are equal base–period quantities (RAP), equal base–period expenditures (quantities inversely proportional to base–period prices) (APR) and equal expenditure shares in both periods (GM). <back
9 The geometric mean of n numbers is the nth root of the product of the numbers. For example, the geometric mean of 4 and 9 is 6 (= Equation: Example showing the geometric mean of n numbers is the nth root of the product of the numbers. For example, the geometric mean of 4 and 9 is 6, which is the square root of 4 multiplied by 9.), but the arithmetic mean is 6.5 ( = (4+9)/2 ). <back
10 For a mathematical proof of this see Diewert (1995). The unweighted indexes will all produce the same result if all prices move in the same proportion (have the same relative). In addition, the RAP and APR will produce the same index number if all base–period prices are equal. Diewert also refers to other studies that compare real world results for elementary aggregate formulas. <back
11 For example, Woolford (1994) calculated these indexes for twenty three fresh fruit and vegetable elementary aggregates of the Australian CPI over the period June 1993 to June 1994. He found that the GM produced the lowest increase in sixteen of the twenty three elementary aggregates, and the APR produced the highest increase for nineteen of the elementary aggregates. The RAP formula produced the middle estimate for thirteen of the elementary aggregates. Combining the elementary aggregates to produce the fresh fruit and vegetables index, the index compiled using the APR estimates was 4.7 per cent higher than the index based on GM estimates, and the RAP was 1.7 per cent higher than the index based on GM. <back
12 See Diewert (1995) for further discussion of unit values. <back
13 In a volume index, prices are held constant between the two periods, and the actual quantities from each period are used in the calculation. The change in the index is then measuring the weighted change in the volume of purchases, expenditure etc. <back
14 The characteristics approach to goods is the basis of the so–called household production theory. The development of this theory is generally attributed to Lancaster (1966), Muth (1966) and Becker (1965). Bresnahan and Gordon (1998) also provide a good example using household lighting, tracing the development from whale–oil lamps through to the electric light–bulb, pointing out how the additional inputs required on the part of households (such as trimming wicks etc.) were an important part of the production of light. <back
15 This point, and the use of characteristics in compiling consumer and producer price indexes, are explained in Triplett (1983). <back
16 Pollak (1983) identifies two characteristics approaches, that of Lancaster (1966) and Houthakker (1952). The Lancaster approach assumes that characteristics are additive across items (e.g. protein from meat can be added to protein from bread) whereas the Houthakker approach assumes characteristics are commodity specific. <back
17 There are many examples in literature of the application of the hedonic technique; for example, Ohta and Griliches (1975). For an overview of household production theory and the hedonic technique, see Muellbauer (1974). Pollak (1983) provides an exposition on the treatment of quality in a cost–of–living index. <back
18 For example, the hedonic technique is now used for estimating pure price change for personal computers and television sets in the United States CPI, and personal computers in Australia. <back
19 It is a moot point whether the increased speed and power of computers is reflected in corresponding increases in consumer utility, which raises questions as to whether the hedonic approach adequately captures quality change from a consumer perspective. However, studies have shown remarkable similarities in price indexes based on a hedonics approach and those for computers based on a comprehensive matched models approach. <back
20 Current thinking as presented in Koskimaki and Vartia (2001) for example is that hedonic equations should have log price as the dependent variable and should be estimated for each period. The use of weighted regressions is also supported by researchers such as Diewert. <back
21 This is often referred to as the Boskin Report, see Boskin (1996). Boskin estimated that the United States CPI was biased upwards by about 1.1 percentage points a year. There were many submissions and views expressed about bias in the US CPI. For a semi–official perspective on the issue see Moulton (1996). <back
22 As noted earlier, the issue of frequency of re–weighting or chaining is not straightforward. In a situation of price bouncing, chaining can introduce substantial bias into indexes (see for example Szulc (1983)). In general, chaining more frequently than annually, even if feasible in practice, is not recommended because it could introduce bias. <back