TECHNICAL NOTE 2: METHODOLOGICAL REVIEW
INTRODUCTION
A review of the methodology used for the Survey of Motor Vehicle Use (SMVU) was undertaken in 2002 to address data quality issues raised in relation to previously published data. This review identified deficiencies in the SMVU population frame which resulted in the selection of a sample that was not representative of the registered vehicle population. This deficiency has been rectified for the selection of the sample for SMVU 2003.
The review identified some minor errors in the adjustments used to account for re-registration. These errors have now been rectified. The review also investigated new vehicle provision calculations. While no errors were identified a number of options to improve these calculations were investigated and implemented.
This Technical Note specifically outlines the investigations that led to the identification of the frame deficiency and the post-stratification technique used to correct it.
FRAME PROBLEM
To ensure the SMVU sample was representative of the population, random selection was used within each stratum. For the SMVU, the random selection process allocated a random number to each unit on the frame. To select the sample, the frame was sorted by random number and a start point was randomly selected. A number of units were selected in order, depending on the number of selection units required for a particular stratum.
An investigation of the SMVU frame revealed a large number of units which had duplicate random numbers. Duplicate random numbers will not produce a bias in a sample as long as the duplicates contain a random assortment of units.
The SMVU frame investigation showed however, that the characteristics of certain variables differed between those units with unique random numbers and those with duplicated random numbers. This was particularly the case with Year of Manufacture. Therefore, the distribution of these variables within the resulting sample was dependant on whether the random start and the units selected incorporated duplicate random numbers. All estimates produced from samples selected under this scenario would contain bias, with the direction of this bias dependant on the inclusion of duplicates.
POST-STRATIFICATION
The collection of SMVU data from 1998 to 2002 had already been completed before the concerns with the frame were identified. To correct for the unrepresentative sample, a process of post-stratification was used.
Post-stratification is a method of stratifying a sample after the responses have been received. It is used to improve the quality of results through stratifying by variables that were not used at the time of sample design.
In the case of SMVU the frame investigation identified six variables to be used in the post-stratification. These variables were State, Vehicle Type, Year of Manufacture, Body Code, Fuel Class and Number of Cylinders. Once post-stratification was applied to the SMVU data, the weights of each unit were adjusted based on the particular post-stratum of that unit to realign sample totals to be representative of population totals. Each of the years from 1998 to 2002 were post-stratified independently and the post-strata will vary over time.
Caution needs to be taken in making comparisons between 1998, 1999, 2000, 2001 and 2002 SMVU data. Comparisons at the broad level are more reliable than those at the detailed level.
IMPACT
The impact of the review on the estimates for the main data items is summarised in the following table. The size of the change varies by data item. The table includes information on 1998, 1999, 2000 and 2001 data only, as 2002 data were produced after the review.
IMPACT OF METHODOLOGICAL REVIEW ON SMVU DATA, Australia
 | | Before review | After review | % change |
|
| 1998 | | | |
| Total kilometres travelled (million) | 173,317 | 167,892 | -3.13 |
| Total tonne-kilometres travelled (million) | 112,832 | 116,147 | 2.94 |
| Total fuel (million litres) | 23,909 | 23,258 | -2.72 |
| | | | |
| 1999 | | | |
| Total kilometres travelled (million) | 177,635 | 173,053 | -2.58 |
| Total tonne-kilometres travelled (million) | 127,311 | 129,874 | 2.01 |
| Total fuel (million litres) | 24,532 | 24,038 | -2.01 |
| | | | |
| 2000 | | | |
| Total kilometres travelled (million) | 180,782 | 184,593 | 2.11 |
| Total tonne-kilometres travelled (million) | 128,702 | 134,378 | 4.41 |
| Total fuel (million litres) | 24,926 | 25,853 | 3.72 |
| | | | |
| 2001 | | | |
| Total kilometres travelled (million) | 187,819 | 190,152 | 1.24 |
| Total tonne-kilometres travelled (million) | 132,756 | 132,422 | -0.25 |
| Total fuel (million litres) | 25,931 | 25,948 | 0.07 |
It is important to understand that the percentage change before and after the review can vary significantly between state and vehicle type. Therefore, the percentage change figures in the above table for all vehicles at the national level cannot be used at the state or vehicle type level to calculate the changes due to post-stratification.
For 1998, 1999, 2000, 2001 and 2002 SMVU data care should be taken in drawing inferences from changes in data over these five years as movements may be subject to high relative standard errors. Therefore the resulting estimates of movements may not be considered statistically significant. There is also potential for increased volatility in the estimates due to the changes that have been implemented as a result of the methodological review.
Users should contact the ABS if they have any queries on the methodological review.
Back to Main Features
Print Page
Print All