9208.0.55.003 - Information Paper: Survey of Motor Vehicle Use Fitness for Purpose Review, Jan 2005  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 18/02/2005   
   Page tools: Print Print Page Print all pages in this productPrint All  
Contents >> 2.0 Data issues this review will address

2.0 Data issues this review will address

This section outlines the issues that this review will seek to address. These are the issues relevant to the SMVU and determined to be significant following the stakeholder consultation. Each data issue includes a brief discussion on recommended investigations to be undertaken in phase 2. Please note that these investigations may be modified when the review team begins preparing for phase 2.

Please note that these are not the final recommendations of the SMVU review. Final recommendations will be developed from the results of the investigations and presented to the SMVU Project board for consideration.

The issues to be addressed fall into three broad categories:

1. Issues relating to what the SMVU is measuring, eg. vehicle kms travelled (VKT) and tonnes kms travelled (TKM).

2. Issues relating to how the data is dissagreggated, eg. the vehicle type classification used and the level of geographic dissaggregation available.

3. Issues associated with the quality of the published survey estimates, eg. sampling error and non-sampling error associated with the estimates.


Each issue contains the following information:

A. Description of the data issue

B. Links to policy and decision making

C. Suitability of current SMVU data

D. Recommendations


2.1 Authoritative data set

Description of the data issue
This data issue details the key data needed by users and their requirement for 'reliable' data, ie. data consistent with other measures of transport activity in Australia. Reliability is intrinsically linked with the sampling error of the survey estimates. This is also covered more specifically in section 2.2 'Data volatility'.

Feedback from the July 2003 TSUG meeting and the SMVU review consultation process identified that users require reliable estimates by vehicle type, at the national, state and area of operation level, for the following key estimates:

A. Kilometres travelled.

B. Tonne kilometres travelled, by broad commodity type, by road freight vehicles.

C. Fuel consumption rates (L/100km).

Tonnes kilometres data by commodity and by area of operation is not currently available from the SMVU. Data on commodity type and area of operation information are collected separately and a sound methodology for publishing a combination of the two is yet to be established.

Users have concerns about the lack of 'reliability' of key SMVU estimates at sub-national levels of aggregation. The consultation process confirmed that reliability is defined here in terms of being consistent with other available information relating to transport activity during the same reference period. For example, some SMVU results are not coherent with other known assessments of the total transport task. This is compounded by the fact that the ABS does not acknowledge or comment on such data disparities other than providing sampling errors. Despite a detailed methodological review confirming the validity of the SMVU data, some users remain unconvinced that the disparity can be fully explained by sampling variability.

Users would like movement in the estimates to be correlated with movements in real world transport activity. In general, users desire to keep the SMVU as it is with improved confidence in data reliability, to ensure time series consistency.

Links to policy and decision making
The four main uses of SMVU data are described below:

  1. Primary source for measurement of state/territory specific Key Performance Indicators (KPIs) such as vehicle accident rates per 100,000 vehicle kms travelled (VKTs) and estimates of emissions by vehicle/fuel type. The policy and decision making implications of SMVU data are quite different depending on the vehicle type. Data is used for the purposes of infrastructure planning, reporting, determination of funding and application of road use charges. Comparisons between jurisdictions are important. The National Transport Commission (NTC) use the data for allocating road costs among vehicle classes and determining heavy vehicle charges. Users want to be able to identify changes in these KPIs by state/territory which are due to actual changes in road transport use and not due to sampling variability. This is a particular concern for the smaller jurisdictions.
  2. As a secondary source for infrastructure and planning purposes. SMVU data is often used as a useful check on results from state/territory specific transport studies. These studies are used for infrastructure planning purposes and their annual movements are validated with the use of SMVU data.
  3. Input into modelling work by users such as the Bureau of Transport and Regional Economics (BTRE) and the Australian Transport Energy Data and Analysis Centre (ATEDAC).
  4. To aid understanding of the nature of road transport activity via trend analysis of key data items. For example, data on heavy vehicles is important for understanding heavy vehicle activity and to identify areas of high concentration of heavy vehicle activity.


Suitability of current SMVU data
Most users are content with the current data items available and the level of dissaggregation of SMVU data. The type of additional data that would add value to the SMVU as an authoritative dataset and allow for enhanced decision making is:
  • travel distribution data on average trip lengths
  • more demographic detail and purpose of journey
  • origin destination data
  • vehicle occupancy (not necessarily required on an annual basis, 3-5 yearly would be sufficient)
  • delineation of bus activity into metropolitan transit and interstate coach
  • interstate/inter-capital freight
  • freight vehicle loads including data on load size/dimensions, extent of vehicle utilisation and unladen (tare) mass

Regarding the 'reliability' of the SMVU data, as noted in the description of this issue, there are concerns about some disparities between SMVU data and some other industry transport measures.


Recommendations
Regarding the 'reliability' of the SMVU data:

  • Consider options for making available some data confrontation analyses where the SMVU data is compared over time and with other sources.
  • Joint ABS/BTRE analytical project investigating disparities between SMVU data and industry measures using SMVU unit record data.
  • Investigate options for reducing the sampling error associated with SMVU estimates. See the data volatility section for more details.

Regarding the additional data items requested:
  • Investigate the feasibility of providing data of sufficient quality (including question design and testing). Frequency and accuracy requirements will also need to be assessed.


2.2 Data volatility

Description of the data issue
During the phase 1 stakeholder consultation process, some users expressed dissatisfaction with the current sample size for SMVU - approximately 16,400 vehicles in the 2003 survey. For example, the stability of the VKT time series has been criticised by these users, who highlight the correlation between increased variability (measured by the estimated standard errors) and smaller sample sizes. Users would accept some of the SMVU data items to be published less frequently and with data items reduced to a core set, if necessary to achieve significantly improved accuracy.

The opinions of key external stakeholders were sought on aspects of data frequency and accuracy, the objective being improvement of overall data quality. It should be noted that methodological changes aimed at reducing 'recall bias' have arguably reduced the non-sampling error associated with these estimates. Moreover an internal methodological review in 2002 identified some erroneous aspects of the SMVU survey frame and new vehicle provision methodology, both of which have now been rectified. A methodological review into optimal survey stratification was conducted in 2003 and the ABS anticipates some minor quality benefits to be identifiable with the release of the 2004 SMVU.

Users would find useful a technical paper, discussing the time series and summarising the changes that have occurred over time, particularly from the period 1995 onwards. Stakeholders did point out that such a paper would not be the panacea for their data quality concerns, increased sample size producing lower Relative Standard Errors (RSEs) was considered paramount. Stakeholders did acknowledge that any potential sample increase may be at the expense of data frequency (see discussion in section 2.5 'Data frequency').

Suitability of current SMVU data
Stakeholders have some useability issues on the key SMVU estimates at all levels of dissaggregation. Some users (smaller jurisdictions) stated that they wanted to see standard error consistency across the states/territories. The consistent message relating to RSE requirements on level estimates are as follows:
  • Current RSEs are too high for VKT (by vehicle type) at the state/territory level (ideally these would be in the range 2% - 4%).
  • Current RSEs are too high for TKM (by vehicle type) at the state/territory level (ideally these would be in the range 2% - 4%).
  • Data with RSEs in excess of 20% are of limited usefulness.
  • RSEs for the greater metropolitan regions preferably not greater than 15%.


Links to policy and decision making
The main use of SMVU key aggregates are for modelling and validation purposes. See also the discussion on Links to policy and decision making in section 2.3 'Measuring level movements'.

Recommendations
  • Conduct investigations to identify costings and data quality ramifications associated with adjusting the frequency of some SMVU data.
Stakeholders are aware that the ABS is not in a position to allocate increased resources toward improving the level estimate RSEs. Given this resource constraint, a number of key stakeholders acknowledged that the desired RSE improvements could only be achieved at the expense of data frequency. The majority of stakeholders accept the trade-off, in favour of increased accuracy with biennial or triennial data frequency. The review team recommends conducting investigations to identify costings and data quality ramifications associated with producing some SMVU data on a biennial/triennial basis (the results of these investigations will link into those proposed investigations within the recommendations discussion in section 2.5 'Data Frequency'). This investigation would need to undertake some time series analysis to identify which data items are relatively stable and are therefore sound candidates for less frequent publication.

  • Develop a strategy to improve the quality/consistency of the survey frame information supplied by the individual State and Territory MVRs. This will be a long-term project.
The inclusion of odometer readings on the administrative data sets provided to the ABS would improve the accuracy of SMVU estimates by providing a data source with which individual survey data can be validated. The ABS also has a strong desire to improve the quality and consistency of other survey frame information. Stakeholders acknowledge the link between SMVU data quality and the quality/consistency of the survey frame information coming from the individual State and Territory MVRs. Stakeholders endorse the idea of establishing a strategy to improve this quality. Such a strategy would need to have the backing of state/territory transport departments and a national organisation such as AustRoads. The strategy would focus on working with State and Territory MVRs to identify the barriers of improving the quality and consistency of survey information and developing solutions. An element of this work may involve joint submissions for funding proposals aimed at developing the following:
  1. Common classifications. The aim here would be to reduce the time lag between the creation of the survey frames and the reference period, which will improve the accuracy of SMVU data.
  2. Frame maintenance/coding. Aim as above.
  3. Odometer readings. The objective being to provide enhanced survey data validation procedures which would improve non-sampling error data quality and may enable a reduction in the time lag between reference period and publication.
  4. Australian Business Number (ABN). The aim here is to improve stratification of the SMVU, thus improving accuracy. A beneficial flow-on effect would be enhanced ability to produce industry estimates, if or when this is considered to be of sufficient importance to users.

  • Quantify the impact of unregistered vehicles (see discussion below in section 2.7 'Impact of unregistered vehicles').

ABS to investigate publishing a technical paper which discusses the time series aspects of the SMVU and the changes that have occurred over time, particularly from the 1995 survey onwards. These include changes to the survey methodology, sample size, systems, processes and questions.


2.3 Measuring level movements in key variables

Description of the data issue
The SMVU sample allocation is currently designed to minimise both provider load and the standard error of the level estimates of key variables. TSUG members have commented that users require an emphasis on the measurement of annual changes for the key variable of vehicle kilometres travelled by broad vehicle type (passenger vehicles, rigid trucks, articulated trucks, etc.) and geographic region (preferably national, state and capital city region). Acknowledging the resource constraint faced by the ABS, key stakeholders generally stated a preference for enhanced accuracy of level estimates over enhanced movement accuracy. Some of the smaller jurisdictions expressed a preference for controlling movement estimates due to the concern that movements between years may be due to sampling variability rather than real changes (see discussion in section 2.2 'Data volatility').

The current SMVU uses a non-overlapping sample. An overlapping sample is generally required to minimise the standard errors associated with movements in the estimates between years. It may be possible to introduce a level of overlap between SMVU samples to improve the accuracy of estimated movements between years.

Users indicated that movement controlled estimates would not be used any differently to the way in which the current SMVU estimates are used.

Links to policy and decision making
Stakeholders expressed a preference for accurate SMVU level estimates where the data is used as input for transport modelling purposes. In instances where the SMVU is used for validation purposes (state/territory specific pre-existing models or annual transport surveys) users have expressed a desire for RSE controlled movement estimates.

The NTC uses SMVU data to identify variations between vehicle and geographic usage associated with different parts of the road network. For example, it is important to identify changes in the proportion of road use at the state/territory level and also changes in the proportion of road use between the sub-state urban/rural areas of operation. Changes in the proportion of road use between different vehicle types are important for assessing regulatory initiatives. These data needs are best achieved via enhancements to level estimate data quality.

Suitability of current SMVU data
Some stakeholders have useability issues relating to the movement of some key SMVU estimates. Users generally have a preference for enhanced accuracy of level estimates.


Recommendations

  • Investigate the feasibility of increasing the sample size of non-overlapping samples to improve the accuracy of level estimates and therefore the accuracy of the associated movement between years. Also consider the option of introducing a level of overlap in the sample.
  • Focusing investigative work toward enhancing the accuracy of level estimates. Initial investigations have shown that increasing the sample size of non-overlapping samples may be the best option to improve the accuracy of level estimates and therefore the accuracy of the associated movement between years. Preliminary investigations by the ABS Methodology Division show that this option produces better accuracy, for both time periods, compared to an overlapping sample. However, some further investigation of overlapping samples could be conducted.

This option will require detailed investigation to determine feasible options on what might be the best way to reduce the RSEs of movements between years RSEs. Selecting the same respondents in consecutive years can affect the response rate, which in turn affects the data accuracy, and would also need to be considered.


2.4 Geographic patterns of activity

Description of the data issue
In modelling transport activity and related impacts, users frequently require data at a level of detail below the national and state/territory level. In particular, data at the level of capital city and surrounding population catchment would be valuable. Such data would require the use of geographic area classifications in the SMVU. Data at the Statistical Division level would be of use to users, but is lower priority for inclusion in nationally collected data sets. If modelling is to be used to impute estimates for finer level geographic areas, users would need some raw data upon which to base these finer level estimates.

For many stakeholders sub-state level data is more important than state/territory level data. Sub-state level data is generally used for transport network and environmental modelling, particularly metropolitan versus rest of state/territory data. For example, the majority of the growth in road transport is occurring in the metropolitan areas. Given policy and community interest in issues such as fuel-based pollutant emissions, traffic congestion, urban sprawl, land use patterns, and other population based traffic issues, metropolitan/urban level data is of greater importance than regional data. In addition, fine level regional data on some aspects of vehicle use can be obtained from some state/territory road authority traffic count data.

TSUG have acknowledged that the primary role of the ABS is to provide stable national and state/territory SMVU data. TSUG have also acknowledged that it is not the role of the ABS to provide intra-regional transport data or transport corridor data. The ABS may have a role in assisting the modelling work for some sub-state data by providing some limited regional data. The ABS is not currently able to produce regional specific data due to resource constraints, however, investigations into improving the data quality of 'metropolitan', 'urban' and 'rest of state' areas of operation will be undertaken (see later in this section under 'Recommendations').

Links to policy and decision making
The preference for enhanced data quality at the state/territory or sub-state level does appear to depend upon the nature of the stakeholder. The majority of stakeholders who provided feedback to the SMVU review, revealed a preference for enhanced data quality at the sub-state level (particularly the greater metropolitan regions). These users have a focus on infrastructure planning and modelling type work and have a preference for sub-state (particularly metropolitan) data. This is especially the case for the larger jurisdictions which are subject to higher traffic congestion in the greater metropolitan regions.

For other users, a state/territory dimension of transport activity is more important than sub-state data. These users have a focus on Key Performance Indicators (KPIs) at the state/territory level. Accurate (consistent) state/territory level data is important to these users, since funding for road repair/construction is determined on the basis of state/territory aggregates. Charges upon heavy vehicle use and vehicle registrations are dependant upon this state/territory specific information. It is important for these users that funding and charges are subject to changes in performance and activity rather than being subject to sampling variability.

Suitability of current SMVU data
The current SMVU generally has the required data items to fulfil the spectrum of user requirements for the given geographic dissaggregation. Some minor exceptions relate to requests for expanded freight commodity detail and data relating to passenger kilometres travelled. Users also require enhanced data accuracy, generally at the greater metropolitan regions. Users have the expectation that greater data accuracy at the Greater Metropolitan Region (GMR) would correlate to increased accuracy at the state/territory level.

The required accuracy for the geographic delineation can be found in section 2.2 'Data volatility'. In some cases State and Territory specific stakeholders have user specific definitions of an optimal GMR, however, the current SMVU geographical definitions are meeting user needs. The definition of GMR, or any decision of which particular sub-state region to collect, should be based on population density.

Recommendations
  • ABS to investigate options for improving data accuracy at sub-state geographic areas important to users, particularly metropolitan and rest of state/territory areas of operation. This will involve considering stratification options and sample size options (also provider load and ABS survey processing costs) as well as the outcomes from the investigation into data frequency.

ABS to investigate sample size ramifications for improving data accuracy at the sub-state level (the results of these investigations will link into those proposed investigations within the recommendations discussion in section 2.5 'Data frequency'). The ramifications for provider load and ABS survey processing costs will need to be assessed. Any final decision on sample size will also be considered in light of the investigation into data frequency (as mentioned in section 2.5 'Data frequency'. It may also be possible to add a sub-state dimension to the survey stratification (using the metropolitan - rest of state breakdown of interest to users) to improve sub-state estimates. Note that it may be difficult to meet the user requirements for the specified RSEs for the given dissaggregations outlined in section 2.2 'Data volatility'.

Note that the data volatility issue recommendations will address possible improvements to the survey data at the state/territory level. Significant improvements to measured standard errors at the sub-state level may be difficult to achieve given the SMVU budget constraints.


2.5 Data frequency

Description of the data issue
Data frequency is discussed here in terms of modifying the time lapse between the collection of data for specific data items, in order to re-direct resources toward other survey improvements.

External users want datasets to be published at regular intervals, with a focus on supporting time series analysis and the ability to compare with other available data. It is important that data series be up-to-date, so that policy work can be based on current information. It might be cost effective to create a 'data gap' by collecting some data items less frequently, but with a larger sample size when it is collected, allowing for increased accuracy. Some users need annual data, therefore it may be prudent to collect selected data items annually at national and state/territory levels, (eg vehicle kilometres travelled, fuel use), and to collect data on other items less frequently (eg data breakdown by vehicle type or freight data by commodity carried). Analysts may then use the annually supplied data to model annual estimates for data that is available less frequently. This approach would require a transparent schedule for availability of the data gaps. TSUG participants recognise that there is high priority in having data that is accurate and reliable - even if cost considerations mean that the frequency of published data must be reduced.

Key stakeholders were provided with the opportunity to state their preferences in terms of data item frequency. Most stakeholders indicated that their preference was for key SMVU estimates to be published annually and with increased accuracy. All stakeholders acknowledged the budget constraints imposed on the SMVU and the consequent trade-off between increased accuracy and reduced data frequency. Some stakeholders are encouraging the ABS to choose the trade-off in favour of increased accuracy. Others are generally supportive of this trade-off, if the review investigations are able to demonstrate that there would be significant improvements to accuracy.

Based on stability of the time series and user data requirements, certain data items are more suitable candidates to be published less frequently than others. Average fuel consumption rates and passenger vehicle data are a possibility for publication less frequently, but with greater accuracy. The trade-off for freight vehicles may be more problematic due to the desire to capture the changes in transport activity associated with changes in overall economic activity.

The ABS will investigate the time series stability of specific data items and the likely user impacts, prior to identifying potential data items for this trade-off. The ABS will not consider altering the frequency of any key data items without being able to demonstrate a significant improvement in accuracy. Biennial data can be incorporated within transport models for data items that are generally relatively stable over a two year period.

Links to policy and decision making
As discussed previously in section 2.1, some users stated a preference for annual data, due to reasons of 'policy sensitivity' and to capture structural changes that are occurring in the transport industry (such as supply chain management). For some stakeholders this policy and industry structure focus is based on regional areas and for others the policy emphasis is on capturing changes at the GMR.

Suitability of current SMVU data
The consistent message from stakeholders is the current SMVU is producing the right data items, frequency and level of detail, but enhanced accuracy is required. As stated above, stakeholders are prepared to trade-off significant improvements in accuracy for less frequent data availability for certain data items.

Recommendations
  • ABS methodology unit will investigate the trade-off between increased data item accuracy with reduced data availability.
Examples of frequency scenarios that can be tested and compared for the optimal outcome are provided below:

Scenario 1. No change to the SMVU sample size or frequency of data availability.

Scenario 2. Maintaining current sample size (over a two year period approximately 34,000) devoting the entire 17,000 sample to freight vehicles in the first year and the other 17,000 to passenger and other vehicles in the second year. By focusing the sample increase (for the vehicle type) at the sub-state level, the intention is to produce estimates with improved accuracy, stability and reliability at the state/territory and the sub-state level.

Scenario 3. Combination of scenarios one and two above. Maintaining current sample size of approximately 34,000 over a two year period, devoting the majority of the sample to freight vehicles in year one (say 12,000 vehicles) to produce enhanced accuracy at all levels of aggregation. The remaining 5,000 sample units could be devoted to providing an accurate national estimate only for passenger and other vehicles during their 'off' year. Passenger and other vehicles would be sampled in year two (with a sample size of say 12,000 vehicles), producing enhanced accuracy at all levels of aggregation. The remaining 5,000 sample units could be devoted to providing an accurate national estimate only for freight vehicles during their 'off' year.

Other potential solutions will also be investigated, ranging from annual to triennial data frequency options.

2.6 Quality of fuel use estimates

Description of the data issue
SMVU fuel use estimates are used in a number of different applications, and have important policy and regulatory applications. In order to forecast income from items such as fuel taxes, it is important that Commonwealth and State governments have data describing characteristics of the fuel use to which taxes and charges are applied. The National Transport Commission (NTC) sets heavy vehicle registration and fuel-based road use charges to cover the costs of road provision and maintenance attributed to heavy vehicles. Greenhouse gas emission estimates are modelled on the basis of fuel use estimates. These uses represent a substantial ongoing need for SMVU fuel use estimates.

The ABS is aware of the criticism that differences exist between SMVU fuel use figures and those obtained from total fuel sales data sources.

A number of key stakeholders don't use SMVU fuel use estimates. Of those stakeholders who do use fuel use estimates, there was general consensus that the average fuel consumption rates by vehicle type are accurate and reliable. These stakeholders expressed concern, however, that the total SMVU fuel use estimates are consistently lower than other data sources such as total fuel sales data. Total fuel sales data is available and the data is considered accurate by BTRE, yet it cannot be delineated by vehicle type and vehicle age. Users are of the opinion that there is a link between the SMVU underestimate of total fuel use and the underestimate of aggregate SMVU vehicle kilometres travelled. Despite this concern, users are reluctant to see modification or abolition of SMVU fuel use estimates, since users have confidence in the average fuel consumption estimates and the SMVU provides the only national split by vehicle type and age.

The consultative feedback process did not reveal a common stakeholder position on the progression of SMVU fuel use data quality, other than the desire for the SMVU to continue to collect this data item. The following options were discussed during the consultation phase:

  • Drop SMVU fuel use questions from the survey form, not publishing such data.
  • Tailor the SMVU sample design to provide accurate fuel use proportions which users can then apply to the administrative total fuel sales data. A solution based on this idea may involve supplementary fuel use questions for a sub-sample of the total SMVU sample.
  • Improve the total SMVU fuel use estimates by improving the accuracy of vehicle VKT.
  • Develop a different strategy for collecting and compiling fuel use data. The objective would be to improve the non-sampling error associated with the fuel use questions.
  • Determine the fuel use impact of unregistered and out of scope vehicles. This could be published as an SMVU feature article, allowing users to adjust SMVU fuel use data according to their specific needs.

It does appear possible that the data need for fuel use estimates may be met by non SMVU sources. Total fuel sales data, producing accurate totals by fuel type, has already been mentioned above. The Australian Greenhouse Office (AGO) provides fuel consumption rate information for new vehicles which are obtained from the vehicle manufacturer in accordance with Australian Standards testing. Older vehicles are not covered by the AGO data, yet it would be feasible to produce a model which tracks changes in the consumption rate with vehicle age, this data may already be available. It is possible that SMVU fuel use data is used as an input to these other alternative fuel use sources, this will need to be investigated. Vehicle counts by vehicle age and type are available from the Motor Vehicle Census. Therefore it appears highly feasible that total fuel use estimates by vehicle type and age can be determined by non SMVU data. This requires further investigation.

In addition to fuel use data requirements already discussed, the NTC requires data on where fuel is consumed. It does appear that the SMVU is the only source of such data. The relationship between where fuel is sold and where it is consumed would need to be investigated to identify the critical nature of this data gap, should this review recommend not continuing with fuel use estimates. The other aspect which will need to be considered is the difference, if any, between fuel consumption rates as determined in testing conditions and fuel consumption incorporating different driver behaviour, traffic conditions etc.

Links to policy and decision making
As above.

Suitability of current SMVU data
Stakeholders have expressed concern that the SMVU fuel use estimates are consistently lower than other data sources such as total fuel sales data (in the order of 10%-15%).

Recommendations
  • Investigate the underlying reasons behind the disparity between SMVU fuel use estimates and data on total fuel sales, consider the differences between these two measures.

This investigation is closely related to the 'out of scope' and unregistered vehicle use investigation (see section 2.7 'Impact of unregistered vehicles and out of scope vehicles').

  • Investigate the potential of producing accurate vehicle type/age delineation of total fuel sales without using SMVU data.

The results of this investigation could establish any of the following:
  1. That the SMVU is the only accurate data source of fuel use by vehicle type and the ABS should continue producing these estimates.
  2. The formation of a justification for not producing SMVU fuel use estimates.
  3. Form a proposal for obtaining industry funding.

Any identified provider load and survey processing savings could then be re-allocated to improving other key SMVU estimates.

  • Develop a new strategy/enhanced questioning, with the aim of reducing non-sampling error when producing SMVU fuel use estimates. Should the project board choose to continue producing SMVU fuel use estimates, work on the development of a new strategy/enhanced questioning, with the aim of reducing non-sampling error.


2.7 Impact of unregistered vehicles and out of scope vehicles

Description of the data issue
Users have raised concerns that unregistered vehicles contribute to total motor vehicle use in Australia, but that these vehicles are not covered by the SMVU as they are not available for selection in the frame. The extent of unregistered vehicle use is unclear. Evidence from Queensland has estimated unregistered vehicles to be around 4.5 per cent of vehicles on the road, whereas in New South Wales there is evidence that unregistered vehicles are around 2 per cent of vehicles on the road. The extent of legal use of unregistered vehicles by industry is also not known, and variability in use of such vehicles is also unclear.

The questions that arise in the context of the SMVU are:
  1. Does unregistered vehicle use explain some of the discrepancy between the SMVU estimates of fuel consumption and total fuel sales?
  2. What is the implication for the SMVU estimates of total VKT?

To answer these questions it is necessary to understand whether the proportion of unregistered to registered vehicles is the same across all state/territories and stable over time. An investigation into the number and use of unregistered vehicles is recommended.

The evidence gathered on unregistered vehicle use during the consultative phase of this review was largely anecdotal. It is likely that some raw data exists within State and Territory MVRs, Police departments, transport portfolios and road agencies. A number of partial investigations have also been conducted in recent years which are vehicle or industry specific, however, this work is yet to be brought together in a complete investigation of unregistered vehicle use.

The BTRE estimates VKT by long-term unregistered vehicles to be around 2-3 per cent of national VKT, (and that the actual value would be unlikely to fall outside the range 1 to 5 per cent of national VKT). The proportion of cars on the road that are technically unregistered would be higher (with a likely national average of 6-7 per cent), but many of these vehicles will only be 'late payers', and thus will still be within scope of the SMVU sample frame (unless they always pay late and so are not on the MVR records provided to the ABS). Given this partial analysis, it is unlikely that unregistered vehicles provide a complete explanation for some of the criticism that SMVU data has not matched industry expectations. However, users do want the ABS to attempt to quantify the problem rather than to assume the issue is insignificant.

With respect to out of scope vehicles, the Australian Bureau of Agricultural and Resource Economics (ABARE) have estimates of unregistered vehicle, fuel and machinery use by industry sector. The Department of Defence will also have estimates of unregistered military vehicle use.


Suitability of current SMVU data
Some stakeholders agreed that a systematic analysis aimed at identifying the significance and the degree of volatility of unregistered vehicle use is warranted. A complete investigation of the unregistered vehicle use would necessarily include a section on the level and variability of 'out of scope' vehicles.

Recommendations
  • Undertake a separate analysis to identify the characteristics of unregistered vehicle use (using data from police records, State and Territory MVR authorities, mining and agricultural industry groups).

Undertake a separate analysis to identify the characteristics of unregistered vehicle use (using data from police records, State and Territory MVR authorities, mining and agricultural industry groups), the results of which could be published as a 'feature article' within the SMVU and implemented by users independently of published data.


2.8 Vehicle classification used by the ABS

Description of the data issue
The vehicle classification used by the ABS for the SMVU has 27 vehicle and 8 trailer categories. This is aggregated into seven vehicle types in the published survey estimates. Vehicles are differentiated by chassis type and axle configuration. Vehicle type is also a stratification variable in the SMVU. The AustRoads vehicle classification, which was agreed to by all State and Territory road authorities following a review in the early 1990s, has 12 vehicle categories covering passenger vehicles, rigid trucks and articulated trucks. Vehicles are differentiated by the number of axles. The AustRoads classification was designed primarily to collect heavy vehicle road use data via weigh-in-motion axle detectors. The AustRoads vehicle classification doesn't have separate categories for motorcycles or buses and coaches, which are required for the SMVU. Users of transport data would prefer the two vehicle classifications be compatible to allow greater comparison and integration of different data sets produced by different Australian agencies, such as administrative data created by the State and Territory MVRs.

The majority of users consulted during phase 1 of the review use the AustRoads vehicle classification. The AustRoads classification is generally used for monitoring and charging purposes, whereas the ABS SMVU classification is used for more detailed modelling work. The NTC is one agency that has to use both classifications, relying on data from the SMVU to set charges for vehicle types defined within the AustRoads classification.

Stakeholders generally find the respective classifications satisfactory to their needs. The main requirement is for consistency between different classifications, thereby allowing disparate data sources to be used together. Stakeholders should note that the SMVU has a number of classification variables which may be available at finer levels of detail than can be found in the publication. Data based on these classification variables is available from the ABS via special request.

Some minor differences between the classifications are evident. For example, the AustRoads classification does not separate motorcycles from light vehicles or buses and coaches from trucks. The ABS classification classes rigid trucks with attached trailers to rigid truck categories, however, under the AustRoads classification rigid trucks with trailers are classified to AustRoads vehicle classes 6 to 9, which include 3-6 axle articulated trucks. The ABS classification allows the separate identification of passenger and load carrying four-wheel drives (4WD), although this data is not generally published in the SMVU. Some stakeholders use the AustRoads classification for screen line vehicle count data and the ABS classification for differentiating between rigid and articulated trucks.

It is highly desirable that the more detailed ABS classification can be collapsed into the AustRoads classification. This could require some adjustments to both the current ABS and AustRoads classifications. A concordance between the respective classifications would then be used by data users to combine data using the different classifications.

Users would like the ABS to consider including some of the following vehicle breakdowns in standard publications:
  • Some split of passenger vehicles by size.
  • Separate identification of taxis as they have different use patterns.
  • Some split of rigid/articulated trucks by usual trailer: no trailer, B-double.
  • Delineation of bus into mainly metropolitan transit use and mainly long-distance use.

Some users may find data on 4WD vehicles useful, but this was not identified as a major data need.

Links to policy and decision making
The classification is the fundamental basis upon which data from the SMVU is collected and disseminated. The links to policy therefore include every link to policy and planning referred to in this paper.

Suitability of current SMVU data
Some stakeholders have useability issues relating to the movement of some key SMVU estimates. Users generally have a preference for enhanced accuracy of level estimates.


Recommendations for investigation

  • Investigate aligning the ABS and AustRoads vehicle classifications more closely.

Begin concordance mapping work between the ABS vehicle classification and the AustRoads vehicle classification. The aim of this investigation should be to identify how closely the 35-bin ABS classification can aggregate to the AustRoads 12-bin classification and recommend changes that allow for complete integration between the two classifications. (Note that the ABS uses 35-bin classification, however, SMVU results are published at a more aggregated level, ie. 7 vehicle types are published.)


2.9 Question changes on SMVU survey forms

Description of the issue
Reducing the non-sampling error (and therefore increasing the accuracy) of the SMVU estimates may be achieved by improving the quality of particular questions on the survey form to more accurately collect the data required. The questions outlined below are known to be difficult for some users to answer.
  • The 'main type of journey' questions need to unambiguous to the provider and align with user requirements. It may be possible to improve the structure of these questions to better align with user requirements and reduce provider misunderstanding.
  • Providers appear to be led by the list of examples used to describe the 'Description of vehicle use' rather than use their own words when answering this question.
  • It may be beneficial to provide more specific wording on the 'trips per week' question.
  • It may be possible to improve/clarify the definition of the 'average load weight' question (ie. average per trip or average per kilometre).
  • The 'business use' question appears to be contradictory in places.
  • It may be possible to improve the fuel use questions, this is covered in section 2.6 'Quality of fuel use estimates'.


The review stakeholders did not identify any additional survey questions that this review should investigate improving.

Links to policy and decision making
The policy and decision making uses of the existing data items outlined above are discussed in section 2.1 'Authoritative data set'.


Suitability of current SMVU data
Although not all stakeholders use the ABS classification, it meets the general needs of those who do use it. Some minor enhancements would be beneficial to stakeholders.

Recommendations

  • Investigate the effectiveness of survey questions known to be problematic for users to answer, and identify possible enhancements where appropriate.

This may involve a type of 'post-enumeration survey' where a sample of data providers are contacted after responding and asked some detailed questions to assess their understanding of specific survey questions.

It should be noted that any changes to the survey questions has the potential to affect the continuity of the time series data. Continuity are important to SMVU data users and this will be taken into account when any decisions on improving the questions is made.



Previous PageNext Page