A Quarterly Information Bulletin from the Methodology Division
SYNTHETIC ESTIMATES FOR INPUT-OUTPUT
The Economic Activity Survey (EAS) collects a number of different categories of expenditure data (amongst other economic data) from businesses, one of which is the remainder category of other operating expenses. This is further disaggregated into approximately 25 subcategories in the EAS selected expense supplement form, which is sent out to a subsample of businesses from the main EAS sample. This second phase survey is also known as the Input-Output (I-O) Survey, as the data collected from it are used to compile I-O tables for the National Accounts.
In 1997-98 there were insufficient resources to collect I-O Survey data from all businesses, and the sample was restricted to around 1000 businesses that either had 200 or more employees or that were complex in structure and relatively large. The problem which faces us is this: how do we produce estimates for the remaining medium-sized and small businesses?
Up until now, the methodology has been to calculate the proportions of other operating expenses falling into each of the I-O subcategories from the most recent year where sample data is available (i.e. 1996-97 in this case), and apply them to the other operating expenses in the current year (i.e. 1997-98 in this example). An obvious shortcoming of this approach is that it does not allow for any changes in the pattern of proportions which may have occurred since the sample data used to calculate the proportions were collected.
We are currently working on a more sophisticated approach in which we model a logit function of the proportions, using data items collected in the main EAS sample as auxiliary (predictor) variables. In other words, we are imposing a multinomial logistic model and assuming that our I-O subcategories are conditionally independent given the EAS data items. Note however that our inferences based on our synthetic estimates do not depend on our model. An important requirement of the methodology we chose is that we be able to produce standard errors on the resulting estimates as an indicator of the sampling variability involved.
We have two main options about which data to use to produce synthetic estimates for 1997-98. We could use the data from the medium-sized and small businesses in the I-O sample in 1996-97 and then apply that relationship to the same sized businesses in 1997-98, thereby assuming that the relationship from 1996-97 continues to hold in 1997-98. Alternatively, we could assuming that the relationship between the I-O subcategories and the EAS auxiliary variables is the same for the large businesses from 1997-98 as it is for the medium-sized and small businesses in the same year. We are currently pursuing the latter approach, partly because of the relatively small sample in 1996-97 and partly because it will be easier to calculate standard errors. Our resulting estimates will be synthetic, made up of the sum of purely design-based estimates from the I-O sample of large businesses and synthetic estimates from the EAS sample of smaller businesses.
Our initial evaluation of this methodology using 1996-97 data shows mixed results. For some industries, our synthetic estimates accord well with design-based estimates; for other industries, less so. However, it is difficult to draw any firm conclusions, as the design-based estimates themselves are based on a relatively small sample and are subject to large standard errors.
Our timetable is to complete the synthetic estimates for 1997-98 by mid-March, and repeat the process for 1998-99 by early April. This will be followed by the calculation of standard errors.
In the future, we intend to return our attention to the method of obtaining synthetic estimates using data from a previous year. This will be more attractive in the future, as from 1998-99 on, the Input-Output Survey has followed a 'rolling industry' approach, whereby a sizeable sample across all business sizes is taken for a subset of industries in any particular year. In the next year, a different subset of industries is sampled in the same way, and so on. We may also look at whether we can improve our purely design-based estimates even where we have a sample across all sizes of businesses by using auxiliary information from EAS. Our estimation in this case would follow a two-phase logistic generalised regression technique.
For more information, please contact Paul Schubert on (02) 6252 5140.
FEASIBILITY OF DISPROPORTIONATE SAMPLE ALLOCATION IN THE MONTHLY POPULATION SURVEY
The ABS has recently investigated the feasibility of using disproportionate allocation in the redesign for the 2001 Monthly Population Survey (MPS) sample. This investigation follows on from a preliminary investigation into the implications of moving away from a constant sampling fraction within each state/territory (i.e. proportionate allocation).
The current sample design for the MPS adopts a constant sampling fraction within each state/territory. All dwellings in a state/territory are divided into strata which are formed by splitting states and territories into sample design regions and then into area types.
Within each stratum the sample is typically selected through a three stage process. Firstly, collection districts (CDs) are selected from all CDs in the stratum; secondly, blocks are formed for each CD selected in the first stage sample, and a block is selected for inclusion in the sample; and finally, all dwellings within the selected block are listed and a cluster of dwellings are selected from the block. The three stage selection process is designed such that all clusters within the state/territory have the same chance of being selected for the MPS sample.
Proportionate allocation has the advantage that it spreads the sample uniformly across different strata. This ensures that the sample size for each sample design region is roughly equal to a constant fraction of the entire population of the region. The distribution of the sample by area types also reflects that of the population within the state/territory.
"Disproportionate Allocation" involves, for example, taking less sample from the more expensive to enumerate areas, and to compensate, more sample from the cheaper to enumerate areas. Another driver for disproportionate allocation is to select more sample from those areas where labour force characteristics vary highly from one locality to another, and less from those areas which are more homogenous in terms of their labour force characteristics. The cost and variance structures of these areas within a state, and how they change under different sample design configurations, are obtained from the cost and variance models.
A number of improvements have been implemented for the 2001 disproportionate allocation investigation which make the results considerably more conclusive than those from the previous investigation. The most important improvements are firstly that the cost model is now based on comprehensive data which fully reflects costs under the telephone interview methodology, and secondly that the variance model is more robust being based on all possible samples under a large number (50) of different designs. In addition these models now cover all area types.
The recent investigation evaluated a number of approaches for undertaking disproportionate allocation. Based on new cost and variance models to be used in the forthcoming 2001 MPS redesign the study compared two disproportionate options with two proportionate options. All of these methods essentially choose design parameters that minimise national cost subject to satisfying both national and state accuracy constraints.
A paper is currently with ABS management for a decision as to whether to proceed with disproportionate allocation for the 2001 MPS redesign. This paper provides a broad assessment of the issues associated with the introduction of disproportionate allocation. While disproportionate allocation has in theory the potential to provide cost savings, there are a number of other factors that need to be considered, such as the implications for the quality of estimates for smaller states and territories, impacts upon the quality of regional labour force data, impacts upon other household surveys that use the MPS design framework, system costs of implementation and the loss of a robust sample design.
For further information, please contact Daniel Elazar on (02) 6252 6962.
QUALITY OF OUR ANALYTICAL PRODUCTS
Last year, the ABS expanded the resources it invests in analysis. One of the aims of the program is to develop new statistical products that have substantial "analytical content". A key issue is whether the prototype products that we develop are of high enough quality to serve users' needs and to become part of the ABS's ongoing product mix.
Our project teams are accumulating experience in defining, monitoring and assuring the quality of particular products (such as the new output measures for government, socioeconomic indexes, the tourism satellite account and estimates of household wealth). The time is ripe to distil that experience into a quality assurance (QA) tool kit that can be shared by all ABS staff who are developing analytical products and other elaborately transformed statistics.
In the next few months, we plan to define the quality dimensions most relevant to analytical products; this will draw heavily on John Zarb's project Quality Measures for Systems of Economic Accounts. Then we shall walk through a dozen or so analysis projects (past and present) to compile a suite of quality indicators. Then we shall develop a set of quality assurance guidelines; these will supplement the quality management module of the ABS Project Management Framework.
We hope that the tool kit will help analysis project teams address the following questions:
- What quality characteristics of the product we are developing will be most important to potential users?
- What practices will help us assure quality during product development?
- How shall we assess the quality of the product at the completion of our project?
- How can we make the quality characteristics of the product visible to users when we hand it over to them?
The first edition of the QA tool kit will be available by November 2001.
For more information, please contact Ken Tallis : (02) 6252 7290.
HEDONICS AND COMPUTER PRICES
The capabilities and effective prices of computers change rapidly. For example, according to a report by the US Bureau of Labour Statistics, the power of a typical desktop computer increased by about fifteen times between 1993 and 1998, but the market price remained almost unchanged. During the same period, many more features (such high resolution displays and onboard sound cards) were added. Price statisticians want to measure the changing prices of computers, taking such quality changes into account.
In late 2000, Economic Accounts Division and Analytical Services Branch (ASB) began a joint research project to develop a computer price index based on so-called "hedonic methods". The project had two aims: developing a hedonic function based on Australian data; and using that function to construct a price index.
Conventionally, price statisticians have used the "matched sample method" to deal with changes in goods on the market. Thus, if an item that was priced last quarter is no longer available, it is replaced by an item with similar characteristics and quality. If the price collectors are unable to find a perfect substitute, an adjustment is made to reflect the difference in characteristics and quality. It is difficult to apply the matched sample method when many characteristics of a good are changing rapidly and simultaneously. This is particularly true of computers and other high technology products.
An alternative approach is to use the hedonic method. This involves three steps:
- Choosing characteristics of the good which encapsulate its "quality". In the case of computers, the characteristics may include processor speed, the amount of memory, the capacity of the hard disk, and so on.
- Estimating implicit prices of the characteristics based on a large sample of computer models. This is usually done using regression modelling.
- Using the implicit prices of the characteristics to adjust for quality change.
The hedonics project team is led by Richard McKenzie from Producer Price Indexes Section (PPI). Poh Ping Lim (ASB), the chief analyst, has estimated hedonic functions for desktop and notebook computers. PPI Section will now use the estimated functions to construct experimental quality-adjusted price indexes.
For more information, please contact Ms Poh Ping Lim on (02) 6252 7271.
COMPARING DISABILITY PREVALENCE
In early 2000, the Family and Community Statistics Section within ABS asked Analysis Branch to undertake a project on disability. This involved comparing the disability prevalence rates suggested by the different questionnaire modules used in various ABS surveys.
The term 'disability' covers a wide range of impairments, limitations and restrictions. Interpretation of these varies from person to person; even for the same person, the interpretation may vary across time. People may be reluctant or unable to identify themselves as having particular types of disability. Furthermore, responses to a disability question may be sensitive to the survey context; asking questions about other topics before asking the questions on disability may encourage or discourage a positive response.
An important source of national data on disability is the ABS Survey of Disability, Ageing and Carers (SDAC), conducted every five years. Apart from SDAC, ABS has included disability modules in some other household surveys, such as the Household Expenditure Survey (1993-94 and 1998-99), the Survey of Education and Training (1992 and 1997), the Survey of Employment and Unemployment Patterns (1995) and the Time Use Surveys (1992 and 1997).
Users ask the following questions : How comparable are the disability populations identified by the modules in the non-SDAC surveys, with the populations identified in the SDAC? How similar or relatable are they in terms of age, sex and severity of handicap? Is it meaningful to use these populations (provided by the non-SDAC surveys) to look at disability?
The analysis is now completed. Results were presented to the project board which met on the 20th December 2000. The most important result was that the number of persons with a disability is significantly different between surveys, so users of data from the different sources need to be careful. The project board asked that the methods and results be written up in several papers for ABS and non-ABS audiences.
For further information, please contact Ravi Ravindiran on (02) 6252 7039.
NEW LOOK WEB SITE FOR THE STATISTICAL CLEARING HOUSE
The Statistical Clearing House (SCH) is responsible for reviewing all surveys of 50 or more businesses conducted by, or on behalf of, Commonwealth government agencies. The primary purpose of the SCH is to reduce the burden of government surveys on businesses by ensuring such surveys do not duplicate existing collections and are of sufficient quality to warrant the burden imposed. The data collection phase of a survey must not begin until the survey has been approved.
The SCH provides advice and assistance at all stages of the review process. Specific requirements on the type of information required for the review can be found on the Internet, on the SCH Web site at www.sch.abs.gov.au.
The SCH has recently launched a new look web site that contains a wealth of information designed to assist survey managers, researchers and others interested in the design and co-ordination of surveys. The new SCH web site has three main sections:
1. Information about the SCH:
This section provides a brief overview of the SCH, a description of the SCH clearance process for surveys including general clearance procedures, the information template to be completed by survey managers and the review criteria used to assess surveys.
It is designed to provide survey managers with the information and tools they need in order to have their survey reviewed.
2. Commonwealth Business Surveys Register:
The Commonwealth Business Surveys Register contains information about business surveys that fall within the scope of the SCH clearance process and have been reviewed by the SCH. For each survey it includes a description of the survey design and provides a direct contact point to the survey manager.
In providing descriptions of Commonwealth Government surveys, the Business Surveys Register aims:
- to increase the awareness, quality, and use of the statistical data generated by these surveys, thus reducing the likelihood of survey duplication; and
- to facilitate the design of future surveys, by making existing designs readily visible.
In addition to the Business Surveys Register, this section also provides information on alternative data sources. It provides links to Statistical Directories which may assist in identifying alternative data sources.
The Business Surveys Register is a valuable reference tool. When researching a particular topic or designing or contemplating a new survey, you can look on this Register to see whether any similar surveys have already been conducted. This will allow greater sharing of the information currently available. It may result in survey managers being able to reduce the content of their survey or possibly remove the need to conduct their survey at all.
Also, by looking at survey design methods used for similar surveys, survey managers may be able to reduce their survey development time by learning from the experience of others in conducting similar surveys. By being aware of how similar surveys have been done, survey managers will be able to ensure that their survey results are comparable with other surveys (e.g. by using the same survey standards and/or classifications).
3. Reference Material:
This section of the web site provides a range of information to assist survey managers, researchers and others interested in the design and co-ordination of surveys. The section contains a number of different sub-sections including:
- Glossary: A glossary of terms and phrases associated with survey design and used on the Web site.
- Survey Design Manuals: Useful reference material for designing surveys or interpreting survey results. Manuals currently available on the web site include: Basic Survey Design; and Forms Development Procedures and Design Standards.
- Standard Classifications: Links to standard classifications (such as industry, occupation, geography).
- International Practices: Standards and procedures on survey quality and best practices documented by National Statistical Offices and other national and international organisations.
- Research Papers: These are likely to be of particular interest to those designing surveys or interpreting survey results. Topics such as non-response and telephone interviewing are typical.
- Alternative Data Sources: Links to Statistical Directories. This will be useful to researchers in identifying other potential sources of data.
This section of the web site will be updated as further reference material is identified.
The search facility on the new web site allows you to target particular areas of interest. For example you could search for a particular word or words (and variants thereof). You can search the entire web site or limit the search to various areas of the web site for example when searching for business surveys, you could limit the search to the Business Surveys Register and hone in on just surveys conducted by particular agencies.
The web site will continue to evolve. As surveys are approved, they are loaded (with clearance by the survey managers) on to the web site. As further reference material is identified, it too will be loaded to the web site. It is designed to be a reference tool that can be scanned for relevant information as the need arises. Users are encouraged to periodically check the site to see what new information has been incorporated.
If you have any questions about the web site, the SCH can be contacted on (02) 6252 5285.
This page first published 16 July 2001, last updated 9 November 2004