Chapter 24 Quality of the National Accounts
24.1 'Quality' in relation to statistics is a multidimensional concept which embodies the notion of 'fitness for purpose'. In order to assist and encourage informed decision making, statistics need to be not only as accurate as possible, but also timely and relevant. There are often trade-offs between the various aspects of quality, and in order to make economic statistics timely enough to be relevant indicators for the analysis of current or recent economic conditions this is likely to be at the expense of some degree of accuracy. The ABS, in consultation with data users, aims to optimise the various aspects of quality.
24.2 The national accounts program is discussed against the seven quality dimensions of the ABS Data Quality Framework. These dimensions are a view of data quality aspects that determine fitness for purpose and relate to the institutional environment, relevance, accuracy and reliability, timeliness, accessibility, interpretability, and coherence. As well as informing users about quality, the framework also provides feedback to ongoing quality improvement programs within the ABS.
24.3 Underlying these dimensions of quality is the notion of integrity – that statistical policies and practices are guided by ethical standards and professional principles which are transparent. The integrity of the ABS is underpinned by legislation within which the organisation operates, and its willingness to subject its operations and performance to both internal and external scrutiny. The principal legislation determining the functions and responsibilities of the ABS are the Australian Bureau of Statistics Act 1975 and the Census and Statistics Act 1905. These Acts provide that the ABS is headed by the Australian Statistician – a statutory office with an independent status and the authority to conduct statistical collections.
24.4 This chapter describes each of the aspects of quality and assesses the national accounts against them. Compilation of the national accounts is a complex task involving many diverse data sources. It is not possible to provide a single, comprehensive measure of the quality of the estimates. Nonetheless, it is possible to gain an insight into their quality by analysing each of the aspects of quality. To obtain an overall picture, all aspects need to be considered together. However, different users may weight each of the aspects differently, and within each aspect what satisfies one user may not satisfy another. Thus, two users may look at the same set of statistics, with one considering them to be of good quality while the other may think that there are quality deficiencies.
24.5 The institutional environment relates to the institutional and organisational factors which may have significant influence on the effectiveness and credibility of the agency producing the statistics. This is an important consideration as it enables an assessment of the context in which the statistics are produced, and this may influence the validity, reliability, or appropriateness of the product.
24.6 The ABS’s mission is to assist and encourage informed decision making, research and discussion within governments and the community, by leading a high quality, objective, and responsive national statistical service.
24.7 The legislative framework, organisational structure, planning and quality management practices of a statistical agency all affect the ability of a statistical program to meet user needs efficiently and effectively. The ABS operates within a framework that includes the Australian Bureau of Statistics Act 1975 and the Census and Statistics Act 1905.
24.8 The Census and Statistics Act 1905 provides the Australian Statistician, who heads the ABS, with the authority to conduct statistical collections and, when necessary, to direct a person to provide statistical information. The Act imposes obligations on the Statistician to publish and disseminate compilations and analyses of statistical information and to maintain the confidentiality of information collected under it.
24.9 To ensure impartiality and independence from political influence, the Australian Bureau of Statistics Act 1975 (the ABS Act) sets out the Australian Statistician's independence. The ABS Act requires the Australian Statistician to prepare an annual report on the operations of the ABS for presentation to Parliament. The ABS Act also establishes the Australian Statistical Advisory Council (ASAC). ASAC is the key advisory body to the ABS and provides valuable input to the directions and priorities of the ABS work program. The ABS is financed, via Parliamentary appropriations, for its administration and programs. Its financial statements are audited annually by the Australian National Audit Office (ANAO) and are published in the annual report. Together, these elements provide an environment conducive to the production of high-quality national accounts and other statistics.
24.10 The ABS has an excellent reputation, both at home and abroad, for providing a statistical service of quality and integrity. A key reason for this reputation is that the ABS is willing to subject its operations and performance to both internal and external scrutiny. Some of the ways this is done is through:
- the advertisement of all scheduled release dates for publications up to twelve months in advance
- the use of daily press and media releases to inform users of publications being released each day
- the strict embargo policy, which is known to the public, that ensures impartiality for the release of all publications
- the publication of the ABS three-year Forward Work Program, which describes the ABS program, including the resources to be used, the outputs, the clients and the uses of statistical information, and the proposed developments over the next three years
- the release of information about statistical standards, frameworks, concepts, sources and methods in a range of information papers and other publications
- the inclusion of details of major revisions to published data in the explanatory notes of the relevant publication.
24.11 The ASNA is based on a culture which focusses on quality, and emphasises objectivity and professionalism within the institutional environment of the ABS.
24.12 Relevance relates to the degree to which statistical information meets the real needs of users. It involves client liaison, program review, priority setting and assuring that the statistics produced together with the underlying concepts conform to international statistical standards.
Client liaison, priority review and program setting
24.13 The ABS regularly reviews all its statistical programs to ensure they remain relevant to user needs and to ensure capacity is available to provide information on new and emerging issues facing policy advisors and other data users. Each year relative priorities and competing resource requirements of all programs are formally and extensively considered by senior management, generally following consultation with users. The ABS maintains a three year forward work program which is rolled forward on an annual basis. Detailed work programs are developed, resources allocated, and performance indicators are established for each statistical program including the national accounts.
24.14 Ensuring ABS national accounts outputs support decision making is achieved by extensive consultation with users. This happens in a variety of formal and informal fora:
- the Australian Statistics Advisory Council
- the State Accounts User Group (SAUG)
- the Input-Output User Group (IOUG)
- the Productivity Measurement Reference Group
- key client manager discussions
- regular contact with the Commonwealth Treasury, State and Territory treasuries and the Reserve Bank of Australia, particularly through quarterly seminars and discussions
- regular informal contact with other key users such as the Productivity Commission, particularly through their use of national accounts data
- ad hoc meetings with academics and other experts.
24.15 While unmet demand for data is taken seriously, it does not mean that the ABS is currently in a position to move forward in all cases. The major area of unmet demand identified by users relates to the availability of state data. The ABS has greatly improved the coverage of state economic statistics in recent years with ongoing improvements to the state accounts and the addition of state multifactor productivity estimates. Users have indicated a requirement for state Input-Output tables and quarterly GSP. At this stage, the ABS has not been able to pursue their development on the grounds of cost and expected insufficient data quality.
24.16 The Economic Statistics User Group is the key forum for seeking user views on statistical issues, emerging data needs and priorities for the national accounts and other economic statistics. It meets regularly and the membership is drawn to provide a wide-ranging representation of the economic statistics user community.
Concepts, definitions and classifications
24.17 An important aspect of quality is that the concepts, definitions and classifications used in the body of statistics are relevant to, and understood by, users and that, in order to achieve this, national accounts statistics need to be placed in an appropriate conceptual framework.
24.18 The framework used in the ABS in the compilation of national accounts statistics is based on the 2008 SNA. The various editions of the System have been developed over many years and involved the input of international organisations, national statistics agencies, academic experts, and users of economic data. Moreover, it is designed as a set of accounts relevant to the analysis of economic issues. The System is reviewed and updated periodically in order to accommodate the changing structure of economies and contemporary economic issues.
24.19 This publication explains the conceptual framework and estimation methodology for Australia's national accounts statistics. It is published periodically to reflect changes in concepts, sources and methods. To keep users informed, these changes are also documented as they occur in the quarterly and annual publications, Australian National Accounts: National Income, Expenditure and Product and Australian System of National Accounts respectively.
24.20 Adaptations are made to the SNA framework to accommodate domestic perspectives, and to ensure that the ASNA is relevant to domestic users. These are generally done in such a way that the ASNA remains comparable to those of other countries.
24.21 The ABS places great emphasis on ensuring that its economic classifications align with the 2008 SNA and other international standards to ensure comparability across its own economic collections and outputs as well as with those of other statistical organisations. Standard classifications are an essential element for the compilation and presentation of statistics produced by national statistical offices. Their use ensures that statistics are comparable across industry and sector boundaries and can be aggregated from various collections. Within the conceptual framework, the ABS attempts to make the statistics as useful as possible by classifying the data in a number of ways to meet user requirements.
24.22 The 2008 SNA recognises that the core frameworks and classifications will not meet all possible needs for economic accounts data. It therefore recognised the need for 'satellite accounts', which can be used to focus on particular areas of the economy in more detail, or to allow different concepts or coverage, while retaining a link back to the core national accounts system. The ABS has developed a number of satellite accounts in response to user demand and these are outlined in Chapter 23.
24.23 For the national accounts statistics to remain as relevant as possible, resources are devoted to the research and development of new statistics. The outcomes of the research and development program are generally discussed with users, and published in Information Papers, prior to changes being implemented in the statistics.
24.24 The relevance of ABS national accounts statistics is also enhanced by the frequent inclusion of spotlight articles on topics of interest in the quarterly and annual national accounts releases. A full list of articles is included on the ABS website, and includes topics such as the impact of COVID-19; the impact of the drought; the relationship between GDP and employment; accounting for the environment in the national accounts; the underground economy and GDP; long-term trends in industry structure of the Australian economy; and income, saving and wealth.
24.25 Evidence that the processes described above are in place is provided by descriptions of the concepts, sources and methods as described in this publication. From the program perspective, evidence of periodic evaluation of the current relevance of each program can be provided and the impact of the results of these evaluations can be assessed.
24.26 Evidence of relevance is also provided by measures of usage, by client satisfaction results and by high profile examples of statistical information influencing or shedding light on important policy issues. Pointing out and publicising new analytical findings also demonstrates relevance.
24.27 The ABS maintains regular contact with key clients and other users of the national accounts to gauge their satisfaction with the services provided, including their views on the relevance of the data, and priorities for future development. Information gleaned from these formal and informal contacts become input into priority-setting processes, the outcomes of which are subject to high-level management review processes.
24.28 For most users, accuracy is the most sought-after attribute of data. Accuracy can be defined as the proximity of an estimate to some notional true value. It is not possible to produce an objective overall measure of accuracy of the accounts because the national accounts draw data from a wide variety of sources, reflecting varying valuations, coverage, frequency, detail and timeliness. Assessments need to be made instead of individual component items within the accounts. Even at this level, the use of multiple data sources in estimating a single item, their variable accuracy over time, and changing compilation methods complicate the picture. As a result, assessment of the accuracy of an item requires a high degree of subjective judgement based on knowledge of the sources, the data and the compilation methods used.
24.29 It can also be useful to make a distinction between the concepts of accuracy and reliability in considering the quality of national accounts statistics. Accuracy is the proximity of an estimate to some notional true value, while reliability is the proximity of initial and intermediate estimates for a particular period to the 'final' estimate for that period. A series which is never revised is reliable, but it may not be accurate. Although reliability can be objectively measured by an analysis of revisions, it is a relative term, and users are likely to have some tolerance to revisions, given the trade-offs with other characteristics of quality.
24.30 In practice, accuracy and reliability tend to be interwoven and reinforcing, at least in Australia's national accounts. Ideally, the size of revisions gets smaller (and the statistics become more reliable) as the estimate for a particular period passes through a sequence of revisions, and the estimate moves closer to the true value (and the statistics become more accurate). In practice, this may not always be the case. Revisions can be reduced by delaying the release of statistics until all or most 'final' data sources are available, but this would mean that the statistics would be less relevant to users. ABS policy is to always aim for the most accurate estimate, even though this may be at the expense of more frequent revisions.
24.31 Judgements can be made as to the extent of error likely to be associated with an estimate by being aware of the factors influencing accuracy. Additionally, historical analyses of the revisions that the ABS makes to its estimates of GDP and its components can provide a quantitative guide to the reliability of the statistics produced.
Factors affecting accuracy
24.32 The range of factors that can influence the accuracy of the national accounts include:
- coverage deficiencies
- input data errors, which include sampling error and error due to the inability of data providers to report on the correct basis, mistakes in the reporting of data, and error due to non-response
- error introduced during the processing of data
- methodological deficiencies
- output error due to inadequate editing and data confrontation.
24.33 The scope of the Australian national accounts is exhaustive within the production boundary recommended in the 2008 SNA, with the exception of illegal production. Illegal activity (which relates mainly to illicit drugs) is omitted from Australia's official statistics because it is difficult to measure with sufficient accuracy.
24.34 Illegal activity is one part of the Non-Observed Economy (NOE). The NOE also includes:
- underground production, which is activity which is legal but is deliberately concealed from authorities
- household production for own final use, such as crops and livestock, own-account fixed capital formation and owner-occupied dwelling services
- informal economy production
- the statistical underground, which is production that is missed due to deficiencies in data collection, such as survey non-response.
24.35 By its very nature, the NOE cannot be directly measured. Estimates of it rely on limited indicative information and a variety of indirect methods, all of which can be regarded as contentious. It is likely that an unknown proportion of underground production would be captured in the official GDP estimates due to the data sources used and the estimation methods employed. Nevertheless, it is accepted that additional adjustments for non-observed activity are required.
24.36 The ABS does not attempt to distinguish between the formal and informal sector, as informal production is relatively unimportant in Australia. Estimates, however, are included for the value of dwelling services consumed by owner-occupiers and owner-built construction activities and goods produced by households for own consumption, such as food consumed on farms and home-grown fruit and vegetables. Volunteer services and unpaid housework (such as cooking and cleaning) are excluded, consistent with the recommendations of the 2008 SNA. Estimates for these services have been prepared as satellite accounts; see Chapter 23 for more detail.
24.37 Explicit upward adjustments are made to account for underground activity which is legal, but is conducted in such a way as to avoid detection by taxation and other government authorities. The method used to obtain an estimation of underground activity was to systemically analyse each component of GDP and make judgements as to the maximum feasible level of underground activity, given anecdotal and available evidence from the Australian Taxation Office.
24.38 Issues involved in measuring the Non-Observed Economy are articulated in Information Paper: The Non-Observed Economy and Australia’s GDP.
24.39 The scope of some data collections may be drawn more narrowly for cost or other reasons. For example, the scope may exclude non-employing businesses; some industries; or business under a certain size. A narrower scope is more likely for monthly or quarterly surveys. Data from the taxation system and other sources allows these gaps to be filled. The annual economic collections are designed around the availability of taxation data, and the supplementation brought to the coverage of small businesses, including non-employing businesses. Non-profit institutions are generally exempt from the payment of income tax, such that data for those units has to be collected by the ABS via the annual economic survey.
24.40 The ABS undertakes a large number of collections that directly feed into the national accounts compilation process. National accounts requirements are a key consideration in the design of these surveys. A range of non-ABS data are either integrated into the survey outputs or are used independently to compile the national accounts. Examples include data from the taxation system, the financial institution regulation system, government financial reporting data, and retail scanner data. External data sources can potentially have issues for quality assurance and measurement, being beyond the control and responsibility of the ABS. Considering these points, the ABS and external sources enter into explicit arrangements concerning roles and responsibilities of the respective agencies; questionnaire content and design; timely data and transference procedures; editing and querying protocols; and appropriate feedback loops to the source agency.
24.41 The quality of ABS statistics is dependent on the application of good statistical methods during the selection and collection phases of a survey. The ABS puts substantial effort into developing standards in terms of classifications, concepts, data item definitions and question modules. All ABS surveys must use these standards. The collection instrument must be well tested and evaluated and this process is supported through documented standards in forms design and forms evaluation.
24.42 A high proportion of information used in compiling the Australian national accounts comes from surveys using the ABS Business Register (ABSBR) to provide the statistical frame from which representative samples are drawn. The ABSBR is based on the Australian Taxation Office's Australian Business Register (ABR), which contains all businesses with an Australian Business Number (ABN). It is expected to be comprehensive because businesses are required by law to obtain an ABN, with very limited exceptions. The ABS has adopted a strategy of building and maintaining its own records for large and complex businesses, with information for the remaining businesses sourced from the ABR. Although the ABS register is comprehensive and current, there are known to be some problems with the quality of the industry coding, which would have implications for the quality of the data for individual industries used in the national accounts.
24.43 Sample design and estimation systems are developed by specialist areas in accordance with internationally accepted practices. Accuracy is considered in terms of both sampling and non-sampling error. The ABS publishes information on the relative standard errors (RSEs) for its various sample surveys. These can provide an indication of the accuracy of the national accounts components to which they relate. It has not been possible to systematically calculate the impact that RSEs have on the various national accounting aggregates, because of the transformations of survey data, and the aggregations made in order to compile the national accounts.
24.44 An important potential source of non-sampling error can result from the inability of some data providers to report on the correct basis, considering the data requirements underlying the national accounts. Data providers can make errors with regard to the content, timing and valuation of their transactions. It is inevitable that some data providers will include extraneous items in their survey responses, and exclude relevant items, although every effort is made to match survey data items with business accounting practices. Survey forms are tested with a small number of providers before approved for use. Sophisticated techniques are used to edit provider responses, but errors can remain undetected. Because the national accounts is a closed system, such errors can lead to inconsistencies, affecting the coherence of the accounts.
24.45 The ABS uses best practices in survey design and operation, given the data intensive needs of the national accounts system, and the cost and respondent load imperatives that apply in survey design. National accounts compilers make do with data that are of acceptable (but less than ideal) quality, particularly for data at a finer level of detail, where the standard errors are often higher. Limited detail might be collected in some cases, or only collected infrequently (such as product details). The ongoing challenge is to ensure the adequacy of input data quality, in order to meet national accounts requirements. The national accounts process itself is designed to mitigate input data problems through data confrontation, as well as balancing in the Supply and Use tables, which are used to benchmark the national accounts. High quality input data, however, are essential to high quality national accounts data.
24.46 The ABS periodically reviews its economic survey strategy to ensure data requirements are met within the constraints of resource availability. The national accounts requirements for data are regarded as of very high importance.
24.47 The data sources and methods used in preparing ABS national accounts statistics are regularly reviewed, and changes are made periodically to the basis of compilation of an item. A major methodological improvement to the Australian national accounts was the introduction of Supply and Use benchmarking and annually re-weighted volume estimates in 1998. The former results in greater coherence of the accounts, and the latter provides a superior measure of economic growth to the previously available volume estimates based on five-yearly reweighting.
24.48 Seasonal adjustment methods used by the ABS are based on the United States Bureau of the Census (USBC) Method II Seasonal Adjustment program, X-11 variant. This approach decomposes original estimates into a combination of modelled trend, seasonal and irregular components. Seasonally adjusted estimates are created by adjusting original estimates using seasonal factors to remove the seasonal component, and trend estimates are obtained by removing the irregular component from seasonally adjusted estimates. Since September 2006, the ABS has usually re-analysed and re-estimated seasonal factors for national accounts time series with each quarterly publication—a procedure referred to as concurrent adjustment. This process generally results in the best quality estimates because the method is most responsive to emerging changes in seasonal patterns. However, in response to the economic disruption caused by the COVID-19 pandemic from 2020 onwards, some components have returned to the previous process of re-estimating and fixing seasonal factors annually, to better manage large movements near the current end of the series.
24.49 As the national accounts are compiled within a comprehensive framework, it is possible to reduce the impact of data errors through the confrontation of the various estimates in the national accounts. Data confrontation is built around the conceptual relationships between data items. The best known data confrontation exercise is the compilation of the annual Supply and Use tables. Estimates of the supply (production) and use (demand) for commodities are compared in the compilation of these tables, with differences in the initial estimates being eliminated. At the same time, estimates of the value of production are compared with estimates of incomes attributable to production, and differences eliminated (further details are included in Chapter 8 on coherence of the national accounts).
24.50 Each collection area is required to confront its data with other data held by the ABS and other organisations as an important part of the process of ensuring the coherence of ABS statistics. Clearance meetings are held for all the major economic collections used to compile the national accounts, as a means of assuring data consistency between those collections and the national accounts, with emphasis on the most recent reference period.
24.51 The national accounts quarterly compilation process incorporates a review process designed to highlight inconsistencies and improbable data movements. Problems are identified, investigated, and resolved in the process of finalising the GDP estimates. There is a feedback loop to the data collection areas, culminating in the clearance procedures mentioned above.
24.52 Since 2005, the ABS has developed quarterly Supply and Use (QSU) tables as an editing tool to assist in the preparation of the quarterly national accounts. Preliminary quarterly estimates for the production and income components are used as inputs to the model which generates estimates of product supply and use in a time-series format. The QSU model enables inconsistencies between the different measures of GDP to be identified and investigated more systemically, and at a greater level of detail than is possible by simply examining the aggregate estimates. At the present stage of its development, the QSU model is used as an aid in the compilation of the seasonally adjusted production and expenditure-based estimates of GDP. A description of the initial QSU model (and how it is used) is contained in the research paper, A Supply and Use Model for Editing the Quarterly National Accounts, Australia. The model has recently been updated to increase the level of product detail and incorporate current price measures.
Subjective accuracy ratings
24.53 Accuracy is concerned with the proximity of an estimate to the 'true' but unknown value of the component being measured. It is concerned with the degree of precision associated with the estimate. As already discussed, the complex nature of the national accounts makes it extremely difficult to produce a tangible benchmark against which to measure accuracy. In practice, the accuracy of the national accounts is evaluated by considering the potential sources of error, and whether those sources have been minimised to the greatest extent possible. One approach to tie all the information related to data quality together is to assign subjective accuracy ratings to each series.
24.54 Subjective accuracy ratings are not derived by a particularly rigorous process, but represent an intuitive assessment by national accounts compilers. A consensus was reached taking into account knowledge about the standard errors on key survey inputs; impressions about the coverage and reliability of administrative data sources; and revisions to initial estimates of growth.
24.55 The next three tables contain an assessment for the initial quarterly estimates of movement for the current price and chain volume estimates of expenditure, income and production components of GDP. These have been chosen as they are generally the most anticipated of the national accounts estimates. Each component has been assigned one of the following grades:
D very poor
|Component||Current price estimates||Chain volume estimates|
|Final consumption expenditure|
|Total final consumption expenditure||A||A|
|Private gross fixed capital formation|
|Ownership transfer costs||A||B|
|Machinery and equipment||B||B|
|Cultivated biological resources||C||C|
|Intellectual property products||C||C|
|Total private gross fixed capital formation||B||B|
|Public gross fixed capital formation|
|Total public gross fixed capital formation||B||B|
|Domestic final demand||A||A|
|Changes in inventories|
|Gross national expenditure||A||A|
|Exports of goods and services||A||A|
|Imports of goods and services||A||A|
|Component||Current price estimates|
|Compensation of employees||A|
|Gross operating surplus|
|Total non-financial corporations||A|
|Dwellings owned by persons||A|
|Total gross operating surplus||A|
|Gross mixed income||C|
|Total factor income||A|
|Taxes less subsidies on production and imports||A|
|Component||Chain volume estiamtes|
|A||Agriculture, forestry and fishing||B|
|D||Electricity, gas, water and waste services||A|
|H||Accommodation and food services||B|
|I||Transport, postal and warehousing||B|
|J||Information media and telecommunications||B|
|K||Financial and insurance services||B|
|L||Rental, hiring and real estate services||B|
|M||Professional, scientific and technical services||B|
|N||Administration and support services||B|
|O||Public administration & safety||C|
|P||Education and training||C|
|Q||Health care and social assistance||B|
|R||Arts and recreation services||B|
|Ownership of dwellings||A|
|Gross value added at basic prices||A|
|Taxes less subsidies on products||A|
24.56 A more objective (but still limited) indicator of accuracy is provided by examining the differences between the conceptually equivalent estimates of GDP, measured by the expenditure, production and income approaches. Quarterly GDP in the Australian national accounts is derived as the average of the three measures, with accounting balance being achieved by an explicit statistical discrepancy item for each of the three elements.
24.57 The three measures of GDP are balanced in annual terms using the Supply and Use approach, up to the year prior to the most recent complete year (and latest two years in the June quarter release). This process eliminates the statistical discrepancies in annual terms, except for the latest year. The quarterly estimates of GDP are benchmarked to the annual estimates, but within-year inconsistencies remain.
24.58 Large and persistent statistical discrepancies relative to GDP can indicate gaps in the coverage of the measures or other quality issues. Because the measures are balanced annually in a Supply and Use table, the magnitude of the quarterly statistical discrepancies is small relative to the level of GDP, and they cancel out over the year. Any significant data coverage issues have been addressed over time. Of more interest in the Australian context is that quarter-to-quarter changes in the level of the statistical discrepancies are associated with inconsistencies between the growth rate of GDP, and its conceptually equivalent measures.
24.59 Figure 24.1 shows the discrepancy between quarterly growth rates of chain volume GDP as measured by the expenditure, production and income approaches and their average, GDP(A). In this graph, positive discrepancies indicate that a given measure has a higher growth rate than the average measure.
Figure 24.1 Quarterly growth in GDP measures
Net errors and omissions
24.60 The statistical discrepancies between the three measures of GDP provide one indicator of the accuracy of the national accounts; the net errors and omissions item in the financial account provides another. It represents the difference between the net lending derived in the capital account, and the conceptually-equivalent net lending item derived in the financial account – the net errors and omissions item is required to balance them.
24.61 Net lending in the capital account is derived as a residual and reflects the expenditure, production and income components of GDP as well as other items such as net property income and net secondary income. The financial account also derives net lending as a residual but using totally independent data – it is the difference between the acquisition of financial assets and the incurrence of liabilities.
24.62 Figure 24.2 plots the net errors and omissions item in annual terms. Some year-to-year differences are evident, reflecting the difficulties inherent in measuring residuals. It reflects timing and other errors in the estimation of income, expenditure and capital flows on the one hand and financial assets, liabilities and financial flows on the other.
Figure 24.2 Net errors and omissions
["","Net errors and omissions"]
24.63 The magnitude and volatility of the net errors and omissions item and the statistical discrepancy can also be used to indicate the quality of the saving estimates in the national accounts. National net saving is derived in the income and use of income accounts by deducting final consumption expenditure and consumption of fixed capital from gross disposable income. Its quality is particularly sensitive to inaccuracies in the series from which it is derived, as a relatively small residual measure derived from very large aggregates. This point needs to be borne in mind when using the data for analysis.
24.64 Saving is a source of funds for both acquisition of financial assets and investment. An alternative and largely independent measure of saving can be derived using acquisition of financial assets from the financial account as the starting point. The accounting relationships within the national accounts system mean that the alternative measure can be calculated by deducting the net errors and omissions item and the statistical discrepancy between GDP(I) and GDP(E) from the official measure of saving. Figure 24.3 plots the official measure of annual saving and the alternative measure as a percentage of GDP, using data expressed in current prices. The series track each other closely, although there are some year-to-year differences.
Figure 24.3 National net savings, ASNA and alternative measures
["","ASNA measure","Alternative measure"]
24.65 The timeliness of statistical information refers to the delay between the reference period to which the information pertains, and the date on which the information is made available. The timeliness of information also influences its relevance; users obviously want data that are as contemporary as possible.
24.66 The source data used to compile the national accounts are available with varying degrees of timeliness, frequency, accuracy and detail. Successive vintages of source data are likely to be more accurate. This is consistent with the nature of business accounting systems where sub-annual data from management information systems are updated and eventually replaced by audited annual accounts.
24.67 The ABS aims to collect the various vintages of source data as soon as satisfactory results are typically available from the respondents' own systems. These may not be final data, and, in some cases, businesses may not be able to respond in accordance with the set deadlines at all, resulting in an expected level of non-response. Survey systems are designed to impute values for late or non-responders. A consequence is that initial estimates are likely to be revised.
24.68 There is an important trade-off between the accuracy and reliability of the estimates and the timeliness of their release. To meet the decision-making needs of users, the Bureau is pressured to produce statistics in as timely a fashion, and as frequently as possible. Complete or accurate data may not be available with the desired timing or can only be obtained at unacceptably high resource costs if at all, thereby compromising data accuracy. Both accuracy and timeliness are characteristics of quality – the ABS relies on feedback from users as to the optimal balance between them.
24.69 The trade-off between timeliness, accuracy and detail is accommodated by way of the sequencing of releases of national accounts publications. The first published quarterly estimates are usually made available about 60–70 days after the reference period. The detailed annual estimates are released between four and five months after the reference period. The most detailed national accounts estimates relating to production, income and expenditure are contained in the input-output tables, which typically become available about 38 months after the reference period. The quarterly financial accounts are usually released in the month following the corresponding quarterly national accounts, and the annual state accounts are released in the month following the annual national accounts. Australia's national accounts rate well against those of other countries in terms of their timeliness – particularly when the level of detail made available is taken into consideration.
24.70 A number of countries, particularly those that release national accounts estimates early, release a preliminary issue of the quarterly national accounts, followed later by 'final' estimates. The preliminary releases are often based on a partial view of the economy and require a certain amount of forecasting. For example, it is possible to base a preliminary estimate of quarterly production-based GDP (or GDP(P)) on two-months data if monthly production indexes are available, and to then forecast the third month. This method obviously improves the timeliness of the national accounts as they can be released soon after the end of the quarter, although the preliminary releases are normally issued with caveats. The ABS has not chosen this approach, and monthly production indexes are not compiled to enable the use of this approach.
24.71 Accessibility of information refers to the ease with which users can learn of its existence, locate it, and import it into their own working environment. Aspects covered include data availability, metadata availability (i.e. information about the data), and the degree of interpretative assistance available to users of the data. Data availability involves issues of data presentation and distribution media, as well as the availability of non-confidential but unpublished data. Metadata availability concerns the availability of information concerning the concepts, sources and methods associated with that data. Accessibility also considers whether the information surrounding or supporting the data is openly available to the public.
24.72 Most aspects of accessibility in the ABS are determined by agency-wide dissemination policies and delivery systems. At the program level, the main responsibility is to choose appropriate delivery systems, and to ensure that statistical products are properly included within corporate catalogue systems.
Accessing the data
24.73 The ABS provides users with ready access to national accounts statistics. Paper publications have now largely given away to a variety of electronic releases which are now the preferred way of accessing statistical information. Everything on the ABS website is free-of-charge, and this includes all standard releases of national accounts information.
24.74 A national accounts theme page on the ABS website lists all the quarterly and annual national accounts datasets currently available. It also provides an extensive range of national accounts information including access to the latest national accounts data. Key results are highlighted.
24.75 Information about the full range of national accounts and other ABS publications is available through the <Statistics> link on the ABS website.
24.76 Some users require more detail than that provided in the standard national accounts releases, and can seek information on request, and, subject to cost-recovery charges, the information will be provided if available, and of sufficient quality.
Accessing the metadata
24.77 The ABS has a range of ways of providing users with information about national accounts data. This publication outlines the major concepts and definitions; describes the data sources and methods used to prepare the estimates; and provides information about the ways in which the relevant international concepts and standards have been implemented in Australia. It also discusses the accuracy and reliability of national accounting concepts and estimates.
24.78 The web page for each publication leads with key statistics. Information is included about future releases and about any revisions and changes in the issue. The <On this page> box describes the major elements of the data, the tables and graphs contain footnotes to alert users to relevant issues that affect the data series. Each publication links to its own <Methodology> page containing a glossary and explanatory notes, which comment on the key concepts, sources and methods and their implications for the quality of estimates. A quality declaration provides a brief assessment of the statistics against the seven dimensions of the ABS data quality framework.
Information about releases
24.79 The ABS website provides information about upcoming releases. Release dates of national accounts publications are announced in advance — in general release advices, and in the < Future Releases> and <Previous Releases> tabs attached to individual publications.
24.80 The ABS adheres to the long-established principle that the results of statistical collections should be made available to all users at the same time, and as soon as practicable. The Bureau will consider giving very limited pre-release access for the purpose of briefing a Minister. An embargo is placed on the release of statistics until 11.30 am (Canberra time) on the designated day of release. There are strict security procedures to ensure that there is no unauthorised access of statistics prior to release.
Consulting with users
24.81 Users are routinely consulted regarding their data requirements, and how they would like to see the data presented in publications and electronic media. On the 21st of September 2020 the ABS launched a new website with a key aim to improve user experience and address feedback. National Accounts significantly improved publication content with the new ABS website. Improvements included more prominent key statistics, more interpretable analysis and other content, interactive graphs and tables, and improved search functionality.
Assistance to users
24.83 Users need to understand the properties of the information in order to make appropriate use of statistical information. That requires the provision of descriptions of the underlying concepts, variables and classifications used; the methods of collection processing and estimation used in producing the information; and an assessment of the accuracy of the information.
24.84 Statistical information that users cannot understand, or can easily misunderstand, has no value and may have negative value. Providing sufficient information to allow users to properly interpret statistical information is therefore essential. Information about information is technically known as metadata. Managing interpretability is primarily concerned with the provision of metadata.
24.85 The information needed to understand statistical data falls under three broad headings:
- the concepts, definitions and classifications underlying the data
- the methodology used to collect and compile the data
- indicators of accuracy of the data.
24.86 The description of methodology also serves as a surrogate indicator of quality – helping users to assess suitability for purpose.
24.87 There are close relationships between these three headings and other dimensions of quality. The underlying concepts and classifications are also a prime determinant of coherence (see next section), and the degree to which they conform to national and international standards should be apparent from the metadata.
24.88 This publication defines and discusses the major concepts, definitions and classifications which underlie national accounts estimates. It also describes the methodology used to transform input data into statistical outputs and discusses the accuracy and reliability of those estimates. The concepts, classifications and terminology follow closely those of the SNA, and, therefore, users can be confident that valid comparisons can be drawn with national accounts data produced by other countries.
24.89 This publication is supplemented by an assortment of information papers, spotlight articles, working papers and technical notes. Articles and papers draw attention to issues impacting on the data, such as changes to the classifications, systems, concepts or standards, major data revisions, and changes in data dissemination practices. Spotlight articles and technical notes are written on a regular basis to inform users of emerging issues and methodological changes, including their impact on the national accounts. They are commonly released in the national accounts’ publications. Information papers and research papers report on various aspects of research undertaken on topics relevant to the national accounts. A list of articles is maintained in the national accounts theme page on the ABS website.
24.90 Coherence of statistical data includes coherence between different data items pertaining to the same point in time; coherence between the same data item for different points in time; and international coherence.
24.91 Judgements can be made as to the extent of error associated with an estimate, by being aware of the factors which influence coherence. These judgements can be enhanced by comparing estimates that should be conceptually identical, or by comparing estimates where a particular relationship between the estimates could be expected. In other words, the extent to which a set of statistics are coherent can provide a guide to the accuracy of the statistics. However, it should be noted that a coherent set of statistics is not necessarily an accurate set, as the statistics that are being compared may suffer from similar magnitudes of error with the errors being in the same direction.
24.92 Users sometimes need to draw on different sets of statistical information derived from different sources and at different times. Appropriate use is facilitated if information can be validly compared with other related data sets. This is achieved using common, or at least comparable, concepts and methodologies across products.
Standard frameworks, concepts, variables and classifications
24.93 Frameworks, concepts, and classifications exist to ensure that the target of measurement is consistent across statistical programs, that consistent terminology is used across programs and that the qualities being estimated bear known relationships to each other. The realisation of this element is normally through the adoption and use of frameworks such as the SNA and ASNA. The issue of international comparability is addressed by considering the adherence of the standards adopted to the international standards as contained in the SNA.
24.94 Although there are a wide range of uses for economic data, national accounts data requirements are a key guiding principle behind most of the Bureau's economic collections. Coherence has been aided by the harmonisation of the various international standards for economic statistics with the SNA. Where administrative data are used, special care is taken to ensure the correct application of standards, and the identification of possible data gaps and overlaps. Appropriate adjustments are made to align the data in cases where administrative data are known to differ from national accounts standards.
Common frames, methodologies and systems
24.95 Data coherence is improved through the development and use of common frames, methodologies and systems for data collection and processing. For example, the use of a common business register across all business surveys ensures that differences in frame coverage do not introduce inconsistencies in data. Differences due to response error are minimised using commonly formulated questions when the same variables are being collected in different surveys. Another strategy is to use common methodology and systems for the various processing steps of a survey, to ensure that these operations do not introduce discrepancies in data.
Data comparison and integration
24.96 A key aspect of the national accounts process is the confrontation of data from different sources, and the subsequent reconciliation. At its most formal, this process occurs within the annual balancing of the Supply and Use tables, which are used to benchmark the national accounts. It also includes the coherence between the national accounts estimates and the various partial indicators of economic activity published by the ABS and other agencies.
24.97 The confrontation of data in the annual Supply and Use framework provides a systematic, if somewhat complex, means of checking the coherence of much of the source data used in the national accounts. In the process, the source data are subject to examination and adjustment in order to achieve coherence in the annual national accounts’ statistics. While coherence is achieved, it cannot be claimed that the results are necessarily accurate. S-U tables are data intensive, and some product information has to be synthesised. More suitable source data may become available later, leading the national accounts compiler to take different decisions to balance the supply and use of products. The balancing process itself also has some limitations because it is not possible to thoroughly assess every imbalance. It can be expected that any significant inconsistencies in major data items will become apparent in the balancing process. Despite some limitations, the Supply and Use methodology is recognised internationally as the best means of checking the coherence of data and assuring the accuracy of the national accounts.
24.98 Considerable attention is given to confronting the consistency of the data used to compile the quarterly accounts, although no attempt is made to completely balance quarterly data. The ABS has developed a quarterly Supply and Use model to help identify internal inconsistencies, and to help focus investigations by the national accounts’ compilers. Timing and other inconsistencies in the data remain after this process and are reflected in the statistical discrepancies that are shown explicitly in the Australian national accounts. As previously mentioned, the statistical discrepancies are only a partial measure of coherence in the data – they represent an aggregation of all the positive and negative discrepancies implicit in the large amount of source data which are used to compile quarterly GDP.
24.99 Another aspect of coherence relates to the consistency of the national accounts with other economic data published by the ABS and other organisations. The ABS publishes a large amount of data on various aspects of the economy. To the extent that many of these are also inputs to the national accounts, it could be expected that they would be coherent with the national accounts. Processes have been implemented to achieve as much consistency as possible. Representatives of the economic collections and economic accounts areas meet formally and regularly to discuss and come to agreement on the statistical treatment and dimensions of recent economic events, such as privatisations, major construction and resource projects, asset purchases and other issues surrounding the reconciliation of economic data more generally.
24.100 A formal process has also been established to involve national accounts staff in the clearance of some quarterly economic indicator surveys that are published before the national accounts. This gives the national accountants an opportunity to ask questions and input any additional information gained from the wider perspective of other data available for the wider economy prior to finalisation of the results. As a result of this formal process, the national accounts and the partial indicators data for recent quarters should be consistent, although some scope, concepts and other differences may mean that they are not exactly equal. Over time, benchmarking procedures may lead to divergence between the national accounts and the sub-annual partial indicator statistics.
24.101 There have been a few occasions where officials and economic commentators have questioned the consistency between the national accounts and other economic data. An example is the complex relationship between employment growth and output growth as measured by GDP, and the data series can occasionally move in directions that appear counterintuitive. The ABS has undertaken an investigation into the relationship between GDP and employment; see the June 2005 release of Australian National Accounts: National Income, Expenditure and Product for more detail. The relationship between GDP growth and growth in tax revenues has also been questioned. On this latter point, it should be noted that there are many differences between operating surplus from the national accounts and taxable income. For example, taxable income includes realised capital gains but deducts net interest payments. Operating surplus excludes capital gains and losses and does not deduct net interest payments. The treatment of these elements can result in different year-to-year movements, and also in longer term growth patterns.
24.102 Quality gates (QGs) link back to the ABS Data Quality Framework dimensions of institutional environment, relevance, timeliness, accuracy, coherence, interpretability and accessibility to ensure that all dimensions of quality have been considered and maintained in the production of the statistical outputs.
24.103 Quality gates reflect:
- the institutional environment through their implementation as a statistical risk mitigation strategy, so that the reputation of the organisation is protected
- timeliness and relevance by ensuring that they are placed at appropriate junctures in the process where they can influence the direction and outcome of the quality of the process. Examples include: preventing delays by identifying problems in time; confirming that the process is okay at a given point in time in order to continue to the next steps; or confirming that the output requirements are achievable from the design of the inputs.
- accuracy and coherence through the use of quality measures and their corresponding tolerance levels
- interpretability by encouraging documentation to ensure knowledge management and a shared understanding across all stakeholders of the quality gate and the underlying processes
- accessibility through the provision of information on the quality of the process at the individual components. In some cases, the desired information may not be available due to current reporting limitations in the processes (such as a particular quality measure) but should be acknowledged as a requirement for future development opportunities.
24.104 The national accounts have implemented several quality gates throughout the compilation process. Quality gates are applied as the source data is received and entered the national accounts processing system, to ensure fitness-for-purpose. Checks at this stage ensure that the dataset is complete; unusual movements are highlighted; and that revisions adhere to the revisions policy.
24.105 Quality gates are vital throughout the editing stage to ensure the data is being processed correctly. Checks at this stage ensure or highlight:
- additivity where applicable
- correct application of benchmarks
- missing and negative values
- unusual movements
- acceptability of revisions.
24.106 Quality gates are also applied when analysing the data. They are useful to highlight inconsistencies and incoherence in the data.
24.107 Revisions in relation to the national accounts may be defined as the differences between a published sequence of estimates for a given reference period for the same national accounting variable over a particular period of time. Revisions arise from the progressive incorporation of more up to date data, re-weighting of chain volume series and reassessment of seasonal factors, and from time to time the introduction of new accounting concepts and improved data sources and methods.
24.108 Revisions analysis is concerned with the reliability of initial (or subsequent) estimates rather than the accuracy of estimates. Accuracy is always the main focus of statistical agencies. However, a standard accuracy measure, such as the standard error of GDP, has been found impossible to produce in practice because of the complicated compiling process and aggregation structure of the national accounts. For example, a Eurostat (1999) task force concluded that:
. . . even under the most optimistic assumption that confidence intervals can be provided for all basic sources, objective error margins for national accounts aggregates appear to be out of reach. The problems posed by identifying, measuring and aggregating all errors that remain after adjustments are made at the various compilation stages seem insurmountable.
24.109 Reliability refers to the ability of the successive vintages of national accounts estimates to present a consistent, reliable picture of the economy, as estimates are revised to incorporate increasingly comprehensive and improved source data. It is therefore a relative measure rather than an absolute one. Revisions analysis is often employed as a diagnostic tool to indicate possible sources of unreliability, but the results should never be used as an adjustment factor in compiling or interpreting the national accounts.
24.110 Revisions should not be seen as synonymous with error. They are an inevitable consequence of the national accounts process. Revisions reflect both the complexity of economic measurement, and the need to provide economic policy advisers and other users with timely estimates, in order to maximise their use in analysis of current economic conditions. A major reason for revisions to initial estimates is the trade-off between timeliness, accuracy and reliability. For example, delaying the release of data by a year or more would not mean that revisions are no longer required. There are several reasons:
24.111 The national accounts process is aimed at bringing together sets of accounts that are coherent for the whole nation, unlike in business accounting, where the aim is to record accurately the transactions and financial position of a particular business entity in isolation. It is, in effect, a quadruple accounting system, and achieving coherence between the transactions of economic agents is an iterative process proceeding over a number of years.
24.112 Some of the most complete data are only available every few years (such as the Census of Population and Housing).
24.113 Seasonally adjusted and trend estimates are updated as information from new quarters is used to improve estimates of the trend, seasonal and irregular components of each time series.
24.114 Improvements to data sources and estimation methods are backcast to earlier periods where possible, as well as occasional changes to national accounting standards.
24.115 ABS and international data quality assessment frameworks include revisions history as one of the indicators of quality, linking it with the accuracy element. Users need to be aware of the potential for revisions. They need to satisfy themselves that initial estimates provide an acceptable indication of later estimates based on more complete data, which has been subjected to coherence checks within the national accounts system. As part of an overall quality assessment program, national accounts compilers are encouraged to regularly conduct an analysis of revisions in order to inform users, and to help guide the continuous quality improvement process within statistics agencies.
24.116 The ABS has published studies of revisions in the past, as have a number of other national statistics agencies. Since 2005, the OECD have occasionally published data on revisions to quarterly GDP and other economic indicators in selected countries. The database has been designed for a limited number of key economic statistics together with an analytical spreadsheet. It is also designed to assist compilers and analysts in the derivation of a standard set of revisions measures, using either OECD data or their own data. The database facilitates international comparisons of revisions.
24.117 The remainder of this section describes:
- the national accounts compilation process;
- the revisions policy; and
- the availability of information on revisions.
National accounts compilation process
24.118 The ABS has implemented a set of rules to manage the timing of revisions to the national accounts. These rules are based around the national accounts compilation process, which itself reflects the availability of new or revised source data, as well as operational factors.
24.119 The basic accounting and price index data from the quarterly surveys can normally be expected to become quite firm the quarter following the initial estimate. The national accounts process requires that the estimates derived from these sources be subject to annual benchmarking. The results of the benchmarking process are first published in the September quarter issue of the quarterly national accounts, coinciding with the availability of balanced annual current price and volume data from the Supply and Use tables.
24.120 The Supply and Use system progressively incorporates business accounts and other data from ABS annual surveys, as well as the taxation system. A balancing process is undertaken to achieve consistency between the supply and use of products in the economy, in both current price and volume terms. Data inconsistencies are reviewed and have to be resolved by altering some of the basic data. Quarterly national accounts estimates are benchmarked to successive vintages of annual data in order to maintain consistency within the national accounts system. This process also introduces revisions to quarterly growth rates in years either side of the new or revised annual data.
24.121 The benchmarking procedure used by the ABS is a mathematical routine designed to equate the quarterly and annual benchmarked estimates, while minimising the impacts on existing quarterly growth rates. Using this procedure, revisions to an annual estimate have the potential to impact on growth rates for all the quarters in that year, in the two previous years and two forward years. Impacts of benchmarking on individual quarter growth rates could be reduced by adopting a simple pro rata procedure, but this would force all of the revision to growth rates into the September quarter. The optimising procedure used is widely considered to produce superior estimates of quarterly growth rates, but at the expense of more widespread revisions. Revisions to annual estimates result from the progressive incorporation of more complete source data balanced in the Supply and Use system, and revisions to quarterly estimates result from the process of benchmarking. In this way, a revisions process is inherent within the national accounts process because more comprehensive and reliable data only become available with a considerable lag.
24.122 Typically, the national accounts process for a given quarter for original current price and volume data proceeds as follows:
- The initial quarterly estimate is based on preliminary quarterly survey data.
- The later vintage quarterly estimates up to the June quarter issue of the national accounts are based on more complete or 'final' quarterly survey data.
- At the end of the current financial year (June quarter) the annual estimates for that year are derived initially as the sum of the four quarters.
- In the September quarter, the reference year values for the whole time series of chain volume estimates are advanced one year to the current price values applying in the previous financial year. Re-referencing impacts data levels, but not growth rates. The base year weights for the current and previous year chain volume estimates are also moved forward. This can result in a revision to growth rates in those years.
- In the September quarter, the quarterly estimates are subject to a benchmarking process to align them with annual current price and volume data that has been balanced in the supply and use system.
24.123 As noted in the Methodology subsection above, the X-11 seasonal adjustment approach used by the ABS decomposes original current price and chain volume estimates into a combination of modelled trend, seasonal and irregular components, which are used to produce seasonally adjusted and trend estimates. The addition of new quarters and data revisions to previous quarters results in a re-estimation of the seasonal factors and underlying trends in the data due to the end-point problem. Revisions to growth rates will result from this process. The usual concurrent seasonal adjustment method involves re-analysing and re-estimating seasonal factors for the whole time series with the addition of each new quarter, and, therefore, revisions to seasonally adjusted and trend estimates potentially flow through with the addition of each new quarter. These revisions tend to be largest over the most recent few years. Components for which seasonal factors are re-analysed and re-estimated annually will show revisions at the time this occurs. In other quarters, the latter approach will not revise seasonal factors, but any revisions to original data may still flow through to seasonally adjusted and trend estimates.
24.124 As a consequence of this process, the following revisions policy applies for GDP and components in normal circumstances.
24.125 The September quarter releases normally allow up to sixteen quarters of revision for original current price and chain volume estimates; that is, the quarters of the current year plus the quarters of the previous three financial years. In these issues, updated annual Supply and Use benchmarks will be introduced.
24.126 In other quarterly releases, revisions to original current price (and chain volume) estimates are restricted to the current year and the previous financial year.
24.127 For chain volume and price measures, the annual re-referencing of the series each September quarter will cause revisions to the levels for the entire series. Re-referencing does not affect percentage movements, but the introduction of updated price weights for the most recent periods could affect growth rates for these periods.
24.128 Revisions resulting from seasonal re-analysis are allowed to flow through to the whole seasonally adjusted time series at the time the seasonal re-analysis is undertaken. This occurs each quarter for series using concurrent seasonal adjustment, and annually for series using annually updated seasonal factors.
24.129 From time to time, the ABS will also implement an 'historical revision' outside of the normal cycle, whereby the whole historical time series is opened up for revision. This allows substantial updates to concepts, classifications, methods and data sources to be reflected across all time periods to which they apply. Since the adoption of the 2008 SNA in 2008-09, significant historical revisions have been made in 2010-11 and 2016-17.
Availability of information on revisions
24.130 At present, the following indicative information is provided:
- Australian National Accounts: National Income, Expenditure and Product – Tables detail revisions to the recent eight quarters for GDP and its expenditure, income and industry components. Additionally, the publication contains commentary in the 'Analysis and comments' section on major sources of revisions, and the ‘Explanatory notes’ provide some discussion of the sources of revisions and the accuracy of the estimates more generally.
- Australian System of National Accounts – The major sources of revisions are documented within the 'Analysis of results' section.
- Occasionally, more detailed studies have been made available.
24.131 Analysis of the revisions to the national accounts is undertaken regularly, and the results are made available to users. This is in keeping with developing the Bureau’s broader thinking about data quality declarations for statistical outputs.