WHAT APPROACH DID THE ABS TAKE IN PRESENTING PROGRESS DATA?
"Not everything that counts can be counted, and not everything that can be counted counts."
Albert Einstein
There are many ways in which data about progress can be presented. These might be summarised into two main approaches:
An accounting approach, which provides a set of consistent accounts which can then, if needed, support the production of a single progress indicator, and
A suite of indicators, or "dashboard" approach, which presents selected informative progress indicators side by side
MAP takes the latter approach - a dashboard approach. It presents data on key aspects of life in Australia and discusses the links between these areas. In this way, readers can review progress across the social, economic and environmental domains and understand the issues unique to each.
The dashboard approach encourages readers to consider the indicators and make their own assessment of whether Australia is, on balance, progressing and at what rate. By comparison, although a useful analytical tool, the accounting approach takes those decisions out of the hands of the general public, by applying weights to each factor in each domain before the data is presented.
In using a dashboard approach, the ABS has avoided the complexity and contestability of a comprehensive accounting system, which is complicated to both compile and interpret, and the potential oversimplification involved in presenting a single progress indicator.
An accounting approach
In an accounting approach, all relevant social, environmental and economic factors would be considered in terms of one measurement unit - usually monetary. Social, economic and environmental "accounts" would then be brought together in one unified system of accounts.
In this approach, the data can be presented in a set of accounts that are consistent with one another, or, because of this consistency, the data can be combined into one or more single numbers. Although this process is mechanically straight forward, it is complicated by the conceptual factors outlined below.
The accounting approach is often put forward when measures of progress are being considered, because it seems like a natural extension of the approach that has long been used to monitor the economy. Many of the ABS Main Economic Indicators released monthly, quarterly or annually are numbers drawn from the internationally applied System of National Accounts (SNA). Readers will be familiar with some of these, such as the Balance of Payments (BOP).
The accounting approach is attractive because it puts all the numbers on the same basis so they relate to one another and can be added together, or aggregated. These aggregate numbers can then be readily monitored to see whether they are increasing or decreasing over time.
The overall information can also be summarised by combining aggregates from different accounts into one number, known as a composite indicator. An example of a composite indicator from the SNA is Gross Domestic Product (GDP). Because composite indicators are fairly easily digested, they lend themselves to publication and media commentary, and can be useful in encouraging debate.
However, composite indicators usually focus on a very particular aspect of life - in the case of GDP, on market production and consumption. Using the analogy of a road map, you might say composite indicators show where the roads are leading - the direction of a given factor - but do not indicate the quality of the road, the ideal route, the depletion of resources used in building the roads, or the scenery.
There are other disadvantages and complexities involved in taking the accounting or single number approach.
First, it is an adaption of a system designed to measure the market economy - a phenomenon that is quite different from the broader economy or the complete environmental and social system that we hope to measure when assessing progress.
In the broader economy, for example, public services such as health care or educational services may not be priced in the same way as private services provided by business for the market. For example, public services are often priced in a way that assists all citizens to access them rather than priced only for market competitiveness. So, the true price of non-market economic activity is unobservable and its outputs are often difficult to identify and therefore measure.
Further, economic goods and services are valued in monetary terms by observing the price paid for these in the market. However, market prices don't unerringly reflect the value of particular goods or services when considered in a wider societal or environmental context (Gittens 2010).
Another complication is that many social and environmental factors are inherently difficult to value in monetary terms. This is partly because these factors are not traded in the market economy - for example, community connectedness or natural habitats and ecosystems. It has only been recently that some environmental resources have been priced in a way that sets their true value in market terms, and there is much contemporary discussion about how other factors affecting environmental wellbeing can be usefully priced, for example, carbon and greenhouse gases.
But it is also because, while unquestionably valuable, such factors are not always material, and therefore able to be put into monetary terms, for example, the quality of human relationships or the beauty of the natural environment.
To allocate a monetary value to intrinsically valuable but "priceless" factors would involve a complex analysis of social values which is difficult to undertake with the objectivity needed for statistical purposes: the ABS is not ideally placed to undertake such analysis.
There are also technical complexities involved in combining a range of social, environmental and economic measures into a single composite indicator or number. The components will usually be measured in different units (e.g. years of life expectancy, dollars of income, numbers of hospital beds, rates of suicide deaths, tonnes of greenhouse gases). Although it may be possible to express these different factors in some common way to make them comparable with one another (e.g. as a rate), this again involves making complex social value judgements about the relative importance of each. For instance, how much money is the rate of infant mortality worth?
There is also the view that aggregate indicators mask important nuances in the information underlying the aggregate, and that a certain level of disaggregation of information is required to inform the community and the government in ways that allow it to respond. For example, it is important to understand health outcomes for sub-populations such as Aboriginal and Torres Strait Islander peoples as well as health outcomes for the total population. It is also important to know the extent to which natural resources are being exhausted, not just whether the production and consumption supported by these resources is growing.
In summary, a comprehensive accounting system across social, economic and environmental domains is complicated to compile and interpret for anyone wishing to quickly form an overall view about Australia's progress. At the same time, the very simplicity of condensing progress into a single number runs the risk of oversimplifying a complex system. There is a danger that a composite indicator will give potentially misleading signals depending on the context in which it is used.
These are some reasons why the ABS has not developed or adopted such a system for measuring progress. Composite indicators can be a valuable complement to summary indicators and the ABS supports work being undertaken in this field. For instance, the Australian National Development Index sets out to provide a consistent set of social, economic and environmental indicators that can support a single number assessment of Australia's wellbeing in its broadest sense. The Australian National Development Index project will potentially provide a single number that can moderate analysis that currently only has at its disposal narrow economic indicators such as GDP.
A more in-depth analysis of economic accounting in the context of measuring progress can be found in the Final Report by the Commission on the Measurement of Economic Performance and Social Progress by Joseph E Stiglitz, Amartya Sen and Jean-Paul Fitoussi.
A suite of indicators approach
In a suite of indicators approach, different kinds of measures relating to various topics (dollars, numbers, rates, amounts) are presented side by side.
The rationale for using the suite of indicators approach can be summed up by the example of a car dashboard. When travelling from one place to another, a driver needs to understand a number of factors to make informed decisions: for example, the distance travelled, the time taken, the temperature of the engine and the amount of petrol available to continue.
All these factors need to be presented separately to be of value, as combining them into a single number would reduce the driver's ability to understand the variables and make important judgements. For example, an overheating car must be actioned regardless of the amount of petrol left, or the distance travelled.
This is why the suite of indicators approach is often referred to as a "dashboard" approach. MAP uses this approach. It presents data on key aspects of life in Australia side-by-side and discusses the links between them.
The dashboard approach encourages readers to consider the indicators together and make their own evaluations of whether Australia is, on balance, progressing and at what rate. By comparison, accounting approaches take those decisions out of the hands of the general public, by applying weights to each factor before the data is presented.
The suite-of-indicators approach makes no single overall assessment about whether the array of statistical indicators presented implies that life is getting better or worse. Instead, it leaves each individual reader to apply their own values and preferences to the evidence, and to arrive at their own overall assessment of national progress.
Presenting too many disparate indicators together can, however, reduce the effectiveness of the dashboard, as these may not be easily assimilated and weighed against each other. This is why MAP 2010 presents a very reduced set of indicators on its home page, then more detailed data as the reader follows links through the product.
In our view, this approach strikes a balance between the complexity of a larger dashboard or complex accounting system and the potential oversimplification of a single number.
"Too much and too long, we seem to have surrendered community excellence and community values in the mere accumulation of material things. Our gross national product - if we should judge America by that - counts air pollution and cigarette advertising, and ambulances to clear our highways of carnage. It counts special locks for our doors and the jails for those who break them. It counts the destruction of our redwoods and the loss of our natural wonders in chaotic sprawl. It counts napalm and the cost of a nuclear warhead, and armoured cars for police who fight riots in our streets. It counts Whitman's rifle and Speck's knife, and the television programs which glorify violence in order to sell toys to our children. Yet the gross national product does not allow for the health of our children, the quality of their education, or the joy of their play. It does not include the beauty of our poetry or the strength of our marriages; the intelligence of our public debate or the integrity of our public officials. It measures neither our wit nor our courage; neither our wisdom nor our learning; neither our compassion nor our devotion to our country; it measures everything, in short, except that which makes life worthwhile. And it tells us everything about America except why we are proud that we are Americans."
Robert F Kennedy March 18, 1968, Address, University of Kansas
Print Page
WHAT APPROACH DID THE ABS TAKE IN PRESENTING PROGRESS DATA?
"Not everything that counts can be counted, and not everything that can be counted counts."Albert Einstein
There are many ways in which data about progress can be presented. These might be summarised into two main approaches:
- An accounting approach, which provides a set of consistent accounts which can then, if needed, support the production of a single progress indicator, and
- A suite of indicators, or "dashboard" approach, which presents selected informative progress indicators side by side
MAP takes the latter approach - a dashboard approach. It presents data on key aspects of life in Australia and discusses the links between these areas. In this way, readers can review progress across the social, economic and environmental domains and understand the issues unique to each.The dashboard approach encourages readers to consider the indicators and make their own assessment of whether Australia is, on balance, progressing and at what rate. By comparison, although a useful analytical tool, the accounting approach takes those decisions out of the hands of the general public, by applying weights to each factor in each domain before the data is presented.
In using a dashboard approach, the ABS has avoided the complexity and contestability of a comprehensive accounting system, which is complicated to both compile and interpret, and the potential oversimplification involved in presenting a single progress indicator.
An accounting approach
In an accounting approach, all relevant social, environmental and economic factors would be considered in terms of one measurement unit - usually monetary. Social, economic and environmental "accounts" would then be brought together in one unified system of accounts.
In this approach, the data can be presented in a set of accounts that are consistent with one another, or, because of this consistency, the data can be combined into one or more single numbers. Although this process is mechanically straight forward, it is complicated by the conceptual factors outlined below.
The accounting approach is often put forward when measures of progress are being considered, because it seems like a natural extension of the approach that has long been used to monitor the economy. Many of the ABS Main Economic Indicators released monthly, quarterly or annually are numbers drawn from the internationally applied System of National Accounts (SNA). Readers will be familiar with some of these, such as the Balance of Payments (BOP).
The accounting approach is attractive because it puts all the numbers on the same basis so they relate to one another and can be added together, or aggregated. These aggregate numbers can then be readily monitored to see whether they are increasing or decreasing over time.
The overall information can also be summarised by combining aggregates from different accounts into one number, known as a composite indicator. An example of a composite indicator from the SNA is Gross Domestic Product (GDP). Because composite indicators are fairly easily digested, they lend themselves to publication and media commentary, and can be useful in encouraging debate.
However, composite indicators usually focus on a very particular aspect of life - in the case of GDP, on market production and consumption. Using the analogy of a road map, you might say composite indicators show where the roads are leading - the direction of a given factor - but do not indicate the quality of the road, the ideal route, the depletion of resources used in building the roads, or the scenery.
There are other disadvantages and complexities involved in taking the accounting or single number approach.
First, it is an adaption of a system designed to measure the market economy - a phenomenon that is quite different from the broader economy or the complete environmental and social system that we hope to measure when assessing progress.
In the broader economy, for example, public services such as health care or educational services may not be priced in the same way as private services provided by business for the market. For example, public services are often priced in a way that assists all citizens to access them rather than priced only for market competitiveness. So, the true price of non-market economic activity is unobservable and its outputs are often difficult to identify and therefore measure.
Further, economic goods and services are valued in monetary terms by observing the price paid for these in the market. However, market prices don't unerringly reflect the value of particular goods or services when considered in a wider societal or environmental context (Gittens 2010).
Another complication is that many social and environmental factors are inherently difficult to value in monetary terms. This is partly because these factors are not traded in the market economy - for example, community connectedness or natural habitats and ecosystems. It has only been recently that some environmental resources have been priced in a way that sets their true value in market terms, and there is much contemporary discussion about how other factors affecting environmental wellbeing can be usefully priced, for example, carbon and greenhouse gases.
But it is also because, while unquestionably valuable, such factors are not always material, and therefore able to be put into monetary terms, for example, the quality of human relationships or the beauty of the natural environment.
To allocate a monetary value to intrinsically valuable but "priceless" factors would involve a complex analysis of social values which is difficult to undertake with the objectivity needed for statistical purposes: the ABS is not ideally placed to undertake such analysis.
There are also technical complexities involved in combining a range of social, environmental and economic measures into a single composite indicator or number. The components will usually be measured in different units (e.g. years of life expectancy, dollars of income, numbers of hospital beds, rates of suicide deaths, tonnes of greenhouse gases). Although it may be possible to express these different factors in some common way to make them comparable with one another (e.g. as a rate), this again involves making complex social value judgements about the relative importance of each. For instance, how much money is the rate of infant mortality worth?
There is also the view that aggregate indicators mask important nuances in the information underlying the aggregate, and that a certain level of disaggregation of information is required to inform the community and the government in ways that allow it to respond. For example, it is important to understand health outcomes for sub-populations such as Aboriginal and Torres Strait Islander peoples as well as health outcomes for the total population. It is also important to know the extent to which natural resources are being exhausted, not just whether the production and consumption supported by these resources is growing.
In summary, a comprehensive accounting system across social, economic and environmental domains is complicated to compile and interpret for anyone wishing to quickly form an overall view about Australia's progress. At the same time, the very simplicity of condensing progress into a single number runs the risk of oversimplifying a complex system. There is a danger that a composite indicator will give potentially misleading signals depending on the context in which it is used.
These are some reasons why the ABS has not developed or adopted such a system for measuring progress. Composite indicators can be a valuable complement to summary indicators and the ABS supports work being undertaken in this field. For instance, the Australian National Development Index sets out to provide a consistent set of social, economic and environmental indicators that can support a single number assessment of Australia's wellbeing in its broadest sense. The Australian National Development Index project will potentially provide a single number that can moderate analysis that currently only has at its disposal narrow economic indicators such as GDP.
A more in-depth analysis of economic accounting in the context of measuring progress can be found in the Final Report by the Commission on the Measurement of Economic Performance and Social Progress by Joseph E Stiglitz, Amartya Sen and Jean-Paul Fitoussi.
A suite of indicators approach
In a suite of indicators approach, different kinds of measures relating to various topics (dollars, numbers, rates, amounts) are presented side by side.
The rationale for using the suite of indicators approach can be summed up by the example of a car dashboard. When travelling from one place to another, a driver needs to understand a number of factors to make informed decisions: for example, the distance travelled, the time taken, the temperature of the engine and the amount of petrol available to continue.
All these factors need to be presented separately to be of value, as combining them into a single number would reduce the driver's ability to understand the variables and make important judgements. For example, an overheating car must be actioned regardless of the amount of petrol left, or the distance travelled.
This is why the suite of indicators approach is often referred to as a "dashboard" approach. MAP uses this approach. It presents data on key aspects of life in Australia side-by-side and discusses the links between them.
The dashboard approach encourages readers to consider the indicators together and make their own evaluations of whether Australia is, on balance, progressing and at what rate. By comparison, accounting approaches take those decisions out of the hands of the general public, by applying weights to each factor before the data is presented.
The suite-of-indicators approach makes no single overall assessment about whether the array of statistical indicators presented implies that life is getting better or worse. Instead, it leaves each individual reader to apply their own values and preferences to the evidence, and to arrive at their own overall assessment of national progress.
Presenting too many disparate indicators together can, however, reduce the effectiveness of the dashboard, as these may not be easily assimilated and weighed against each other. This is why MAP 2010 presents a very reduced set of indicators on its home page, then more detailed data as the reader follows links through the product.
In our view, this approach strikes a balance between the complexity of a larger dashboard or complex accounting system and the potential oversimplification of a single number.
"Too much and too long, we seem to have surrendered community excellence and community values in the mere accumulation of material things. Our gross national product - if we should judge America by that - counts air pollution and cigarette advertising, and ambulances to clear our highways of carnage. It counts special locks for our doors and the jails for those who break them. It counts the destruction of our redwoods and the loss of our natural wonders in chaotic sprawl. It counts napalm and the cost of a nuclear warhead, and armoured cars for police who fight riots in our streets. It counts Whitman's rifle and Speck's knife, and the television programs which glorify violence in order to sell toys to our children. Yet the gross national product does not allow for the health of our children, the quality of their education, or the joy of their play. It does not include the beauty of our poetry or the strength of our marriages; the intelligence of our public debate or the integrity of our public officials. It measures neither our wit nor our courage; neither our wisdom nor our learning; neither our compassion nor our devotion to our country; it measures everything, in short, except that which makes life worthwhile. And it tells us everything about America except why we are proud that we are Americans."Robert F Kennedy March 18, 1968, Address, University of Kansas