|Page tools: Print Page|
Building a national statistical agency: From the Commonwealth Bureau of Census and Statistics to the Australian Bureau of Statistics
Australia has produced statistics since the beginning of European settlement. Initially, progress in the colonies was monitored in 'the mother country' through the yearly dispatch of statistical details covering mainly the population and availability of food. Over the years the statistical content became more pervasive, as populations grew, colonies multiplied and farming emerged, followed by commerce. In 1822 the British Colonial Office set up a more formal system, known as the 'Blue Books', in which statistical requirements were prescribed. This led to the development, in the Australian colonies, of statistical officers and in time, statistical offices. By the end of the 19th century, each self-governing colony had a functioning statistical office headed by a 'Statist'. Although efficacy varied considerably between colonies, some produced statistics of a very high standard.
To prepare for the Federation Census scheduled for 1901, (the first census for the new nation) a Conference of Statisticians was held in March 1900 in Sydney. Timothy Augustine Coghlan, the NSW Statist, reported to Sir William Lyne (NSW premier):
Although Conferences of Statisticians were held in 1902 and 1903 to discuss unifying statistics, progress towards unification was very slow. In framing the Constitution, the founding fathers had given the Parliament '... power to make laws for the peace, order and good government of the Commonwealth with respect to: ... (xi) census and statistics'. Rather vague on detail, the Constitution leaves the best way to exercise this power to the judgement of Parliament. The Government determined that a Commonwealth Bureau of Census and Statistics (CBCS) was required to ensure fair treatment of the states.
The Census and Statistics Act (Cwlth) was given assent on 8 December 1905. Under the Act, the census and some Commonwealth statistics became Commonwealth functions. Other general statistics were still to be collected by the states. A role remained for the Conference of Statisticians.
George Handley Knibbs was appointed in 1906 as Australia's first Commonwealth Statistician. Knibbs was given the responsibility to set up the CBCS and to unify the states' statistical collections.
The infant national statistical system
It soon became clear that the goal of uniform national statistics was not to be easily achieved and the CBCS found it necessary to undertake original compilations, and to take over responsibility for some branches of statistics where it was obvious that the state bureaus were either unable to provide the data in reasonable time, or lacked the will as the data was not critical to state priorities. The first of these was commerce statistics where it was arranged that shipping returns should be sent directly to the CBCS.
The second of these was vital statistics. It was quickly realised that these would be very late and meagre, and possibly inconsistent from one state to the next, if relying on the state bureaus. So it was arranged for state registers of vital statistics to be made available direct to the CBCS.
Within the first decade, the CBCS was also producing banking, insurance, cost of living, and labour and wages statistics. However statistics of production, that is, agricultural, pastoral, dairying, mining, manufacturing, forestry, fishing etc. continued to be controlled by states.
Initial attempts at unification
Charles Henry Wickens became the second Commonwealth Statistician in 1922. He had come from the Western Australian Government Statistician's Office, where he worked on the 1901 Census, and had previously been Commonwealth Director of Census. He had experience in working directly with the states towards a common goal.
In May 1923 a Premiers' Conference again passed a resolution in favour of creating one statistical authority for the whole of Australia. Details were to be decided later at a Conference of Statisticians, October 1923.
Opinion at the Conference was divided:
Following the 1923 Conference, Wickens and Giblin negotiated a fairly straightforward path towards transfer. The Statistical Bureau (Tasmania) Act 1924 contained precise details regarding the statistical responsibilities of the Tasmanian office and the duties to the Tasmanian state government and the Commonwealth Government.
Meanwhile, the states in the non-unification bloc maintained their stance, despite changes of government in two states (South Australia and Western Australia). By June 1924 they had all formally declined. Queensland, though originally agreeable, also declined.
Victoria came closest to transferring. The process was halted by the Commonwealth Government in September 1925 due to funding constraints. The Depression, and then the ill-health of Wickens, meant that he did not have the chance to finish the task of unifying Australian statistical offices.
The Tasmanian connection
Following the departure of Wickens, Giblin was appointed acting Commonwealth Statistician and Chief Economic Adviser in 1931. His appointment was on the understanding '... that I should be sufficiently relieved from administrative routine to be able to give the greater part of my time to special investigations required by the Minister' (LF Giblin quoted in Forster and Hazlehurst,Year Book Australia 1988). During his tenure he appointed Roland Wilson as economist with the idea of quietly grooming him to be Commonwealth Statistician in the near future. He also encouraged several other young men working in the Bureau to undertake a university course. Among them were KM Archer and JP O'Neill who both went on to be Commonwealth Statisticians.
Edward Tannock McPhee, also originally from Tasmania, was appointed Commonwealth Statistician in 1932, seemingly with the aim of remaining only as long as his health permitted and hopefully long enough to get Wilson 'bedded down' (Keith Archer, interviewed in 1971). Wilson was appointed Commonwealth Statistician following McPhee's retirement in 1936.
In setting up this succession, whether deliberately or just through his ability to choose the right people, Giblin was to have enormous influence on the Bureau's direction long after he left. His emphasis on economics and economic statistics put the Bureau in an influential position as the Australian economy diversified and gathered pace, and demand for economic statistics grew.
Expanding role of the Bureau
As Commonwealth Statisticians, Giblin, McPhee and Wilson each focused on economic and statistical issues and chose not to take on the task of unification of the remaining state statistical offices.
Early in 1941 Wilson was co-opted into other war-related duties, and Stanley Carver, the NSW Statistician, became acting Commonwealth Statistician. Though Wilson was to return for short periods several times after the War, this effectively marked the beginning of the end of the Wilson period.
The immediate post-War period was characterised by Keynesian-style management of the economy. The pre-War work of Giblin and his protégés in developing economic statistics, such as putting values on theoretical concepts like national income and investment, placed the Bureau in a sound position to respond to post-War demands for economic statistics. As post-War reconstruction took off, governments were interested in measuring the success of their policies.
Amalgamation finally achieved
Under the stewardship of Carver, amalgamation of the Commonwealth and state statistical offices was finally achieved. The process of bringing the remaining state bureaus into the CBCS was initiated by Prime Minister Chifley in 1949, in discussions with the premiers, and continued under Prime Minister Menzies.
Various arrangements for war-time management of the economy had resulted in increasing responsibility for the Commonwealth Government and a decreasing role for state governments. This was compounded by the move to the Commonwealth Government, of responsibility for income tax collection. In an environment of greatly reduced budgets and no involvement in economic management, state governments' requirements for statistics diminished, and so, consequently, did the capacity of state statistical bureaus to produce them.
In the post-War period, as Commonwealth demands for statistics grew, the duplicative and cumbersome system started to crack at the seams. Conference of Statisticians minutes of 1945, 1949, 1950 and 1953 all commented on increasing demands for statistics, and lack of resources. The 1950 Conference also noted 'with approval', moves initiated to prevent various Commonwealth agencies from collecting their own statistics without reference (or deference) to the CBCS. Both issues highlighted the need to have clear authority over statistics residing in one body.
Though the need to amalgamate the various statistical agencies across Australia's states was widely recognised, it was Carver's relationship with the other state Statisticians, and their trust in him, that finally allowed this amalgamation to take place.
The Statistics (Arrangements with States) Act (Cwlth) was given assent in May 1956. During the second reading speech, Sir Arthur Fadden (Treasurer) referred to discussions already taking place with Western Australia, South Australia and New South Wales. Subsequently, agreements authorised under the Act were made with all the states.
The legislation allowed for the provision of statistical services by the CBCS to state governments, in the same way that such needs had been provided by state bureaus in the past. However in practice the assimilation of the state offices into the CBCS led to major changes to state level statistics.
Relationship with the Treasury
With the move of Wilson to the Treasury, the relationship between the Bureau, Treasury and the Government began to change. Wilson took his economic acumen and a number of economists with him and proceeded to build up the economic policy skills within Treasury. The role of the Bureau changed from economic adviser to Treasury to providing statistics to Treasury's economic advisers.
While he remained in Treasury, Wilson supported the Bureau's economic expertise. However the relationship had permanently altered and, under successive Treasury heads, Treasury's economic capacity grew and the Bureau's role solidified as provider of statistics.
Foundations for the future
In 1959 the growing need for professional statisticians led to the introduction of a Statistical Cadetship Scheme. This initiative involved the selection of around 12 outstanding students who were brought to Canberra to undertake Honours Degrees, with majors in statistics, mathematics and economics. This scheme and its later companion, the Graduate Cadetship Scheme, were to produce many leaders, both in the Bureau and more broadly in the Australian Public Service.
Following their successful use for official statistics in the United States, the 1950s saw the emergence of probability based sample surveys, an important innovation for the Bureau's official statistical work. This allowed statistically valid surveys to be undertaken by the Bureau, at a lower cost than complete enumerations.
Released from the onerous necessity of conducting a census for every collection (apart from those based on administrative data), the Bureau could produce more statistics than it had before, so satisfying the increasing demands of Treasury and the Government. The development of surveys also facilitated the growth of social statistics from the late-1960s and through the 1970s, based on the Bureau's household survey program.
While this freed resources and allowed the Bureau to establish many new collections it did not necessarily mean an improvement in the service provided to State governments. A consequence of increased surveys was that while there was an increase in the range of available statistics, small area data became more difficult to obtain.
Under Keith Archer (Commonwealth Statistician 1961-1970) and following much research, the Bureau's first computer was installed in 1961. To properly exploit the possibilities this created, a large number of programmers were recruited from the United Kingdom. They were to form the basis of the Bureau's fledgling computing team and many remained with the Bureau pursuing their careers.
The Bureau was among the first Commonwealth agencies to acquire a computer, and the capacity of this computer was initially greater than the Bureau's need. As part of the deal, and to help justify the cost of the computer, the Bureau took on the processing of administrative records in many areas, such as health and trade. The side effect of this policy was that the Bureau was able to make use of the administrative by-product statistics thus produced. This responsibility was to remain with the Bureau for the next two decades.
The 1960s was an era of great change. Although legally accomplished by the end of the previous decade, the task of assimilating the various state offices and the CBCS into one organisation took many years. In practice it was probably not completed until the late-1970s when Roy Cameron (Australian Statistician 1977-1985) made special efforts to bring closure to this issue.
The decade saw a major push within the Bureau to integrate its economic data collections, both censuses and sample surveys. It had been increasingly apparent that there was a need to provide for users a range of statistics that were comparable, so that '... you could relate employment to production and wages, and ... you could ... relate overseas investment to these other categories of macroeconomic statistics, which is partly a matter of standardising the units in which they were collected and standardising the concepts' (Frank Horner, interviewed in 2000). The twin aims of the economic integration were to provide comprehensive, related, quality industry statistics and to provide data for use in compiling the national accounts. The model for the project was the United Nations System of National Accounts. Achievement of that goal took a lot of the energy of the Bureau for a number of years. As with many enormous and innovative undertakings, the initial results were disappointing. The first integrated economic census was run in 1969, but took far longer than envisaged to process. However in time the value of integration of economic data collections came to be fully recognised.
The introduction of household surveys was another major initiative, with the initial aim of providing comprehensive estimates of the Australian work force at quarterly intervals between population censuses. The survey estimates also supplemented the existing statistical series of employment (derived from employer surveys) and unemployment (derived from administrative data about recipients of unemployment benefits). Subsequently this became the basis for producing a much fuller range of social statistics.
A new beginning
In 1973, the Whitlam Labor Government established the Committee on Integration of Data Systems (known as the Crisp Committee after its chairman, LF Crisp). The Government had been concerned about recent discrepancies in statistics from various Commonwealth departments and the lack of statistical data on key areas of the economy, and believed that this could interfere with its reform agenda. As a result, the Committee undertook a wide ranging examination of Australia's statistical system.
Within the Treasury portfolio, the Bureau was reliant on Treasury for funding, even though the power to collect statistics was conferred on the Bureau by the Census and Statistics Act 1905 (Cwlth). The Commonwealth Statistician consequently had freedom to initiate new statistical collections, but he operated within the practical constraint that funding would be more assured if these collections were supported by Treasury.
The Crisp Committee reported in March 1974. It recommended the establishment of the Australian Bureau of Statistics as the central statistical authority with full statutory powers, administratively independent of any department and thereby perceived to be policy neutral. The Australian Statistician was to be a statutory appointee vested with the powers of a 'Permanent Head' under the Public Service Act (Crisp Report 1974). This led to the development of the Australian Bureau of Statistics Act 1975 (Cwlth) under the guidance of Jack O'Neill, the Commonwealth Statistician of the time.
Under the Australian Bureau of Statistics Act 1975 (Cwlth), the Australian Bureau of Statistics (ABS) was established with the role of central statistical authority for the Commonwealth Government and, by arrangements, for the governments of the states. It provides statistical services for those governments on a number of levels:
Jack O'Neill was appointed the first Australian Statistician in 1975 having been the Commonwealth Statistician since 1970.
The Australian Bureau of Statistics Act 1975 (Cwlth) also established the Australian Statistics Advisory Council. Its role is to advise the Minister and the Australian Statistician on the improvement, extension and coordination of statistical services provided for public purposes in Australia, and annual and longer-term priorities and programs of work that should be adopted in relation to major aspects of the provision of those statistical services. The council consists of a part-time Chairman, the Australian Statistician (ex-officio), and between 10 and 22 part-time members, including one nominee of each state Premier and the Chief Ministers of the two territories. Generally, agenda papers for council meetings are prepared by Bureau employees.
As Bill McLennan (Australian Statistician 1995-2000) said in 'The development of official statistics in Australia, and some possible future challenges', Year Book Australia 2001,
In the late-1970s Bureau senior management implemented a rolling long-term management system. Designed to force the incorporation of long-range strategic planning and thinking into Bureau decision making, it enabled the Bureau to foresee changes to its external environment. This has resulted in a greater ability to quickly respond to changing community statistical needs and changing government budgetary policy. The Bureau operates on a continually re-worked three-year forward work plan.
Meanwhile, the establishment of the ABS, and implementation of the legislation, further entrenched the shift away from a decentralised, state-based statistical system. Though this trend could be said to have started with the creation of the Bureau in 1905, and certainly solidified with the merging of the state statistical bureaus into the CBCS, the Australian Bureau of Statistics Act 1975 (Cwlth) was a major move towards centralisation of statistics in Australia. As such, it reflected the belief of key players in the Australian community, in the efficacy of a centralised system of statistics. At a more practical level, the legislation, by giving the Australian Statistician permanent head status, further raised the status of that position relative to the heads of state offices, and thus giving the Australian Statistician greater authority over the statistical system.
In 1979 the Australian Law Reform Commission released a report Privacy and the Census. This highlighted the need for a review of the terms of confidentiality in the statistics legislation.
Substantial amendments to the Census and Statistics Act 1905 (Cwlth) were passed in 1981, in two separate amendment acts. The first of these incorporated the recommendations of the Australian Law Reform Commission. The second amendment provided the opportunity to thoroughly re-work the act, incorporating the original act and subsequent amendments into a more coherent framework, using more modern language and terminology. The amendments legislated those powers, given to the Statistician in the Australian Bureau of Statistics Act 1975 (Cwlth), to determine the timing and method of statistics collection. They also obliged the Statistician to compile, analyse, publish and disseminate collected information, and made possible the release of data in unit record files.
The main purpose of the second amendment was to make possible, within the privacy constraints of the first amendment, the release of a wider range of information, including the release of unidentifiable microdata. There are many instances where release of data is appropriate either because no private individual details are thus exposed, or because the data in question is already within the public domain. However the legislation as it stood did not allow such releases. It was recognised that releases of data should be treated as exceptions to the privacy protections contained in the Act, and governed by very tight and specific safeguards, which might vary from one release to another. This level of detail was not considered appropriate within the legislation. The potential to release data was achieved by making provision within the Act for the Minister, in a written determination, to authorise the Statistician to make specific information releases. Determinations of this nature must be tabled in Parliament, and it remains the final decision of the Statistician, whether to release the particular information.
In the early to mid-1980s, under Roy Cameron, the Bureau subjected itself to rigorous external examination, in the form of the Joint Management Review (JMR), which was convened to examine the effectiveness of the top management structure of central office and the state offices, in terms of guiding the Bureau soundly through current and future challenges, properly utilising state and central office resources and adequately addressing client needs. Conducted by Touche Ross and the Public Service Board, the JMR found a number of key areas in which improvement was needed. These recommendations shaped subsequent management planning, and contributed to the process of integration and of modernising the Bureau and making it more outward looking.
In 1981 the Lynch Committee Review of Commonwealth Functions (known as 'the razor gang') released its report. The report recommended wide-sweeping cuts be made to the entire public sector, and in an operational sense this translated into significant budget reductions. In 1982 the Statistician made the decision that the Bureau would no longer be a processing agent and handed the coding, data capture and editing of administrative records back to the administering authority, some of which were state government authorities. Bureau resources were thus freed to be utilised elsewhere, for example in the burgeoning household surveys. The administrative authorities, however, faced with finding an alternative means of processing their records, were less appreciative, and the repercussions were long-reaching.
In 1985, prompted by the findings of the JMR, the Statistician implemented a version of matrix management. Under this policy, division heads within the Bureau were responsible to the Statistician for the work of their division, both within central office and throughout the state offices of the Bureau. State office heads meanwhile, were seen as responsible for ensuring that the state components of each division's work were carried out effectively, that the particular needs of their state were represented in Bureau decisions, that links with state clients were adequately supported and, as the major communicators with respondents, that data quality was maintained. The document outlining this strategy paved the way for a strong corporate focus that complemented and enhanced the effectiveness of the forward work plan.
Throughout much of its history the Bureau was run with each area producing its own statistics with little reference to the work of other areas. This is understandable in the light of the level of complexity involved in producing each area of statistics, and particularly developing new statistics and new methodologies. However, as the Bureau grew and became involved in a much wider range of statistics, it became necessary to forge a deeper relationship between these areas, both to avoid duplication of effort and to ensure a unity of purpose across the organisation. Emphasis has therefore been placed on building a corporate mentality at all levels within the Bureau.
With the new corporate identity came a questioning and reassessment of the purpose of the Bureau, and an acknowledgment that it was more than a factory for publications. Out of this process emerged the mission statement, the concept of corporate objectives and a commitment to statistical coordination and analysis. These were conveyed in the first ABS Corporate Plan which was developed under the guidance of Bill McLennan while he was Deputy Australian Statistician. Throughout this period the Bureau increasingly focused on efficiency, producing more while resources remained static.
The Bureau's first marketing plan was released in 1989. It followed a Government decision that part of the Bureau's budget should be funded through cost recovery. In producing this plan the Bureau demonstrated that it had come a long way from its statistical factory past. The plan outlined a major re-think in the way the Bureau regarded its products and its clients. The second marketing plan, which came out in 1992, maintained the emphasis on product and the importance of saleability, and out of this emerged the concept of clients. The third plan, in 1996, placed great emphasis on clients. Thus marketing was an important plank in the reinvention of the Bureau as an outwardly focused, forward looking agency.
Important in this process was the growth in the Bureau's analytical capacity. The influence of Ian Castles (Australian Statistician 1986-1994) was paramount. He strongly believed the ABS could add considerable value to its statistics by judicious use of analysis and analytical methods. At the same time, the Bureau started to devote more publication space to the interpretation of its statistics. The need for greater effort in this area had been highlighted by the mission statement in 1987, but the authority came from the Australian Bureau of Statistics Act 1975 (Cwlth). In 1995 a special Analysis Unit was established, recognising the potential of statistical methods and models for producing official statistics, improving methods or better understanding statistical relationships.
Throughout the 1980s and into the 1990s, Conferences of Statisticians continued to be held, despite the major governance changes that had occurred to Australian statistics throughout the previous 40 years. At the 1996 Conference, the Australian Statistician concluded that the Conference in its present form was not effective and that there must be better ways for the ABS to assess state and territory statistical needs and priorities. This was driven in large part by the reducing seniority of the representatives of the states. It was subsequently decided that more thorough utilisation of the Australian Statistics Advisory Council and greater involvement of Deputy Australian Statisticians in identifying state government requirements, would render the Conference of Statisticians unnecessary.
In 1992 the Bureau introduced a new approach in terms of data processing and utilising state office resources. Under this new system, National Project Centres, with responsibility for all data collection, processing, output (of standard products) and associated support and development activities for specific areas of statistics, were set up in state offices. This enabled the realisation of the advantages of concentrating data collection and processing operations, such as economies of scale and improved data quality, and the avoidance of data quality and inconsistency problems that can arise through decentralised data collection.
Throughout the 1990s there was an increasing emphasis on the use of administrative by-product data. This was influenced by Bill McLennan's sojourn as Director of the United Kingdom National Statistical Office. He noted the much more extensive use of administrative data in the United Kingdom statistical system. The relationship with the Australian Taxation Office, in particular, grew strong and a number of cooperative agreements enhanced the useability of taxation data to derive statistics.
In the new millennium, a number of important initiatives have been implemented with a view to the future. In 2002 the National Statistical Service was set up with the specific aim of better coordinating and utilising those statistical resources residing with other agencies. In 2003, the various fora by which communication was maintained between the state and central offices of the Bureau, were formalised into the State Statistical Forum, on the suggestion of the previous meeting of the Australian Statistics Advisory Council. This new approach involves state and territory advisory council representatives and Bureau Regional Directors meeting to discuss specific statistical matters relating to states and territories.
In 2002 the Bureau embarked on the Business Statistics Innovation Program. This is a three-year program which involves through the use of innovative technologies and methodologies, a major re-engineering to the way the Bureau conducts its business statistics processes. The program aims to result in improved provider relations and data quality, increased capacity to respond to emerging statistical needs, provision of a better National Statistical Service, enhanced opportunities for staff and significant budget savings. Similar initiatives have now commenced in the household survey program.
The last quarter century has been a period of great change. The result is a modern statistical bureau, with solid legislative underpinning, capable of meeting the challenges and needs of the information age.
These documents will be presented in a new window.